Claude API costs $15-30 per long build session. Sensitive client context shouldn’t leave the machine. Some tasks (summarization, formatting, light reasoning) don’t need a frontier model. This is the local drop-in for those cases.
The Problem
The agent harness routes everything to Claude API by default. There’s no local LLM path. That means every task, regardless of complexity, runs at frontier model pricing. And sensitive client prompts leave the machine.
What It Is
An Ollama adapter that sits between the agent harness and the model layer. Task routing rules in a config file determine which tasks use the local LLM (LLaMA 3, Mistral, or Phi-3) vs. which fall back to Claude. On error, the adapter falls back automatically. Local conversation logs write to Obsidian.
Features
- Ollama setup with 3 model options: LLaMA 3, Mistral, Phi-3
- Per-task model selector driven by a YAML config
- Automatic Claude fallback on errors
- Performance benchmarking script to compare local vs. API
- Local conversation storage in Obsidian
Stack
| Tool | Role |
|---|---|
| Ollama | Local LLM runtime (CPU or GPU) |
| LLaMA 3 / Mistral / Phi-3 | Model options |
| TypeScript / Python | Adapter layer |
| Windows 11 | Host machine |
Cost Case
Internal tooling. The value is measured in API cost reduction and data privacy, not revenue.