Routing work to local models, so the expensive one only does expensive work

Watch what actually happens in a long working session. A large share of it is summarising a file, describing an image, reformatting some data, or explaining a function that is already clear. None of that needs the most capable model available. All of it is being billed as though it does.

Routing is the practice of deciding, per kind of task, which model should handle it — and then making that decision automatic so you are not thinking about it during the work.

The gain is not marginal. Moving the bulk categories off the expensive path typically changes the cost of a session by more than half, with no visible drop in the quality of the output that matters.

• Planning and architecture — the frontier model. This is where capability translates directly into a better outcome, and where a cheap answer costs you a week later.
• Orchestration and routine edits — a mid-tier model. Fast, cheap enough to run constantly, good enough for work with a clear specification.
• Code explanation and local refactors — a local coding model. It has the file in front of it and the task is well defined.
• Image and video description — a local vision model. Describing a frame is exactly the kind of bounded task a small model does well.
• Bulk, non-critical processing — whatever is cheapest that can do it. Summarising a hundred files does not need judgement, it needs throughput.

Run a local server for text and code work, and a second one for vision. They have different memory profiles and you do not want them competing for the same allocation while you are working.

# vision: describe frames, read screenshots
ollama run qwen2.5vl

# code: explain, refactor, summarise files
# LM Studio, server mode, exposed to the agent as a tool

Expose each as a tool the agent can call. The point is that routing happens inside the session automatically — the agent reaches for the local model because it is there and appropriate, not because you remembered to ask.

The instinct is to route by how big the task looks. That is the wrong axis. A one-line architectural decision can cost you a fortnight; a thousand-line summary usually costs you nothing if it is merely adequate.

The better question: if this answer is mediocre, what happens?

• Nothing, or I notice immediately → local model.
• I find out in an hour and redo it → mid tier.
• I find out in a week and it is expensive → frontier.

This also tells you when to break your own rule. A task that is normally clerical becomes a frontier task the moment its output is going to be built on without review.

Anything touching security. Authentication, input handling, credentials. The failure mode is not a bad answer, it is a bad answer that looks fine.

The first pass on an unfamiliar codebase. Understanding structure is exactly the capability you are paying for. Once the map exists, the work inside it can be routed down.

Anything you will not read. If the output goes straight into the project without you checking it, it needs the model you trust most. Routing assumes a human notices when something is off; where that assumption fails, the saving is imaginary.

Track two things for a week: what a session costs, and how often you had to redo something because a cheap model handled it. If the second number is near zero, route more aggressively. If it is climbing, you have pushed a category down that did not belong there.

The equilibrium is specific to how you work, which is why copying someone else's routing table only gets you started. The categories are general; the boundaries are yours.

What is universal is the direction of travel: almost everyone starts by sending everything to the most capable model, and almost everyone is surprised by how much of it did not need to go there.