How to control coding agent costs with intelligent model routing

Intelligent model routing for coding agents sends each message in an agent session to the best model and reasoning level to maximize accuracy and minimize cost.

When done correctly, intelligent model routing cuts inference cost by 30%+ without lowering quality. On the other hand, when done incorrectly, naive model routing can degrade quality and increase costs.

This guide is written to help you approach model routing the right way. We’ve written this primarily for engineering and finance leaders looking for technical solutions to the coding agent cost problem that preserve both output quality and developer experience.

Why coding agent bills are so high

Across enterprise deployments, Anthropic now reports an average of $250 per developer per month on Claude Code, with the 90th percentile spending up to three times that. In our own work with enterprises, many are well past $1000 per developer per month, and the number keeps climbing.

The increase in coding agent inference spend is driven by three deeper trends: model prices continue rising, more capable models are enabling longer sessions with more parallel agent work, and enterprise adoption is still growing month over month.

Behind the growing bills is a specific set of factors that drive the majority of coding agent spend:

1. Defaulting to the most powerful model for every task
2. Using small models for tasks they can’t solve efficiently or correctly
3. Misconfiguring the reasoning effort for a given task
4. High subagent utilization, fan-out, and dynamic workflows
5. KV cache misses and expirations

Understanding how these factors relate to each other is critical to building an effective cost management solution for coding agents. Model routing can help us optimize points 1-4. However, it’s important that model routing solutions also take into account point 5, otherwise they will almost certainly be ineffective or even counterproductive.

Why complexity and semantic classifiers don’t work

The simplest version of a model router relies on either a complexity classifier or a semantic classifier that sorts prompts into various buckets and assigns model targets to each bucket. Unfortunately, this approach fails to take into account various dimensions that affect routing outcomes in non trivial ways.

Data > heuristics

While classifier-based routers can be fully data-driven, they often rely on manual heuristics to route prompts to specific model assignments. In many ways this simply shifts the burden of manual model selection away from the user and to the opaque opinions of the router designer who is far from the front line. It also requires constant updates to the classifier architecture as the salient classes shift with the model landscape.

Managing the KV cache

KV caching is a central cost lever in long agent sessions. The KV cache is the mechanism that allows providers to re-ingest previous messages in the session at a much lower cost, typically at a 90% discount. However, these savings are constrained to a single model. If a developer is deep into a long session and a router sends the next turn to a "cheaper" model, you may still end up paying more money because the new model has to rewrite the entire cache. A router that is not cache-aware will typically end up costing you more money in agentic settings for this reason.

Long horizons

The consequences of a model choice are not bound to a single turn. Instead, they shape all future turns with respect to both cost and quality. A mid-session switch can change the probability of success on the next steps, and in some cases it can cause downstream issues or cache lock-in on a specific model that outweighs the savings it seemed to initially offer. Routing must be considered across the entire trajectory, not simply on the current call. Classifier-based routers, by taking into account only the state of the current turn, fail this condition.

How to route in long-horizon coding agent settings

Coding agent routing is a long-horizon optimization problem. The objective is to minimize cost and maximize quality across the whole trajectory, rather than optimizing for local optimums by finding the cheapest model that can successfully complete each turn. Sometimes that means spending more on stronger models for specific turns and keeping the cache warm. Routing also needs to be considered at multiple levels of agentic orchestration, both within the main session and within sub-agents.

At Not Diamond, we consider model and reasoning effort recommendations with respect to the KV cache state based on semantic and non-semantic payload features, and we optimize for sparse rewards over the course of the trajectory such as implicit feedback signals and session outcomes.

In many ways, the problem resembles familiar problems in multi-agent or robotics settings in which you have an environment (the coding agent session), an action space (a set of models and reasoning efforts to choose from), and sparse rewards over long horizons. At each step we must choose the optimal model and reasoning effort recommendation to maximize future rewards and minimize future costs.

In contrast to simpler classification-based strategies, we have found that only a more sophisticated approach of this kind can handle long-running agentic workloads reliably and scalably.

How do you evaluate routing for coding agents?

Whether you are deciding to buy or build model routing for coding agents, these are the questions you should be asking of any solution:

• Are model recommendations based on static rules, the semantic contents of each request, or the full trajectory of the agent session?
• Is the router cache-aware?
• Is the model pool vendor-neutral, or is it biased or constrained to specific providers?
• How much latency and cost does the router itself add?
• Does the solution maintain privacy and security or does it require you to send sensitive data over the wire?
• What impact does the solution have on developer productivity?
• Can the solution reliably deliver at least 10-20% cost savings without impacting quality?

For more information on tying coding agent spend and savings initiatives to impact, see our resource on how to measure ROI in coding agents.

Frequently asked questions

• Why is my coding agent bill unpredictable? When new models become even just a little bit better, they can run much longer autonomously and in parallel. This means that subtle variations in the model landscape can lead to major changes in your inference spend. Model routing helps ensure you are not overpaying for the entire course of additional workload volume.
• Can I just impose spend limits or restrict access to specific models? Spend caps and model limitations can offer some cost containment, but they also negatively impact the productivity you are paying for in the first place. Low spend limits can psychologically constrain developers to avoid using coding agents unless truly necessary, and restricting access to new powerful models can result in slower adoption of new capability unlocks. If you implement this, approach it more like a safety mechanism to prevent unintended runaway spend, or gate it to specific groups so you don’t hold back your best engineers.
• Is routing different from a gateway? AI gateways give you access to various models. Intelligent model routing automatically recommends the best model at the lowest cost for each request. Routers and gateways are easy to confuse. Many products marketed as routers are simply gateways doing rules-based routing. It’s important to distinguish them however because they perform very different objectives and also require very different capabilities to build, deploy, and maintain. For more information, see our guide on Model Routing vs. Gateways.
• Do I need a router if I only use one provider? Even within one provider, routing across model sizes and reasoning effort levels can deliver significant cost-savings. Multi-provider routing adds more gains, particularly with open-source models in the mix, but single-provider routing is a reasonable starting point.

If coding agent spend is a growing concern, we can help. Book a demo of Not Diamond's intelligent model routing for coding agents.