We Made Our AI Coding Agent Judge Its Own Cost

If you’ve spent time working with agentic coding tools, you know the reflex: when in doubt, reach for the best model, turn the reasoning dial up, and let it think as hard as it wants. It feels safe. It’s also how you quietly burn through a token budget on tasks that never needed the expensive tool in the first place, a routine file edit, a grep sweep, a scaffolding pass that a mid-tier model handles in one pass at a fraction of the cost.

The Habit Nobody Notices They’ve Formed

The insidious part isn’t that this is a one-time mistake you catch and fix. It’s a standing configuration. You set your model and effort level once, at the start of a session, and every prompt in that session inherits the same setting, the trivial ones and the genuinely hard ones alike. Nobody decides badly on purpose. The system simply has no mechanism for noticing the mismatch, so the waste compounds quietly, one session at a time, until the bill makes it visible.

We’ve written before about the scaffolding we build around AI coding agents, guardrails that catch drift before it becomes a commit. This is the same instinct pointed at a narrower target: not correctness, but cost.

Why We Didn’t Build a Router

The obvious fix is a classifier: something that inspects each incoming task and routes it to the right model tier automatically. We considered it, and we didn’t build it, for the same reason we’re cautious about any system that makes a resource decision on your behalf without being able to explain itself.

A hardcoded router has to get the classification right, silently, every time, or it fails in a way you don’t notice until the bill arrives, or worse, until it under-provisions a task that actually needed the harder reasoning and ships something wrong. It also has to be maintained forever as task shapes drift. That’s a lot of infrastructure in service of a decision an experienced engineer already makes correctly, instinctively, several times a day: does this need the expensive tool, or the fast one?

So instead of building a system that decides for the agent, we built one that hands the agent the facts and asks it to decide as itself.

What the Hook Actually Does

It’s small. Once per coding session, before the agent acts on the first prompt, a lightweight script checks which model and effort level are currently configured and injects one instruction: judge whether the task in front of you is mechanical, a routine edit, an inventory sweep, boilerplate scaffolding, or genuinely hard, architecture work, adversarial review, debugging that requires real reasoning depth. If the configured setup looks mismatched, the clearest case being a top-tier model at maximum effort pointed at a five-line fix, the agent stops and asks, out loud, with its own one-line reasoning, before it spends a single token on the actual work.

If the configuration already looks reasonable, it says nothing and proceeds. The interruption is the exception, not the default behavior.

We also log the mechanical facts, which model and effort were configured when each session started, to a running ledger, purely as a shape-of-usage record. Not to grade the agent’s calls after the fact yet, just to know, over time, whether the pattern we’re seeing matches the pattern we assume we’re seeing. That’s a discipline worth having on its own, independent of whether the hook exists.

The Part Worth Generalizing

The mechanism itself is small, well under a hundred lines of shell script. What we think is worth sharing isn’t the code, it’s the design choice underneath it: when a system has to make a judgment call that a competent human would make differently depending on context, don’t hardcode the judgment. Supply the facts, state the heuristic, and let the entity closest to the actual task make the call, with its reasoning shown and a human able to override it in the moment.

That’s the same instinct behind everything else we build. We treat AI as a governance partner, not just a generator, because generative AI produces enormous volumes of creative output with none of the accountability that governed systems take for granted: no record of what decided what, no way to audit a judgment call after the fact. We built our product to make AI-generated creative work explain its own provenance. It turns out the same discipline is worth pointing inward, at the tools we use to build the product itself. If we expect our customers’ AI-generated assets to account for themselves, our own AI agents should meet that bar too.

What’s Still Unsolved

We’re being honest that this is a first pass. The ledger currently records what was configured, not whether the agent’s judgment call was actually correct. We don’t yet have a feedback loop that distinguishes “the agent recommended switching and the user agreed” from “the agent recommended switching and got overruled.” Without that, we can observe the pattern of usage, but we can’t yet tell whether the self-judgment mechanism is well-calibrated or just confidently wrong in a way we haven’t caught. That is the next thing on the list, not a solved problem we’re presenting as one.

Key Takeaways

• Default-to-maximum-effort is a silent, standing cost, a per-session configuration rather than a per-task decision, so waste compounds quietly.
• A hardcoded classifier isn’t automatically the safer choice. It just moves the failure mode somewhere less visible.
• Giving an agent the facts and a heuristic, and letting it reason out loud before acting, beats both always-ask and silently-decide. The interruption only fires on a genuine mismatch.
• The same governance instinct that should shape how you handle AI-generated output should shape how you run your own AI tooling.