The New Default. Your hub for building smart, fast, and sustainable AI software
Table of Contents
Token efficiency in AI-assisted engineering means getting the most working, accepted software out of every token your team pays for. It doesn't mean spending the fewest tokens.
The difference matters now that vendors bill AI coding by use. GitHub Copilot began moving all plans to usage-based billing on June 1, 2026. GitHub now charges AI Credits for input, output and cached tokens at each model's API rates. For many engineering organizations, AI tooling went from a flat seat fee to a variable line item the CTO has to explain.
To manage it, a CTO first needs to know where AI token costs come from. Then come the metrics that show whether the spend pays off, and the levers that cut costs without slowing developers down.
Executive Summary
AI coding ROI depends less on how many tokens a team burns than on what each accepted change costs. That cost includes retries and review.
Most of that cost lands on verification. In a 2026 study of a multi-agent coding system, code review consumed 59.4% of all tokens. Bain found that when developers complete roughly 21% more tasks with AI, review time rises about 91%.
The solutions are prompt caching, selective context, model routing, platform-level budgets and shared, machine-readable context. Rewarding token consumption, or rationing it per developer, is not a good strategy.
Why Are AI Token Costs Becoming a CTO Problem in 2026?
AI token costs have become a CTO problem because of two reasons. Billing moved to usage, while agents multiplied the tokens per task.
Flat-rate coding assistants hid their compute cost. CODE Magazine compares that era to an all-you-can-eat buffet. You were never billed for redundant prompts or re-sent context. Now, agents and reasoning models turn one developer request into many model calls. Each call carries the context accumulated so far.
This makes cost per task unpredictable, and different between workflows. The costs can quickly skyrocket, even when they're not tied to any actual deliverables. It's a bit like the early days of cloud spending. Rapid adoption first, then FinOps and cost discipline.
What Is Tokenmaxxing, and Why Does It Hurt AI Coding ROI?
Tokenmaxxing is the practice of tracking and rewarding the highest consumers of AI tokens, often on internal leaderboards, to push developers toward AI. DORA traces the trend to early 2026 and calls raw token count a vanity metric.
A token count says nothing about results, and it's easy to game.
DORA cites reports of developers running agents on nonsense projects to stay above the company average.
The returns also diminish. DORA cites Adnan Masood: the teams with the biggest budgets often get only a marginal gain in delivery throughput, but pay about ten times the cost for it. Technical debt also grows when pressure for output lowers review standards.
DORA also argues it's too early to pour energy into hyper-efficiency. Teams still learning what AI does well need room to experiment. Token efficiency aims for value per task.
Where Do Tokens Go in AI-Assisted Software Development?
In one study that measured it, most tokens went into code review and into agents re-reading each other's context.
In January 2026, Concordia University researchers published "Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering". They ran 30 development tasks through the ChatDev multi-agent framework on GPT-5. The iterative code review stage consumed an average of 59.4% of all tokens. Input tokens made up 53.9% of usage, because agents kept re-reading what other agents passed them. The authors call the results preliminary, and they come from a single framework.
The human side looks the same. Bain's 2026 technology report found that developers complete roughly 21% more tasks with AI, while review time rises approximately 91%. Only code generation got cheaper, not verification.
Any saving on generation goes to waste if review doesn't scale with it. You must plan reviewer capacity and automated code review checks alongside AI adoption. Otherwise, the bottleneck moves to your most senior engineers.
Which Metrics Show Whether AI Coding Spend Pays Off?
The metrics that show whether AI coding spend pays off are a few cost and quality signals. Read them alongside the DORA delivery metrics you already track: deployment frequency, lead time, change failure rate and recovery time.
DORA itself recommends pairing those with efficiency and risk indicators, such as cost per accepted change and code rework rate. In DORA's view, you don't need a new headline metric.
Metric | What it tells you |
|---|---|
Cost per accepted change | The full cost of a merged pull request, including retries, failed agent runs and human fixes. A cheap call that needs three retries costs more than one that works the first time. |
Code rework rate | How much AI-assisted code the team reworks or reverts soon after merging. It catches the technical debt that cheap generation creates. |
Review time per merged PR | Whether verification keeps pace with generation. This is our suggestion, not an industry standard. |
Cache hit rate | How much of your repeated context the provider bills at the cached discount. |
Latency | Time to first token for interactive assistants; the slowest 5% of tasks (P95) for agents. |
To get these numbers, tag every model call with the team, the workflow stage (planning, execution, review, retry), the model and the token type.
Report the results per team, application or use case. According to DORA, company-wide aggregates are meaningless, and individual scores are harmful. Shopify, for example, replaced its AI leaderboard with a usage dashboard.
Why Is Token-to-Commit Ratio a Misleading AI Productivity Metric?
Token-to-commit ratio is misleading because it divides one easily inflated number by another. DORA lists commit frequency among the classic vanity metrics, alongside lines of code. IBM warns that activity metrics are easy to collect but rarely reflect business value.
The ratio rewards splitting work into tiny commits. However, it says nothing about whether the code survives review. If you want a ratio, compare a team's AI spend with its accepted changes instead.
Which Token Efficiency Levers Give Engineering Teams the Best ROI?
Start with levers that are cheap and leave quality untouched, such as caching, shared context and clear specs. Then move to the ones that need evaluation, such as model routing and context compression.
Judge every lever by the completed task costs. GitHub learned this when it tested RTK, a tool that shortens shell output before an agent reads it. Each response got shorter, but agents re-ran commands to recover what was missing. Ultimately, the whole task used more tokens and took longer. GitHub calls this the local metric trap.
Lever | Who owns it | What the evidence says | Main risk |
|---|---|---|---|
Prompt caching | Platform | 41–80% lower API cost and 13–31% faster time to first token on long agent tasks across OpenAI, Anthropic and Google | None to quality; any change in the prompt prefix breaks the cache |
Selective context (repo maps, on-demand tools, compressing only build and test noise) | Platform, tool vendor | GitHub's carefully tested changes to Copilot (compressing repetitive output, leaner file reads) saved about 3–6% each | Over-trimming makes agents fetch context again, which raises cost |
Plan with a frontier model, execute with cheaper ones | Platform, developers | Input prices between model tiers differ by up to 100× | Misrouted coding tasks fail; caches don't carry over between models |
Reasoning and output budgets per task type | Platform | 21.1% fewer tokens with accuracy held in one reasoning study | Uniform caps hurt hard tasks |
Hard limits: per-team budgets, loop and retry caps, circuit breakers | Platform, leaders | Caps the worst-case bill; Shopify uses circuit breakers to catch runaway agents | Tasks cut off before they finish |
Specs and machine-readable context (AGENTS.md, rules, skills) | Leaders, developers | Stripe merges about 1,300 AI-authored pull requests a week, each built from a spec and reviewed like any other change | Outdated docs raise the cost of every task |
Session habits: a new session per topic, undo instead of piling on corrections, cheap tests before expensive ones | Developers | Google's practitioner guidance; no figures | Needs enablement, not mandates |
Treat any percentage, including the ones above, as a hypothesis for your own workload.
A change that cut costs in Copilot code review raised them in Copilot's command-line tool, so GitHub didn't ship it there. The same applies to prompt and rules files. When GitHub halved one of Copilot's prompts, a live experiment caught a regression. Agents had started running one at a time instead of in parallel.
How Should a CTO Roll Out Token Efficiency Without Slowing Developers Down?
By instrumenting first, then fixing platform defaults, then fostering good habits. IBM frames it as three levers that only work together: developer practice, leadership policy and platform defaults.
Instrument and build an evaluation set from your own tasks. Public benchmarks rarely match your codebase. Include tests for prompt and rules-file behavior, so a shorter instruction can't silently remove something developers depend on.
Take the low-risk wins. Turn on prompt caching, keep shared context in version-controlled rules and skills, and set output limits per task type.
Move budgets and model choice into a shared AI platform. Developers shouldn't pick a model for every request, and leaders shouldn't approve routine spend by hand. Per-team budgets and circuit breakers should work automatically.
Spread habits through enablement. Shared skills, rules and training scale better than individual targets. DORA's advice is to treat AI adoption as a team activity, not an individual sport.
Budget AI as an operating cost with an owner. Model prices and capabilities change several times a year, so routing rules and cache layouts need someone to maintain them.
Match the effort to your volume. If your monthly AI bill is smaller than a week of an engineer's time, caching and output limits are probably enough. Teams still learning what AI does well should protect experimentation and rely on circuit breakers. Teams using AI at scale should optimize value per task.
What Can Go Wrong When You Optimize AI Coding Costs?
The main risk is saving tokens in a way that costs more somewhere else.
Quality regressions are one version of that. Trimming context or shortening prompts can remove information or behavior nobody had a test for. The cost shows up later as retries, rework and incidents.
Rationing backfires in a different way. IBM points out that pushing developers to use less AI tends to drive work into unmanaged tools, where you lose both visibility and control.
Review costs can also move out of sight. If you speed up generation without adding review capacity and automated checks, the cost moves to your senior engineers' calendars. There, it doesn't appear on the AI bill at all.
Model choice is also a data decision. Models and providers differ in data retention terms. Some top-tier models require retention periods that regulated teams in HealthTech or FinTech can't accept by default. Check where your traces and cached prompts go before you route work to a cheaper or newer model.
Key Takeaways
Measure cost per accepted change, read alongside your DORA metrics.
Code review drives the lion's share of AI coding costs, both in tokens and in hours.
A shorter prompt or tool output can make the whole task more expensive.
Put budgets and model choice in the platform, where they apply automatically.
Report token use by team and use case, avoid leaderboards because they reward waste.
What Does Sustainable AI Coding ROI Require From Engineering Leaders?
Sustainable AI coding ROI comes from treating tokens like any other metered resource. That means measuring value per accepted change, letting the platform enforce limits, and planning review capacity alongside generation.
With those in place, a team can respond to the next model price change by updating its configuration. If your AI tooling spend is growing faster than your delivery, Monterail's AI consulting team can review your AI-assisted development workflow. We'll show where the tokens and review hours go and help you decide which controls are worth it at your scale. Get in touch to start with a diagnostic.
Token Efficiency and AI Coding ROI FAQ
Maciej Korolik
Senior Frontend Developer and AI Expert at Monterail
Maciej is a Senior Frontend Developer and AI Expert at Monterail, specializing in React.js and Next.js. Passionate about AI-driven development, he leads AI initiatives by implementing advanced solutions, educating teams, and helping clients integrate AI technologies into their products. With hands-on experience in generative AI tools, Maciej bridges the gap between innovation and practical application in modern software development.
