Cut AI coding costs 20-40% without budget caps or slowdowns
Engineering leaders keep reaching for budget caps when AI coding costs spike. It’s the wrong tool. Coinbase found that 91% of its engineers never hit their caps. The caps frustrated the majority and interrupted the few who were hitting them, who were often the most productive engineers on the team.
Larridin compiled a playbook from engineering leaders at Uber, Coinbase, Fanatics, and others. Their customers are seeing 20 to 40% cost savings within days and weeks, without touching model quality or slowing anyone down. Here is what actually works.
Step 1: Fix your prompt cache TTL
Most engineering leaders assume AI coding spend goes to output tokens, the code the model writes. That is wrong. A significant share goes to re-reading the full conversation context on every turn, because coding agents remember nothing between requests.
Prompt caching stores the unchanged prefix and re-reads it at a discount: 10% of the input price on most Anthropic models. The cache-read rate is the most important number on the price sheet. When Anthropic cut one model’s cache-read rate from $1.00 to $0.25 per million tokens, the same task fell 42% in cost with no change to the model or the work.
The five-minute expiry problem
Anthropic’s default cache lifetime is five minutes, reset on each hit. A ten-minute coffee break kills it. So does a code review or a CI pipeline run. The next turn re-reads everything at full input price. One measured rebuild of a 61,000-token context cost $1.14. Uber lists expired-cache resumes among its top waste patterns.
The fix: set it to one hour
A one-hour cache lifetime costs more to write (2x vs. 1.25x), but reads cost the same. It pays off in any session with gaps longer than five minutes, which is most of them. A cache warmer on a five-minute window fires 12 times an hour at roughly the cost of one full rewrite. The one-hour cache beats it anyway. Uber runs one hour for interactive sessions and five minutes for subagents.
The catch: Claude Code requests one hour automatically on a subscription, but defaults to five minutes on an API key, Bedrock, or Vertex, which is how most enterprises run it. Setting promptCacheTtl: 1h and subagentPromptCacheTtl: 5m fixes this. Push them org-wide with managed settings. Larridin estimates these two settings alone cut an enterprise Claude bill by 10 to 15%.
️ Step 2: Put a gateway in front of everything
Cache policy cannot depend on every engineer remembering a config setting. An AI gateway sits between your tools and your providers, sees every request, and enforces the cache lifetime on anything that arrives without it. It also tracks spend per person and per tool, and handles routing.
Databricks reports that routing each task to the cheapest capable model cuts average cost by more than 30% at near-equal quality. The gateway is the single infrastructure piece that makes all the other levers manageable at scale.
Step 3: Change the defaults, not the choices
Larridin worked with a customer to start all tasks on GLM 5.3 instead of Opus by default. The result was an immediate 20% cost reduction. Engineers kept full ability to switch models when they needed to. Only the default changed. You can enforce this at the gateway layer without touching individual developer setups.
Also watch these cache-breaking patterns:
- Switching models or effort levels mid-task discards the cache entirely.
- Tool definitions sit at the front of the prompt. A tool server connecting or disconnecting invalidates everything behind it.
- Compaction resets the conversation cache by design, and costs most when the cache has already expired.
Step 4: Show engineers what they spend
Engineers fix numbers they can see. Uber put a live cost counter in the terminal, set alerts at 50%, 80%, and 100% of expected spend, and built a dashboard that flags 16 waste patterns, each with a dollar cost and a fix. Databricks escalates gradually: warning, then an approval gate, then a cheaper model. Suspension is rare and starts a conversation rather than ending one.
Simply surfacing cost data inside Claude Code and Codex made engineers more aware, and they started making smarter choices on their own. Coinbase raised its cache hit rate from 5% to 60% and credits that shift, combined with routing, for halving its overall bill.
The framing matters here. High spenders are often the highest producers. Make waste visible, not usage frightening.
Step 5: Measure cost per unit of shipped work
Steps one through four reduce spend. None of them asks what the tokens actually produced. That measurement is what closes the loop.
The number that matters is cost per unit of durable output: code that ships, passes review, and does not come back as incidents or rework. Larridin’s benchmark across more than 100 teams puts median AI coding spend at $213 per engineer per week, with a 10x spread across that population.
Spend alone predicts nothing. According to Larridin’s data, AI-native engineers kept converting dollars to output at $1,300 a week. Partial adopters saw returns halve past $600 a week. Low AI engineers stayed flat across a 20x spend range. Same tools, same prices. The difference is skill and fluency, not settings.
Without this measurement, a high spender is either your best engineer or your most expensive mistake, and you cannot tell which. With it, you know where budget converts to shipped work and where the constraint is something no config change can fix.
Common pitfalls
- Relying on budget caps: 91% of engineers never hit them. They punish the few for the spending habits of no one.
- Five-minute cache on enterprise API keys: This is the default. It is almost certainly costing you money right now.
- No gateway: Without a central routing layer, cache policy and model defaults depend on individual developer discipline, which does not scale.
- Tracking spend without tracking output: Cost data without velocity data tells you how much you spent, not whether it was worth it.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
