How GitHub Copilot cut AI coding costs without breaking anything
The GitHub Copilot team published a detailed breakdown of four changes that reduced AI coding agent costs without degrading task quality. The post is written by Erik Kristensen and covers real A/B experiment results, not benchmarks run in a vacuum.
The core argument: optimizing individual tool calls for token count is the wrong goal. The right goal is optimizing the completed task from start to finish.
️ The four changes shipped
1. Selective output compression. Install, build, test, and lint output gets compressed because it’s repetitive noise. Source-like output, git diff, cat, and arbitrary script results are returned unchanged. Search results are reorganized without dropping any matches. Early versions were too aggressive and caused agents to reopen original output or rerun commands. The shipped version is conservative because that’s what the evaluations supported, not as a design principle.
2. Removed line-number prefixes from file reads. The view tool previously prefixed every line with a number. Current file-editing tools match surrounding code and don’t use those numbers. Removing the prefixes caused model-inference cost to fall by roughly 5% in offline benchmarks. The online experiment reduced average daily model-inference cost per user by about 3%, with no material regression in quality or satisfaction metrics.
3. Compressed the task tool prompt by roughly half. A meta-prompting loop, where Copilot iteratively rewrote its own prompt, cut about 1,300 task-tool prompt tokens per turn. The first online experiment hit a regression: the rewritten prompt had turned cautious parallelism guidance into a hard scheduling policy, forcing independent agents to run sequentially. The team stopped the experiment, wrote a regression test for the exposed behavior, and fixed it with one sentence: Independent agents can run in parallel; consider side effects. The shipped version delivers approximately 1.8% fewer total prompt tokens per session and 2.9% lower normalized cost per active hour.
4. Batched background completion results. When a shell command or sub-agent finishes in the background, Copilot previously sent a notification without the result, forcing an extra retrieval turn. Now the harness batches eligible completions and delivers results directly. What previously required four model calls for two background results now takes one. This reduced average token-related usage, measured in AI Credits, by about 2.3%.
The lesson about RTK
The team also evaluated RTK (Rust Token Killer), a utility that shortens shell output before an agent reads it. In their harness and benchmark configuration, RTK shortened individual responses but caused the model to reopen original output or rerun commands when the omitted text mattered. Individual tool responses got shorter. End-to-end task cost went up. Fewer tokens locally, more tokens globally.
Five takeaways for anyone building agentic workflows
- Optimize the completed task, not the individual tool call.
- Eliminate model turns the harness can handle deterministically instead.
- Compress by what the output represents: preserve exact source content, prefer lossless transformations.
- Prompt rewrites can remove behaviors silently. Test intended behavior before and after.
- Results are local to the workload. Re-evaluate in every product surface where a change ships.
None of these changes made the model smarter. They removed work the model never needed to do in the first place.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
