Why your AI coding agent keeps reprocessing the same context
AI coding agents are not chatbots. One developer instruction can trigger dozens of sequential model calls as the agent inspects files, runs tests, reads failures, and decides what to do next. That loop creates a systems problem most builders overlook: how much of the context the model already processed can actually be reused?
What the production data shows
A study of sampled GitHub Copilot coding-agent traces from June 2026 analyzed roughly 13 million sessions, 3.2 million users, 761 million LLM calls, and 95 trillion tokens. The key finding: KV-cache reuse averaged around 90% within a single agent turn but dropped to about 55% across turn boundaries.
About 87% of LLM calls in the workload were agent-initiated, not user-initiated. One developer message generates a long chain of model invocations, which makes cache reuse a first-class concern rather than an infrastructure footnote.
Two events that tank cache reuse
Context compaction happens when a long-running session accumulates too much history and the agent summarizes it down. Useful for token management, but the study found compaction was associated with cache cold-start behavior. Changing the context structure means previously computed state may no longer be usable.
Model switching is worse. Even when the textual context is identical, cache state does not transfer across models. The study reported that after a model switch, only about 8% of context remained cached in the analyzed workload. Switching models mid-session to save a few cents on a simpler subtask can cost more in reprocessing than it saves.
Practical design rules
- Keep the prefix stable. System instructions, tool definitions, and repository configuration should not be regenerated or reordered between model calls unless the content actually changed.
- Treat compaction as a workload event, not a free optimization. Compact when retained context stops providing value, not on an arbitrary schedule. Preserve decisions, modified files, known failures, and remaining work. Drop raw logs and duplicate tool output.
- Avoid model switches without a strong reason. Better capability or a provider failover are legitimate reasons. A marginal cost difference mid-session probably is not.
- Separate persistent agent state from conversation history. Store task state explicitly so compaction does not have to reconstruct the entire project context from scratch.
- Filter large tool outputs. Give the model what it needs for its next decision, not the full compiler log.
What to measure
If you are building agent infrastructure, cache hit rate, prefix length, compaction frequency, model switch count, and tool calls per turn should all be tracked together. Token reduction alone is not a useful signal. A strategy that produces a smaller context but forces more repository rediscovery may cost more overall than one that retains state and reuses it.
The full article includes C# code examples for a session-aware model router and an explicit agent state record. Worth reading if you are designing agent orchestration at any scale.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
