Augment Code’s semantic retrieval beat Claude Code by 33% on tokens
There are two competing philosophies in AI coding harness design right now. Anthropic’s Claude Code runs a lean harness and assembles context on the fly, mostly through grep-based search. Augment Code pre-indexes your entire repository using embeddings, a retrieval model, and a vector database, then pulls conceptually relevant code in sub-milliseconds when the agent needs it.
In an interview with Ars Technica, Augment Code VP of Engineering Vinay Perneti made the case for the semantic approach and responded directly to Anthropic’s position that structured context retrieval produces no measurable eval gains.
Where semantic retrieval has the clearest advantage
Perneti’s argument rests on one key distinction: public versus private codebases. On public, open source repos where most benchmarks run, frontier models have effectively memorized the code. They already know where to look, so grep-based search gets to an outcome quickly.
Private repos are different. The model has never seen the codebase, so the agent spends far more iterations exploring before it can execute. Pre-indexed semantic understanding collapses that exploration loop.
The benchmark number worth noting
Augment Code ran Terminal-Bench using the same underlying model as Claude Code. The reported result: similar accuracy, but 33 percent better token efficiency. That gap matters when your team is watching API costs on production agent workflows.
Perneti’s explanation for why Anthropic doesn’t see the same gains: not all retrieval systems are equal. Augment spent roughly 18 months before its 2022 founding researching embedding and retrieval models specifically for large codebases. A generic RAG implementation bolted onto an agent is not the same thing as a purpose-built context engine, he said.
The intelligence-plus-context framing
On the question of whether fast-improving models make harness design irrelevant, Perneti pushed back with a two-variable frame: outcomes require both intelligence and context. Models are getting more intelligent exponentially. But intelligence alone doesn’t give the model knowledge of your private repo. Context is the other variable, and how efficiently you deliver it is an engineering problem.
He also flagged a forward-looking cost argument: as open-weight models improve, teams may route well-specified, routine coding tasks to cheaper or locally hosted models while reserving frontier models for harder problems. A harness that can manage that routing efficiently will matter more as token budgets tighten.
On tech debt and trust
Perneti acknowledged the tech debt concern directly. Augment’s own internal teams found that agents duplicate code at a notable rate. Their response is dedicated sprints focused on reducing that debt, using agents to execute against a spec that defines what debt reduction looks like. He also said agents are currently not good at writing specs, which means humans still need to guide that part of the workflow before handing execution off.
The broader operator takeaway: if your team is working primarily in large private codebases and watching token costs, the grep-versus-semantic debate is worth your attention. The two tools are not optimizing for the same thing, and the right choice depends on what your codebase looks like and what your budget ceiling is.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
