AI agent refactors 189 files in a 717k-line codebase with no human code review
A new arXiv paper documents something most senior developers would call impossible: an AI coding agent dismantling a core architectural invariant across a 717,725-line production TypeScript codebase, touching 189 files, with no human reviewing the generated code before it shipped.
The Task
The target system spanned 3,648 files. The invariant being removed was the guarantee that a UI panel stays open for the lifetime of an AI request. The new behavior required a streaming generation to survive its panel closing and reattach, on reopening, to the same live stream with no data loss or duplication. The author assessed incremental refactoring as effectively infeasible for a change of this scope.
The Protocol
The agent did not just write code. It followed a specification-first loop:
- The agent wrote a formal specification.
- 14 refinement cycles audited that specification against the source code.
- Atomic implementation with a compile and test feedback loop.
- 17 verification cycles audited the generated code against the frozen specification.
Across 31 total audit passes, the agent caught and corrected 201 defects before any human executed the program. The convergence criterion was strict: two consecutive verification passes returning zero findings.
The Numbers
- Codebase size: 717,725 lines across 3,648 files
- Files changed: 189 (31 net new), plus an extraction phase totaling 288 files
- Insertions / deletions: 34,770 inserted, 16,422 deleted
- Defects corrected before human execution: 201
- Elapsed time: 3 days
- Cost: USD 2,430
After the change landed, the software behaved as specified across the first session and roughly thirty later sessions. No bugs were observed.
Why This Is Worth Tracking
The full specification and raw session logs, over 1,500 pages written in French, are published alongside the paper. That means the process can be inspected directly or fed into another language model for consistency checking. Most AI coding case studies give you the result. This one gives you the receipts.
For solo developers and small teams sitting on large codebases, the relevant question is whether this protocol scales to their stack. The paper does not generalize beyond this single case study, but the audit loop structure is replicable in principle.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
