AI agent refactored 189 files in a 717k-line codebase for $2,430
Joel Abenhaim published a fully instrumented case study of an AI coding agent completing a large-scale architectural refactoring that he assessed as effectively infeasible through incremental work. The kind of change that would normally mean a rewrite.
The Codebase and the Problem
The system: a 717,725-line production TypeScript application across 3,648 files. The task: dismantle a core lifetime invariant guaranteeing that a UI panel stays open for the duration of an AI request. The target behavior was that a streaming generation survives panel closure and can reattach, on reopening, to the same live stream with no loss or duplication. No test oracle existed to validate that target behavior before work began.
The Protocol
Abenhaim used a specification-first approach with no human review of the generated code:
- The agent wrote a formal specification
- 14 refinement cycles audited that specification against the source code
- Atomic implementation with a compile and test feedback loop
- 17 verification cycles auditing the code against the frozen specification
The convergence criterion was empirical: two consecutive verification passes returning zero findings. Across 31 total audit passes, the agent corrected 201 defects before any human executed the program.
The Output
The change touched 189 files, with 31 new files created. Combined with an extraction phase, the two commits total 288 files, 34,770 insertions, and 16,422 deletions. Elapsed time: three days. Cost: USD 2,430.
Across the first session and roughly thirty later sessions, the software behaved as specified. No bug was observed.
The Evidence
The full specification and raw session logs, over 1,500 pages written in French, are published alongside the paper. Abenhaim notes they can be submitted directly to a language model for consistency checking.
For solo developers and small teams facing deep architectural debt, the protocol described here is worth reading carefully. The specification-first, audit-then-implement structure is the specific mechanism that kept the agent from drifting, and it is the detail most likely to transfer to your own use cases.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
