7 defensive layers that keep AI-generated code from going sideways
The standard objection to AI coding agents goes like this: sure, you ship more code, but doesn’t quality drop? The honest answer is yes, it can. But only if you skip the defensive work that keeps any codebase stable.
Developer Iouri Khramtsov has been running AI agents on a production team and reports a 2-3x increase in output without a corresponding rise in bugs. The setup he uses has seven layers. Most of them are familiar quality practices, now made cheaper to run with AI. A few are new.
️ Layer 1: Fix the requirements before you write a line
Khramtsov switched to spec-driven development and noticed an immediate drop in freshly shipped bugs. Before the switch, teams he worked on spent up to a third of total feature effort on post-development polishing: finding and fixing bugs that slipped through because edge cases weren’t thought through, a developer was tired, or a designer didn’t fully spec a scenario.
The fix is a specific step in the spec process: have the AI review the requirements or tech design and hunt for gaps, edge cases, and unexpected interactions with existing code. Unlike a tired developer, the AI doesn’t give up early. The one watch-out is that it can invent problems that aren’t there, so the proposed edits to requirements need a careful human review before anything gets locked in.
Layer 2: Unit tests at above 95% coverage
Coding agents make test-driven development (TDD) cheap enough that there’s no reasonable excuse to skip it. The pattern that works:
- Instruct the agent to think through test scenarios based on the requirements.
- Write the test cases first.
- Write the implementation.
- Run the implementation against the test cases and fix anything that breaks.
- Backfill any remaining coverage gaps, keeping the requirements in mind throughout.
The key guard here: don’t let the agent write passing tests for bugs it just introduced. Tests come from requirements, not from the implementation.
Layer 3: Manual testing still has no substitute
A human, whether that’s you, QA, a PM, or someone else, still needs to actually use the feature. Going through edge cases manually and seeing whether things behave as expected catches a different class of problem than automated tests do.
This is also the layer that’s seen the smallest productivity gain from AI so far. Khramtsov cites it as the main reason output has increased 2-3x rather than something closer to 10x. Some automation opportunities may exist here that haven’t been explored yet.
⚙️ Layer 4: Extensive automated end-to-end tests
End-to-end (E2E) tests are the most important tests in the codebase because they verify that new changes haven’t broken existing functionality from the end user’s perspective. Ideally they run on pull requests, in staging environments, and in production after every deployment.
AI can write E2E tests effectively, but it needs access to debugging tools to do it well: a browser tool or MCP access to logs to trace failures. One important framing: E2E tests are not a substitute for manual testing. They’re a rough check that nothing critical broke, not a full coverage signal.
Layer 5: Dedicated AI code quality passes
Coding agents aren’t reliable at following complex instructions buried in AGENTS.md or CLAUDE.md. They perform better when given a separate, focused pass to find and fix specific categories of issues. Useful passes include:
- Security issues
- Overcomplicated or duplicated code
- Compliance with naming, file organization, or formatting rules
- A general logic review pass
- Overly long comments written in AI prose instead of plain English
- Any other specific patterns you want to catch
Added to the planning or implementation workflow, these passes cost roughly 5 to 15 minutes of automated time with no additional human attention required.
Layer 6: PR reviews by both humans and AI
For minor tweaks and simple bug fixes, Khramtsov is becoming comfortable making human reviews optional, as long as the other six layers are in place. For complex changes, human review is still necessary. He reports regularly catching big-picture mistakes, missed adverse interactions with other features, overcomplicated implementations, and odd word choices like “mint” instead of “generate” or “stamp” instead of “set.”
On his current team, both Claude and Cursor review every pull request. Notably, each tool finds different problems. Additional custom review passes for security, efficiency, or cross-repo interactions are possible, though AI reviewers can be overly nitpicky. A separate agent pass to prune meaningless AI-generated PR comments is worth adding.
Layer 7: Monitoring and alerting in production
Once the code ships, the minimum viable setup is someone periodically reviewing logs, watching user session recordings in a tool like Fullstory, or checking dashboards that track error rates and latencies.
A dedicated error tracking service like Sentry or GCP’s Error Reporting that detects and deduplicates errors is better. The strongest setup is using Claude, Cursor, or a similar agent to auto-diagnose production errors, identify the root cause, and open a pull request with a proposed fix.
The bottom line
The core argument is simple: AI agents make it cheaper to add more and deeper quality checks than were practical before. More tests, more review passes, faster production diagnosis. The increased output doesn’t have to come at the cost of reliability.
With all seven layers in place, Khramtsov reports it’s possible to double delivery speed while keeping bugs under control, or even reducing them.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
