Wrong AI reviewer pair drops Claude’s pass rate from 91% to 83%
Adding a second AI reviewer to your coding pipeline sounds like a free quality gate. A new controlled study says it can actively make things worse, depending on which model you put in charge.
What the Researchers Tested
Researchers ran Claude Opus 4.7 and Codex GPT-5.5 through 116 medium and hard Python tasks from the LiveCodeBench benchmark. They tested each model alone, then swapped them into writer-reviewer pairs to see what changed.
Codex alone passed 71.6% of tasks. With Claude reviewing its output, that climbed to 89.7%. That direction works.
Flip it: Claude alone scored 91.4%. Hand its output to Codex for review, and the final pass rate fell to 82.8%.
Why the Pairing Broke Down
The numbers behind the drop tell the story. Claude repaired 26 of Codex’s failed answers while breaking only 5 that had previously passed, a net gain of 21. Codex fixed just 3 of Claude’s failures while turning 13 correct solutions into incorrect ones.
Co-author Zuodong Xiang puts it plainly: asking Codex to review Claude is like putting a junior engineer in charge of rewriting a principal’s code. The weaker model overwrites correct work it does not fully understand.
“You always want a more capable model to review your work. Just like with humans, you want someone more senior reviewing the work.” — Zuodong Xiang
The Cost Is Not Trivial
Beyond accuracy, the paper’s own estimates show the bill adds up fast. Having Claude double-check Codex output pushed average cost per task from $0.19 to $0.44 and latency from 38.5 seconds to 112.4 seconds. Cost more than doubled. Latency nearly tripled.
Xiang notes that smaller companies may find the accuracy gain worth it on critical code, but harder to justify for routine changes.
The Operator Takeaway
Researcher Suzhen Zhong, a PhD student at Queen’s University in Canada, cautions that public leaderboard scores alone are a poor guide to real deployment decisions. Different projects carry different complexity levels and coding conventions, and benchmarks may not reflect that.
If you are building an agentic coding workflow, the model hierarchy is a real architectural decision. Put your strongest model in the reviewer seat. A second opinion only helps when the second opinion is more qualified than the first.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
