Laravel Boost benchmarks hit 100%: what comes next
The Laravel Boost benchmark suite started with a single question: can an AI coding agent write correct Laravel code? The answer is now definitively yes.
With Boost supplying framework context, GPT-5.6, Claude Fable 5 and Mythos 5, Gemini 3.x, and comparable frontier models all clear the suite’s 17 evals at or near 100% accuracy. That’s up from 99.4% on a previous run, where the hardest tasks still caught the models out. The finish line moved.
Why the benchmark is being retired as the main signal
This is not just a Boost story. SWE-bench Verified, the headline coding benchmark for the field, has also effectively saturated. Leading models now cluster within a point or two of each other near the human-expert baseline, after sitting around 60% a year ago. HumanEval stopped separating leaders even earlier.
OpenAI stopped reporting SWE-bench Verified results after an audit found a large share of problems had flawed test cases. Cursor found that many “successful” fixes came from models locating the known answer in GitHub or .git history rather than reasoning through the problem, and scores dropped sharply once that history was removed.
Boost is less exposed to those issues. Every run starts from a fresh Laravel app with no answers tucked into .git, and architecture tests run alongside behavioral ones. But a passing Pest suite still only proves a model can satisfy the suite. It does not prove the model writes Laravel the way an experienced developer would.
The two new metrics that matter
The Laravel team is now treating two things as first-class signals instead of footnotes.
Correct code per token. When every model reaches the right answer, the spread in cost to get there becomes the differentiator. According to the post, one model can spend an order of magnitude or two more than another on the same task and land at nearly identical results. Boost already records tokens and cost per run. Those numbers are moving to headline status.
Idiomatic Laravel, not merely correct Laravel. A controller can pass every test while still using a raw query, ignoring Route::resource(), hand-rolling validation instead of using a form request, and skipping $fillable. It works. It does not belong in the codebase.
Boost’s official best-practices skill covers 19 concrete conventions: Cache::remember() over hand-rolled caching, form requests over inline validation, with() for eager loading, Route::resource(), and more. The first rule is “Consistency First”: match the sibling files already in the application before reaching for a different pattern. Reference-free LLM-as-judge scoring is now good enough to evaluate idiom compliance without needing a golden answer to diff against.
What’s changing in the benchmark going forward
The existing 17 evals stay in place as a regression suite. On top of that foundation, the team is exploring:
- Token and cost reported per eval and per model alongside pass rate
- Idiomatic scoring layered on top of existing Pest and architecture checks
- Context tightening inside Boost itself: dropping redundant guidelines, tightening tool outputs, indexing only what an agent actually needs
For Laravel developers evaluating AI coding tools, the practical implication is this: “did the tests pass?” is no longer a sufficient question. The more useful questions are how many tokens it took to get there, and whether the output looks like code your team would actually merge.
The Boost repo is open if you want to follow along.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
