Coinbase cut fraud 30% by layering an LLM agent on existing ML
Most teams building AI fraud detection start from scratch. Coinbase took a different approach: they kept their existing ML model and rules on Onramp and built an LLM risk agent on top.
Online A/B experiments showed a 30% reduction in fraudulent transactions and a 22% drop in fraud value. That is a meaningful result without replacing what was already working.
Then they went further
After the A/B results held up, Coinbase post-trained Qwen3.5-9B on its proprietary Onramp fraud dataset. The fine-tuned model outperformed Opus 4.5 on every fraud detection metric they measured. It also delivered 55% lower end-to-end online serving latency compared to the larger model.
The pattern here is worth noting for any operator running ML in production: layering an LLM agent onto an existing system is a faster path to a result than rebuilding from scratch, and post-training a smaller open model on your own data can beat a frontier model on your specific task while cutting inference costs.
Coinbase is publishing this as part one of a three-part series on owning intelligence. The full writeup is on the Coinbase blog.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
