ProofRun gives AI coding agents a cryptographic alibi for test runs
When your AI coding agent says “all tests pass,” there are two very different things that phrase could mean. It could mean the agent ran the tests right now, against the current code, and they passed. Or it could mean the agent ran them three edits ago and is inferring they probably still pass. From the words alone, you cannot tell the difference.
ProofRun is a free, MIT-licensed CLI that closes that gap. It binds every test result to a cryptographic fingerprint of your exact code state at the moment the check ran.
️ How It Works
Every result is tied to your current git HEAD plus a SHA-256 hash of git diff HEAD combined with the contents of any untracked, non-ignored files. Change a single byte after a check passes and proofrun status reports STALE automatically. No one has to remember to ask whether the result is still valid.
There are exactly four statuses: PASS, FAIL, STALE, and NOT RUN. Every one of them comes from an observed execution or a documented absence of one. There is no fifth status that means “probably fine.”
$ proofrun run test -- pytest
test: pass (exit 0, 1841ms)
$ proofrun status
test PASS (exit 0, 1841ms)
# code changes after this point
$ proofrun status
test STALE (last run: pass, exit 0 — code changed since)
Key Design Decisions
- No LLM calls. ProofRun does not use AI to verify AI. It spawns a real subprocess and reads its real exit code.
- Fully offline. Zero network calls, zero telemetry, zero accounts required.
- Argv-exact matching. Checks are declared as argv arrays, not shell strings. A check declared as
pytest -k "foo bar"cannot be satisfied by a command that merely looks similar when flattened to text. - Deliberate scope. It does not parse test output, judge code quality, or auto-fix anything.
The Adversarial Review Detail Worth Reading
ProofRun was written by Claude Code under human direction. Before the first release, it went through several rounds of independent, read-only adversarial review. That review found a real vulnerability: a misquoted shell argument could make a check silently run zero tests and still report PASS. The full reproduction, the fix, and why a simple patch was not enough are documented in docs/case-study.md. Every fix was verified against a real reproduction before being accepted.
⚙️ CI Integration and Known Limitation
A GitHub Actions integration is available via yebiguo/proofrun@v1. It checks out the exact PR head commit directly, clears any receipt file that came in on the PR branch, downloads a checksum-verified binary, and runs proofrun run-all before gating on proofrun status --strict. No result from a receipt checked into the PR is ever trusted.
One known limitation: the Action does not block a PR that weakens or removes a check inside .proofrun.yml. It warns via a build annotation when that file differs from the base branch, but the gate does not block on it. You still need to review that diff manually.
Install and Pricing
Free. MIT licensed. Install via a pre-built binary for Linux, macOS, and other platforms at the GitHub releases page, or build from source with go install github.com/yebiguo/proofrun/cmd/proofrun@latest. Currently at v0.2.0, pre-1.0, with structured output support for pytest, Jest, and JUnit planned for v0.3.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
