Guardix on EVMBench: 104 of 117 known vulnerabilities (88.9%)
Guardix detected 104 of the 117 high-severity, loss-of-funds bugs in EVMBench (also written "EVM Bench"), an attributed recall of 88.9% across 40 audits. Every ground-truth bug was found in 30 of the 40 audits, and no audit scored zero. This is the completed August 2026 board, published on September 11, 2026.
| Metric | Result |
|---|---|
| Metric Audits scored | Result 40 |
| Metric Ground-truth vulnerabilities | Result 117 |
| Metric Attributed detections | Result 104 |
| Metric Recall | Result 88.9% |
| Metric Audits with 100% recall | Result 30 / 40 |
| Metric Audits with 0% recall | Result 0 / 40 |
| Metric Remaining misses | Result 13 |
What EVMBench is
EVMBench is a public benchmark for smart contract vulnerability detection, part of OpenAI's frontier-evals suite (OpenAI: Introducing EVMbench). It holds 117 high-severity, loss-of-funds bugs from 40 Code4rena contests. Human auditors found each bug, judges confirmed it, and Code4rena paid a bounty.
Scoring is attributed. A judge checks whether our report described the same mechanism, code path and fix as the original finding. A nearby finding in the same file doesn't count. The score is detections out of 117, not how many findings the report listed.
Results by audit
"Findings" is how many rows the judge saw for that audit. It doesn't affect the score. A dash means we have no count for that audit.
| Audit | Known bugs | Detected | Recall | Findings |
|---|---|---|---|---|
| 2024-01-renft | 6 | 6 | 100% | 76 |
| 2024-08-phi | 6 | 6 | 100% | 240 |
| 2024-07-munchables | 5 | 5 | 100% | 52 |
| 2024-01-curves | 4 | 4 | 100% | 195 |
| 2024-12-secondswap | 3 | 3 | 100% | – |
| 2023-07-pooltogether | 2 | 2 | 100% | 69 |
| 2023-10-nextgen | 2 | 2 | 100% | 206 |
| 2023-12-ethereumcreditguild | 2 | 2 | 100% | 196 |
| 2024-01-canto | 2 | 2 | 100% | 93 |
| 2024-03-canto | 2 | 2 | 100% | 139 |
| 2024-05-munchables | 2 | 2 | 100% | 62 |
| 2024-06-thorchain | 2 | 2 | 100% | 85 |
| 2024-06-vultisig | 2 | 2 | 100% | 94 |
| 2024-07-basin | 2 | 2 | 100% | 59 |
| 2024-07-traitforge | 2 | 2 | 100% | 181 |
| 2025-10-sequence | 2 | 2 | 100% | – |
| 2026-01-tempo-stablecoin-dex | 2 | 2 | 100% | 51 |
| 2024-02-althea-liquid-infrastructure | 1 | 1 | 100% | 97 |
| 2024-03-coinbase | 1 | 1 | 100% | 58 |
| 2024-03-gitcoin | 1 | 1 | 100% | 40 |
| 2024-03-neobase | 1 | 1 | 100% | 70 |
| 2024-05-arbitrum-foundation | 1 | 1 | 100% | 68 |
| 2024-05-loop | 1 | 1 | 100% | 61 |
| 2024-08-wildcat | 1 | 1 | 100% | 61 |
| 2025-01-liquid-ron | 1 | 1 | 100% | – |
| 2025-01-next-generation | 1 | 1 | 100% | – |
| 2025-02-thorwallet | 1 | 1 | 100% | – |
| 2025-05-blackhole | 1 | 1 | 100% | 322 |
| 2026-01-tempo-feeamm | 1 | 1 | 100% | 51 |
| 2026-01-tempo-mpp-streams | 1 | 1 | 100% | 41 |
| 2024-04-noya | 20 | 19 | 95% | 228 |
| 2024-07-benddao | 7 | 6 | 86% | 198 |
| 2024-03-abracadabra-money | 4 | 3 | 75% | 310 |
| 2024-06-size | 4 | 3 | 75% | 51 |
| 2025-04-virtuals | 4 | 3 | 75% | 770 |
| 2024-01-init-capital-invitational | 3 | 2 | 67% | 102 |
| 2025-04-forte | 5 | 3 | 60% | – |
| 2024-05-olas | 2 | 1 | 50% | 107 |
| 2025-06-panoptic | 2 | 1 | 50% | 77 |
| 2024-03-taiko | 5 | 2 | 40% | 92 |
| Total | 117 | 104 | 88.9% |
How not to read 88.9%
-
It's a score on a public benchmark, not a score for your repository.
-
Guardix doesn't replace a manual audit. We still missed 13 known bugs, on Init Capital, Abracadabra, Taiko, Noya, Olas, Size, BendDAO, Forte, Virtuals and Panoptic.
-
These runs use the same hours-long pipeline a customer gets, not a shortcut.
Earlier results
Our April run scored 59.8% (70 of 117, with 21 of 40 audits at full recall). It was a different run on the same 117 bugs, so don't average the two. The September write-up explains what changed.
Other published EVMBench results
| Tool | Result | Set | Source |
|---|---|---|---|
| Tool Guardix | Result 88.9% (104/117), attributed recall | Set 40 audits, 117 bugs | Source Guardix blog |
| Tool Nethermind AuditAgent | Result 67% recall (after validation), April 2026 | Set 40 repos, 120 bugs | Source AuditAgent blog |
Both results are self-reported, on different EVMBench versions (117 vs 120 bugs) with different scoring, so they aren't directly comparable.
Limitations. EVMBench has public critiques, for example Re-Evaluating EVMBench. A benchmark shows how a tool does on known, past bugs, not on your code.
Run the same pipeline on your repo
The pipeline behind these numbers is the one you get: $200 flat per repo and commit. What a pre-audit is · Compare AI auditors · Pricing
FAQ
- What is EVMBench?
- A public benchmark in OpenAI's frontier-evals suite with 117 high-severity bugs from 40 Code4rena contests. Tools are scored on how many of those bugs they find.
- What does 88.9% recall mean?
- Guardix's reports described 104 of the 117 known bugs with the same mechanism, code path and fix as the original finding.
- Is Guardix the best AI auditor?
- We can only point to published numbers: Guardix 104 of 117 (88.9%), Nethermind AuditAgent 67% on a 120-bug version. They're self-reported on different sets, so they aren't directly comparable.
- Does EVMBench recall mean my code is safe?
- No. It measures past, known bugs. Your code needs its own review, and high-value code still needs a manual audit.