The Ethereum Foundation has shared fresh findings from its Protocol Security team, detailing how AI agents are being used to hunt for vulnerabilities in Ethereum's core code. The headline number: nearly 90% of AI-generated bug reports are false positives.
High False Positive Rate Remains a Bottleneck
According to the team's tests, AI agents produce detailed reports including attack flows, code analysis, root causes, and exploitation scenarios — making them look convincing. Yet researchers still have to manually reproduce, test, and verify each alert. Out of every ten AI warnings, only about one turns out to be a real vulnerability. The foundation acknowledges that while AI accelerates the initial screening, human expertise is still the deciding factor.
Multi-Agent Collaboration for Better Coverage
Instead of relying on a single model, the team deployed a multi-agent architecture where different AI agents handle separate tasks. Some search for attack surfaces, others analyze logic and propose hypotheses, and a third group reviews outputs to remove duplicates, add test cases, or even design new attack methods. This mirror a human security team's workflow, aiming to cross-validate findings. Still, every candidate bug must be replayed and verified by a human researcher before a patch is submitted.Human Judgement Remains Central
The foundation stresses that the real metric for AI systems is not how many reports they generate, but how many actual bugs are confirmed. With false positives still hovering around 90%, AI serves as a first-pass filter. As large language models improve, AI could take on more code audit work and reduce the risk of vulnerabilities slipping into production. For now, the priority remains boosting search efficiency, lowering false positives, and building a more effective human-machine pipeline — with the final call always in human hands.