Anthropic recently launched Claude Mythos Preview and Project Glasswing, a cybersecurity initiative involving 12 tech giants. Mythos claims to autonomously discover thousands of zero-day vulnerabilities, signaling an AI-driven security defense era. But within a week, cybersecurity startup AISLE replicated Mythos' flagship demo results using an open-source model with only 3.6B active parameters at a cost of $0.11 per million tokens.
Three Test Sets: From False Positives to Complex Math Reasoning
AISLE designed three tests. First, an OWASP false-positive test: a piece of Java SQL code that looks like SQL injection but is logically safe. Results showed a reverse scaling effect — the 3.6B small model correctly ruled it harmless, while Claude Sonnet 4.5, multiple GPT-4.1/5.4 series, and all Anthropic models up to Opus 4.5 confidently misclassified it as a high-risk vulnerability. Only a few top-tier models (o3, OpenAI-pro, Sonnet 4.6, Opus 4.6) got it right.
Second, the FreeBSD NFS vulnerability CVE-2026-4747, a 17-year-old unauthenticated remote code execution bug. All 8 tested models succeeded, including the 3.6B small model. Every model correctly identified the stack buffer overflow and rated it Critical RCE. AISLE concluded such detection capability has become "commoditized."
Third, the OpenBSD SACK vulnerability (27 years old), requiring true mathematical reasoning: tracing a signed integer overflow through multiple logical steps. The difficulty spike caused model divergence. GPT-OSS-120b (5.1B active) fully reproduced the exploit chain, scoring A+; Kimi K2 scored A-; while Qwen3 32B incorrectly concluded "code is robust," scoring F. Yet even here, an ultra-cheap open model matched the flagship system.
The Jagged Frontier: Subtask Heterogeneity
AISLE decomposes AI security pipeline into five subtasks: broad scanning, vulnerability detection, triage and validation, patch generation, and exploit construction. Each has different scaling properties. Mythos presents them as an integrated system, but model requirements vary drastically — some subtasks saturate at 3.6B parameters, others require deep reasoning. This echoes the "Jagged Frontier" concept from Harvard researchers: AI capability boundaries are uneven. Users deploying AI within the frontier see ~40% productivity gain; deploying beyond can degrade performance by 19%.
AISLE offers a practical corollary: "A thousand competent detectives searching everywhere will find more vulnerabilities than one genius detective guessing where to look." Deploying many cheap models for broad scanning may outperform scheduling a single expensive one. Since mid-2025, AISLE has run a real-world vulnerability discovery system, finding 15 CVEs in OpenSSL (12 in a single security version, CVSS 9.8 Critical), 5 in curl, and over 180 externally validated CVEs across 30+ projects.
The Moat Lies in the System, Not the Model
AISLE's analysis neither fully criticizes nor endorses Anthropic. Mythos proves that "AI security" is a real category, not a lab concept. Anthropic maximizes "intelligence density per token," which remains valuable for deep reasoning. But AISLE highlights a more fundamental issue: the moat lies in the system, not the model itself. The architecture embedding deep domain expertise — how to decompose tasks, schedule different-cost models across subtasks, and maintain maintainer trust in production — is the true differentiator.
Ultimately, AISLE's report shows cheaper, open models can already replicate Mythos' core demos. The real question may be: who will run the five-subtask architecture in production first.