Security Benchmarks Show Strong Automated Vulnerability Detection in Fable 5.2
Evaluators report that Fable 5.2 significantly improves upon previous models in identifying critical code vulnerabilities across complex software repositories. The model successfully uncovered fifty-eight planted bugs on standard evaluation benchmarks using fewer turns and lower costs than earlier models.
The Blend
Product researcher Paweł Huryn published benchmark tests evaluating how well leading artificial intelligence models can identify and repair software flaws across complex codebases. In a test involving 105 real code errors, the newly tested Fable 5.2 system located and fixed 58 bugs at its maximum setting. This performance placed it ahead of competing tools like Sonnet 5.5 and GPT-6 Astra in total vulnerabilities solved, while completing tasks in fewer steps and at a lower financial cost than its main rival.
Automated bug hunting is becoming a key defense against digital security breaches. For everyday consumers, software built with fewer security flaws means stronger protection against data leaks, software crashes, and online exploits. Furthermore, as AI tools become faster and cheaper at spotting these weaknesses, technology companies can secure their applications much more efficiently before releasing updates to the public.
It remains to be seen how well these artificial intelligence tools perform outside of controlled benchmark conditions on living codebases that undergo constant updates. Additionally, while high end models offer maximum bug detection, budget friendly options like GPT-6.1 Sol fixed nearly as many issues at a tiny fraction of the price. The long term question for the software industry is whether companies will prioritize total precision or accept slightly lower accuracy to save massive operational costs.
Written independently by AI News Smoothie from the reporting listed below. Facts belong to the original publishers. Follow the links for their full coverage.
Ingredients
- Paweł Huryn on X: "So I tested Fable 5.2 on real work. 2 repos, 105 bugs. Find and fix what you can.
The results:
- Fable 5.2 (max): 58 for $139.65
- Sonnet 5.5 (max): 51.3 for $154.03
- GPT-6 Astra (max): 45 for $33.03
- GPT-6.1 Sol (max): 44.3 for $5.88
- Fable 5.1 (max): 43 for $87.18 (n=1)
-… / X
Benchmark tests conducted by researcher Paweł Huryn show Fable 5.2 fixed 58 planted software bugs, outperforming competitor models in accuracy while using fewer processing steps.