Open Models Show Strong Performance in Cybersecurity Evaluation

Cybersecurity company Aikido evaluated ten major artificial intelligence models to see how well they could detect real-world software security flaws. By running tests across recently disclosed vulnerabilities, Aikido discovered that open and lower-cost AI models, when prompted multiple times, frequently matched or exceeded the bug-hunting performance of leading closed-source options.

In the tests, systems like DeepSeek V4 Pro outperformed heavyweights such as Claude Opus 5 and Grok 4.6 when scores were pooled over three separate attempts. Running a budget model repeatedly allowed it to discover flaws missed on its initial pass. This multi-run strategy proved significantly cheaper than relying on expensive proprietary software, delivering high vulnerability detection rates for a fraction of the token expenditure.

However, the researchers noted a distinct downside to this budget approach: lower-cost models produced a much higher volume of false alarms that automated filters or security staff had to sort through. This raises a crucial operational question for engineering teams deciding whether saving money on API fees is worth the extra labor required to clear out bogus security alerts.