AI-Generated Code Has a Verification Problem

By Swayam Swarup Panda

CVEs traced to AI-generated code climbed from 6 in January to 35 in March, and security pass rates on AI-generated code have stayed flat near 55% for two years despite every other benchmark improving. Here's what the data actually says.

The gap between "works" and "safe"

There's an uncomfortable pattern showing up consistently across independent research this year: AI coding tools have gotten dramatically better at writing code that runs, while barely improving at writing code that's safe. Those turned out to be two different skills, and only one of them has been improving.

Veracode's Spring 2026 testing across more than 100 large language models found security pass rates stuck near 55% on security-sensitive coding tasks — essentially flat across two years of model releases, even as syntax pass rates for the same models climbed past 95%. Carnegie Mellon's SusVibes benchmark reached a similar conclusion from a different angle: roughly 61% of AI-generated solutions were functionally correct, but only about 10.5% passed a security review. Fewer than 11 in every 100 AI-generated code snippets meet basic security standards, even as the code itself works fine on the happy path.

The numbers are trending the wrong way

Georgia Tech's Vibe Security Radar, which traces CVEs backward to determine whether they originated in AI-generated code, tracked the trajectory closely: six CVEs attributable to AI coding tools in January 2026, fifteen in February, thirty-five in March — more in that single month than the entirety of 2025 combined. Researchers estimate the true number is five to ten times higher, since most AI coding tools leave no commit metadata distinguishing AI-authored code from human-written code, making most of the true count invisible to this kind of tracing.

Separately, GitGuardian's State of Secrets Sprawl 2026 report found AI-assisted commits leaking hardcoded credentials at more than double the rate of human-only commits — 3.2% versus 1.5%.

Why "it works" keeps getting mistaken for "it's done"

The uncomfortable truth underneath all of this is straightforward: AI coding tools are optimized to produce code that satisfies the prompt and passes the tests you gave it. Neither of those things checks for the class of vulnerability that only shows up when someone deliberately tries to break your assumptions — an attacker changing a value in a URL, an unvalidated input reaching a database query, an endpoint with no authentication check because the happy-path demo never exercised it. Apiiro's research on AI-assisted code at scale found 322% more privilege escalation paths in AI-assisted code compared to human-only code from the same teams — the same pattern, showing up in production.

What actually reduces the risk

The research is consistent on one point: structured oversight closes most of this gap. Unstructured "describe it and ship it" workflows produce the worst numbers across every study. Workflows with rich specifications, systematic code review, dedicated security scanning, and a deliberate verification step before merge look meaningfully different — not perfect, but no longer in "62% of applications ship with a critical vulnerability" territory.

Concretely, that means: don't skip code review just because an agent wrote the diff instead of a person. Run security-specific static analysis on AI-generated code the same way you would on a junior developer's first pull request, because in security terms, that's roughly the trust level it warrants. And treat "the tests pass" as a floor, not a finish line — the tests only check what someone thought to write a test for.

The takeaway

None of this is an argument against using AI coding tools. It's an argument against treating their output as pre-verified just because it compiles and the demo works. The verification step didn't go away when the code got faster to produce. It just got easier to skip — and the data says a lot of teams are skipping it.

Follow me to keep in touch

Where I share my creative journey, design experiments, and industry thoughts.