Read This Before You Trust Any AI-Written Code

We are in the era of vibe coding, allowing artificial intelligence models to generate code based on a developer’s prompt. Unfortunately, under the hood, the vibes are bad. According to a recent report published by data security firm Veracode, about half of all AI-generated code contains security flaws.

Veracode tasked over 100 different large language models with completing 80 separate coding tasks, from using different coding languages to building different types of applications. Per the report, each task had known potential vulnerabilities, meaning the models could potentially complete each challenge in a secure or insecure way. The results were not exactly inspiring if security is your top priority, with just 55% of tasks completed ultimately generating “secure” code.

Now, it’d be one thing if those vulnerabilities were little flaws that could easily be patched or mitigated. But they’re often pretty major holes. The 45% of code that failed the security check produced a vulnerability that was part of the Open Worldwide Application Security Project’s top 10 security vulnerabilities—issues like broken access control, cryptographic failures, and data integrity failures. Basically, the output has big enough issues that you wouldn’t want to just spin it up and push it live, unless you’re looking to get hacked.

Perhaps the most interesting finding of the study, though, is not simply that AI models are regularly producing insecure code. It’s that the models don’t seem to be getting any better. While syntax has significantly improved over the last two years, with LLMs producing compilable code nearly all the time now, the security of said code has basically remained flat the whole time. Even newer and larger models are failing to generate significantly more secure code.

The fact that the baseline of secure output for AI-generated code isn’t improving is a problem, because the use of AI in programming is getting more popular, and the surface area for attack is increasing. Earlier this month, 404 Media reported on how a hacker managed to get Amazon’s AI coding agent to delete the files of computers that it was used on by injecting malicious code with hidden instructions into the GitHub repository for the tool.

Meanwhile, as AI agents become more common, so do agents capable of cracking the very same code. Recent research out of the University of California, Berkeley, found that AI models are getting very good at identifying exploitable bugs in code. So AI models are consistently generating insecure code, and other AI models are getting really good at spotting those vulnerabilities and exploiting them. That’s all probably fine.