The Biggest Problems With AI Coding Are Only Getting Worse

In March, AI figureheads crowed that their own employees would be relegated to the dustbin of history. "I think we will be there in three to six months, where AI is writing 90% of the code," proclaimed Anthropic CEO Dario Amodei. "And then, in 12 months, we may be in a world where AI is writing essentially all of the code." Even Ozymandias might have cautioned against such self-serving grandiosity, but Amodei's claims were not out of place. Amazon CEO Andy Jassy predicted AI significantly cutting the company's labor force, including programmers. Skeptical voices like that of tech analyst Ed Zitron, who called Amodei "a liar, a crook, a carnival barker, and a charlatan" in a newsletter published just prior to Amodei's coding claim, couldn't deflate the AI hype bubble.

Six months later, the fundamental problems with AI as a coding tool are being laid bare, and they're only getting worse. Far from AI heading toward sentience as proponents had predicted, companies that embraced AI in the hopes of speeding up work while shedding workers are learning the hard way that it's not a silver bullet. Meanwhile, research continues to show that not only has AI failed to speed up coding work, but that it actually slows programmers down. Code produced with AI is unreliable, and most often leads to dangerous security vulnerabilities. The basic issue is that AI makes mistakes, which means coders must choose between double-checking everything, or crossing their fingers when they commit the code. Those problems don't show any signs of abating, and there's no real solution in sight as AI-coding headaches proliferate across various industries.

If you've ever dealt with an incompetent employee or coworker, you know what a headache it can be. Constantly supervising them or double-checking their work not only wastes time but pulls you away from your own responsibilities as well. Talented coders who turn to AI for a productivity boost quickly find themselves in the same situation. The "hallucination" problem which persists in large language models isn't just Google telling people to eat glue. It's also producing bad code.

In July, AI research nonprofit METR (Model Evaluation and Threat Research) found that developers who used some of the best AI coding tools took 19% longer to complete tasks. Developers in the study accepted less than 44% of AI-generated code, and spent considerable time polishing the code they did use to make it functional. Researchers also observed that the more complex the environment, the less helpful AI was to the coders. Even more shockingly, some developers did not seem to notice that AI was slowing them down. After participating in the study, they estimated it had sped up their work by about 20%, nearly inverse with the reduction in efficiency the data showed.

Given this decrease in efficiency, it's no wonder companies that rushed to adopt AI aren't seeing the results they may have expected. AI is a headache to deal with when simply taking your Taco Bell order, so expecting it to generate complex code and replace four-year degreeholders never sounded rational. Nonetheless, research released in April from software company Orgvue found a 55% regret rate among the 39% of senior businesspeople who laid off employees to replace them with AI.

So AI-generated code might grind development time to a crawl, but on the other hand, it's also riddled with security vulnerabilities. Findings published in early September by security firm Apiiro reveal that when companies mandate the use of AI in their workforce, they begin to ship 10 times more "time bombs" in just a six-month period. The most common issue, with a 322% increase, was privilege escalation. That's when code includes vulnerabilities that allow hackers to gain high-level access to a system. When considering how many companies are now all-in on AI, the implications are alarming. Exploits like these are considered time bombs because the code may appear to work perfectly until a malicious actor discovers them. From banks to electric cars, easily exploited updates may already be compromising safety.

That study was foreshadowed with a joint paper published in May by researchers from the University of San Francisco and other institutions, which found that AI code increased critical vulnerabilities by 37.6% after just five "iterations." Similar outcomes persisted even when using different prompting techniques. In plain English, not only is AI-generated code creating security problems, but those problems get worse the more AI is used.

There's only one solution each respective research team recommends: strict human oversight to double-check AI-generated code before it can cause harm. As noted above, such oversight is a likely explanation for the reduced efficiency AI leads to in development. It's a chicken-and-egg problem — a Catch-22. Using AI to speed up development causes security nightmares, but checking the code for errors slows down development.