An analysis of 470 real-world open source pull requests published today finds code generated using artificial intelligence (AI) tools introduces significantly more defects across every major category of software quality, including logic, maintainability, security, and performance, compared to human-authored code.

Conducted by CodeRabbit, a provider of a platform that employs AI to review code, the report finds that pull requests generated using AI contain ~1.7x more issues on average than ones created by a human.

Additionally, logic and correctness issues rise 75%, including business logic errors, misconfigurations, and unsafe control flow, while security vulnerabilities increase 1.5–2x, according to the report.

Finally, code readability problems such as elevated naming and formatting inconsistencies increase by a factor of more than 3x, while performance inefficiencies, such as excessive I/O, appear nearly 8x more often in AI-generated code.

David Loker, director of AI for CodeRabbit, said the report makes it clear that while AI makes developers more productive the amount of technical debt currently being generated by these tools is unsustainable. Short of abandoning these tools, the only way to address the fundamental challenge is to rely more on AI to review the code being generated, he added.

For example, organizations need to implement tests for non-trivial control flow, mandate nullability/type assertions, standardize exception-handling rules, and enforce effective guardrails. Code reviewers should also explicitly ask if error paths are covered, if concurrency primitives are correct, if configuration values are validated, and if passwords are handled via the approved helper, all of which are areas where AI coding tools tend not to address.

DevOps teams should make sure that application developers are exposing prompt snippets, repo-specific instruction capsules, and configuration schemas that expose business rules, configuration patterns, or architectural constraints that the AI tool needs to observe. Continuous integration (CI) tools that enforce formatters, linters, and style guides can also eliminate entire categories of AI-driven issues.

DevOps teams should also further centralize credential handling, block ad-hoc password usage, and automatically run static application security testing (SAST).

It’s not clear how many organizations are revisiting DevOps workflows in the wake of adopting AI coding tools, but it’s apparent the amount of technical debt that needs to be addressed is rising as the pace at which code is being developed only continues to accelerate. DevOps teams need to pay particular attention to the readability and maintainability of their code bases to ensure the highest quality applications possible are being deployed in production environments, noted Loker. That goal, however, can only be achieved by ensuring best DevOps practices are followed, a goal that might soon be easier to achieve as AI is used to verify the quality of code being generated, he added.

Hopefully, the quality of the code generated by AI coding tools will improve. In the meantime, however, DevOps teams might need to prepare for the worst as the number of issues that engineers will need to address after code has been checked in starts to exponentially increase.