Bottom line: Green CI/CD checks are not a reliable indicator that an AI-generated pull request is production-ready.
Three AI-based code assistants were tested side-by-side in a real software repository to evaluate their practical performance in solving identical tasks. The test reveals the limits of current code generation systems and what remains missing between passing checks and production-ready software.
The real-world test compared Claude Code, Codex and Cursor in an existing repository based on clearly defined tasks. All three systems were confronted under identical conditions with the same code context and the same requirements. This creates a basis for direct comparison rather than abstract benchmark scenarios.
A central finding of the test was that automated tests and CI/CD pipelines only capture part of code quality. Although some AI-generated changes passed all defined checks, they showed significant shortcomings in practical evaluation by humans – such as when handling edge cases, consistency with existing conventions, or maintainability of the resulting code. A green check in the repository system is therefore not a guarantee for a merge-ready pull request.
The test thus highlights a growing discrepancy in AI-supported development: while generative models increasingly perform better on automated metrics, manual code review remains a necessary filter. Engineers should use AI assistants as productivity tools without delegating full responsibility for validation, security, and architectural consistency.
Source: www.golem.de · Published June 5, 2026
Lumi AI News — AI-assisted curation pursuant to Art. 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.6.4.