Tool touted as 'first AI software engineer' is bad at its job, testers claim
The auto-coder is called “Devin” and was introduced in March 2024. The bot’s creator, an outfit called Cognition AI, has made claims such as “Devin can build and deploy apps end to end," and "can autonomously find and fix bugs in codebases." The tool reached general availability in December 2024, starting at $500 per month.
"Devin is an autonomous AI software engineer that can write, run and test code, helping software engineers work on personal tasks or their team projects," Cognition's documentation declares. It "can review PRs, support code migrations, respond to on-call issues, build web applications, and even perform personal assistant tasks like ordering your lunch on DoorDash so you can stay locked in on your codebase."
The service uses Slack as its main interface for commands, which are sent to its computing environment, a Docker container that hosts a terminal, browser, code editor, and planner. The AI agent supports API integration with external services. This allows it, for example, to send email messages on a user's behalf via SendGrid.
Devin is a "compound AI system," meaning it relies on multiple underlying AI models, a set that has included OpenAI's GPT-4o and can be expected to evolve over time.
