World's First AI Software Engineer 'Devin'

Meet Devin, the world’s first fully autonomous AI software engineer.

There is a new AI tool so smart that it can write code, create websites, and software with just a single prompt. Devin, created by the tech company Cognition, is the first AI software engineer. It can do pretty much everything you ask it to do. And the AI tool does not come with the intention to replace human engineers, it is designed to work hand-in-hand with them. The makers say that the AI tool has not been launched to replace human engineers but to make their lives easier

What All Does Devin Do?

A software development assistant, Devin does more than just complete and suggest code. It can independently manage the whole process of creating and releasing whole software applications, in contrast to large language models (LLMs) like Google's Gemini or OpenAI's ChatGPT.

With its own code editor, command line, and browser, Devin works autonomously, systematically putting software into practice by segmenting work into manageable steps.

Devin is expected to completely transform the way code is developed, debugged, and implemented.?As the AI assistant Devin can transform a basic prompt into a working website or software program, it is being heralded as the first AI of its kind in history.

The startup asserts that its AI assistant has completed actual jobs on Upwork, passed real-world engineering interviews with top AI companies, and is the new state-of-the-art on the SWE-bench coding benchmark.

SWE-bench is an assessment framework that includes 2,294 software engineering problems taken from real GitHub issues and the pull requests associated with those issues from 12 well-known Python repositories.?

Devin's Performance

On testing Devin on SWE-bench, a challenging benchmark that asks agents to resolve real-world GitHub issues found in open source projects like Django and scikit-learn. Devin correctly resolves 13.86%* of the issues end-to-end, far exceeding the previous state-of-the-art of 1.96%. Even when given the exact files to edit, the best previous models can only resolve 4.80% of issues.

*Devin was evaluated on a random 25% subset of the dataset. Devin was unassisted, whereas all other models were assisted?(meaning the model was told exactly which files need to be edited).

Core technology remains undescribed

AI in software development is no new feat. There have been tools in this space for quite some time, right from the popular GitHub Copilot and StarCoder to Replit, which has a few small AI coding models on Hugging Face, and Codeium, which recently nabbed $65 million series B funding at a valuation of $500 million.

However, most of these offerings have largely focused on using AI to assist with coding. They can generate barebones code from text prompts, summarize it with relevant IDE context or retrieve snippets, accelerating the workflow of the team. With Devin, Cognition AI appears to be going a step (or multiple steps) further, giving a full-fledged AI worker to handle entire projects.

While the tool remains to be tested, its ability to handle multiple steps – while staying on track – to complete a software engineering project is the biggest unique selling point. Cognition has not shared how exactly it has achieved this feat and whether it is using its own proprietary model or that from a third party, but it does note that the work is the result of its “advances in long-term reasoning and planning.”

Devin AI Overview

https://www.youtube.com/watch?v=fjHtjT7GO1c

References :

https://www.cognition-labs.com/blog