Lets Talk About Devin...
Let's Talk about it!

Lets Talk About Devin...

This is not to disregard what the team achieved but just my 2cents.

If you haven't seen it, where have you been? If you have what did you think? The link to the video and demos will be below:

Link 1 here: https://www.youtube.com/watch?v=fjHtjT7GO1c&pp=ygUGRGV2aW4g

This was a very interesting demo. Which became even more interesting with hindsight.

I was quick to praise the team. I was blown away the first time I saw the demo. Then I looked a bit closer. Having built a few automated systems. what amazed me about the demo was the robustness and errors it was able to handle.

HOWEVER! That was when I saw it. Watch this:

Link 2 here: https://www.youtube.com/watch?v=ReE2dFJn_uY

Notice anything different between the first video and the second?

Well Time Stamps.

Image 1, The problem and initial start time. 11:16 AM
Image 2, The solution and the finish time 12:12PM


it Took Devin 56 mins to correct the Github project. While this is still amazing it is not AGI! It still is amazing. Devin was able to interact with the code while conversing with the user, which is nice. But there were steps I do believe a capable software engineer would be able to overcome much faster. Using another LLM say GPT-4 or Claude 3, any capable software engineer is 100% faster than Devin.

But for the average Joe, this is witch craft.

AND POWERFUL! Although potentially problematic.

This is a AI Demo warning. We are so far over the technological edge, that demos now feel like firework displays. to the average person and shadow puppets to those with domain experience. Which makes conversations polarising and hard for anyone to follow and parse.

Since then I haven't been able to decide. I don't know whether it was a "little" disingenuous, or part of software demos. (Looking at you Google). It's an issue we faced recently. Edit the video and make it snappy, or be honest and show how long it takes. We decided on the later, but I can see how people chose the former.


DEVIN is a great example of this! To fully grok how Devin works you need a few tools in your tool kit:

  1. A computer science degree. (or comparable understanding of how a computer works from silicone to snapchat)
  2. An understanding of software development, stacks and how we actually speak to computers.
  3. Domain knowledge and Experience. You need to know what it is doing, and how you would do what it did.
  4. Fundamentals of Big Data and Data science.
  5. Introduction to Machine learning and Deep learning.
  6. Artificial Neural Networks.
  7. Algorithm Deep Dive. (slightly Optional)
  8. Transformer Architecture.
  9. Large Language Models.
  10. RAG (retrvial augmented generation) and other performance frameworks (COT, SBS etc.)
  11. Audio processing, Image processing and NLP(Natural Language Processing)
  12. And a lot of minutia!

So no wonder most people thought it was amazing.

It is the perfect example of how asymmetric information can lead to different conclusions.

think about it most people have never seen computer code. So it's already amazing (to them) it can handle it. When code to a developer is a much better search space with defined rules. Code either works or not. So this is GREAT for computers. Its all in the nuance and Minutia.

I am not trying to undermine Devin. But bring some nuance to the conversation. What it achieved is amazing. But if it takes 1 hour to build certain projects and costs an arm and a leg. Then this is still cool, but not the dream we are selling.

So yes, Software jobs are gone, yet!

What may happen is, the few developers we have now will split. into AI enhanced programmers and OG programmers. In the beginning OG programmers may be competitive, but given enough time (t) will become a niche. Like a handmade McLaren vs a Tesla! one company makes millions of vehicles per year. The other will be lucky to ship 5,000 cars. But both sell...

But we do need to start thinking about what happens when even software jobs become optional. Because it will not take long for Devin to improve and there is no defined limit... That is the scary part, we are already on that road. its just (t) now...


要查看或添加评论,请登录

Benjamin Sangwa的更多文章

  • Why Elon Musk Matters.

    Why Elon Musk Matters.

    100,000 Coherent H-100s What does this mean, why is it important? Thats what this article is going to explain and why…

  • There's a GAP between Public and Private health in the UK. It needs to be filled, filled with CARE.

    There's a GAP between Public and Private health in the UK. It needs to be filled, filled with CARE.

    Health Care in the UK: The Decline of Trust I wasn’t born in the UK, and where I come from, relying on public…

  • Agents... What the heck is even that!

    Agents... What the heck is even that!

    I thought long and hard about a name on Twitter. I ended up picking Dr.

  • I Don't Think we Think Anymore...

    I Don't Think we Think Anymore...

    Stop! Think. What do you think thinking is? As a young man entering my professional career, I was told this.

    1 条评论
  • The Alignment Problem (Biology vs Compute)

    The Alignment Problem (Biology vs Compute)

    This is Part 1. The alignment problem is a central challenge in both AI and biology, and refers to the difficulty of…

    1 条评论
  • CAIO's Why Does Everyone Need One?

    CAIO's Why Does Everyone Need One?

    What is a CAIO? Introduction Chief Artificial Intelligence Officers, or CAIOs, will become more and more common in…

    1 条评论
  • Complexity VS Complication

    Complexity VS Complication

    I think this should have been the first article. Without this simple razor I would be lost.

    11 条评论
  • ARE YOU ABOVE OR BELOW THE AI WATERLINE?

    ARE YOU ABOVE OR BELOW THE AI WATERLINE?

    This is a list of skills we think AI is better at than humans, where humans and AI can replace entire teams and where…

    1 条评论
  • What Is Intelligence?

    What Is Intelligence?

    Where did it come from? Can we build it? What is Intelligence? The definition of intelligence is a complex and…

    1 条评论
  • A New (but old) View On AI ???

    A New (but old) View On AI ???

    I think, and I want to be careful about how I say this. But, I genuinely think (on average) we are more scared of AI…

    4 条评论

社区洞察

其他会员也浏览了