Lets Talk About Devin...

This is not to disregard what the team achieved but just my 2cents.

If you haven't seen it, where have you been? If you have what did you think? The link to the video and demos will be below:

Link 1 here: https://www.youtube.com/watch?v=fjHtjT7GO1c&pp=ygUGRGV2aW4g

This was a very interesting demo. Which became even more interesting with hindsight.

I was quick to praise the team. I was blown away the first time I saw the demo. Then I looked a bit closer. Having built a few automated systems. what amazed me about the demo was the robustness and errors it was able to handle.

HOWEVER! That was when I saw it. Watch this:

Link 2 here: https://www.youtube.com/watch?v=ReE2dFJn_uY

Notice anything different between the first video and the second?

Well Time Stamps.

it Took Devin 56 mins to correct the Github project. While this is still amazing it is not AGI! It still is amazing. Devin was able to interact with the code while conversing with the user, which is nice. But there were steps I do believe a capable software engineer would be able to overcome much faster. Using another LLM say GPT-4 or Claude 3, any capable software engineer is 100% faster than Devin.

But for the average Joe, this is witch craft.

AND POWERFUL! Although potentially problematic.

This is a AI Demo warning. We are so far over the technological edge, that demos now feel like firework displays. to the average person and shadow puppets to those with domain experience. Which makes conversations polarising and hard for anyone to follow and parse.

Since then I haven't been able to decide. I don't know whether it was a "little" disingenuous, or part of software demos. (Looking at you Google). It's an issue we faced recently. Edit the video and make it snappy, or be honest and show how long it takes. We decided on the later, but I can see how people chose the former.

DEVIN is a great example of this! To fully grok how Devin works you need a few tools in your tool kit:

1. A computer science degree. (or comparable understanding of how a computer works from silicone to snapchat)
2. An understanding of software development, stacks and how we actually speak to computers.
3. Domain knowledge and Experience. You need to know what it is doing, and how you would do what it did.
4. Fundamentals of Big Data and Data science.
5. Introduction to Machine learning and Deep learning.
6. Artificial Neural Networks.
7. Algorithm Deep Dive. (slightly Optional)
8. Transformer Architecture.
9. Large Language Models.
10. RAG (retrvial augmented generation) and other performance frameworks (COT, SBS etc.)
11. Audio processing, Image processing and NLP(Natural Language Processing)
12. And a lot of minutia!

So no wonder most people thought it was amazing.

It is the perfect example of how asymmetric information can lead to different conclusions.

think about it most people have never seen computer code. So it's already amazing (to them) it can handle it. When code to a developer is a much better search space with defined rules. Code either works or not. So this is GREAT for computers. Its all in the nuance and Minutia.

I am not trying to undermine Devin. But bring some nuance to the conversation. What it achieved is amazing. But if it takes 1 hour to build certain projects and costs an arm and a leg. Then this is still cool, but not the dream we are selling.

So yes, Software jobs are gone, yet!

What may happen is, the few developers we have now will split. into AI enhanced programmers and OG programmers. In the beginning OG programmers may be competitive, but given enough time (t) will become a niche. Like a handmade McLaren vs a Tesla! one company makes millions of vehicles per year. The other will be lucky to ship 5,000 cars. But both sell...

But we do need to start thinking about what happens when even software jobs become optional. Because it will not take long for Devin to improve and there is no defined limit... That is the scary part, we are already on that road. its just (t) now...