To Be or Not To Be an Agent

Should software assist humans or act on their behalf?

In 2016, the question was easy to answer : sell Ironman not Robocop. Technology hadn’t reached the level of sophistication we have attained today where AI is 90% as capable as a high-school student, the MMLU benchmark for AI is precisely this.

The next generation of software startups have a strategic question with different terminology & potentially a different conclusion.

To be or not to be an agent, acting on behalf of workers?

Copilots, like Github’s, complete their humans’ sentences in code, an AI pair programmer. Copilots have proven to increase productivity by 50-75% according to data points from Microsoft & ServiceNow.

Devin AI, the world’s first AI software engineer aka agent, authors software in place of a human.

We don’t yet know how productive agents will be, but if Devin is any indication, the productivity gains could mirror that of robots in manufacturing.

A 2020 MIT study found 1 robot in a manufacturing facility replaced 3.3 workers. Instead of a 50% improvement, these robots were 2.3x more productive. Robots can work 24h in contrast to a human’s shift of 8 hours, which increases productivity 2.0x simply through longer shifts.1

Can a startup capture more value selling a copilot with 50% performance improvement or an agent with a 230% performance improvement?

It hinges on a key development in GTM - the subject of tomorrow’s post.

1I’m ignoring maintenance & downtime in this naive calculation.