How Generative AI will impact product engineering teams

What impact will Generative AI coding and productivity tools have on teams that build software products?

I think it’s probably safe to assume that we’re now past the point where there is still any debate that generative AI tools — like ChatGPT, Midjourney and Dall-E — are going to make an impact on the way that we work. The only remaining questions are really ‘how big?’ and ‘when?’

For the past few months I have often found that when I think about staffing tech teams, I end up questioning a pretty stable model for teams that I’ve been able to rely on for years. With the speed of the evolution in Generative AI tools and the amount of funding being redirected into the space, there’s one thought that keeps troubling me:

The next-generation product engineering team will probably have far fewer engineers than today.

In this series of six articles I’m going to explore the current state and structure of teams that build digital products, the dramatic shift that has become apparent in the last few months and what the potential impact of the new generation of AI coding tools will have on teams themselves. By the end of the article, Tech and Product leaders, as well as engineers themselves, should understand the scope of the change and have a foundation with which to consider how they will respond.

This series is primarily aimed at tech and product leaders with an understanding of product development, but I’ve tried to explain any more technical concepts in context where I can. It is not an exploration or comparison of AI tools in detail, but rather the impact that they may have on teams.

Where we are today — The 5:1?Ratio

Five to One. This is my very rough rule of thumb for the number of engineers per product manager in a team that builds digital products.

My day job is working as a consulting CTO, advising executive teams or working directly alongside product & tech leaders on their strategy and the structure of their product & tech teams. When someone asks for advice on how to structure their product engineering teams, my answer will invariably be something like ‘cross functional teams of five to nine people, with all the skills needed to deliver the business outcome’.

This advice stays remarkably consistent, regardless of the company size. When startups are in their first flush of youth and their engineering teams are growing fast, ten to twelve people working in one team is usually the point at which the team size starts to cause drag and a loss of clarity. At this critical inflection point, when there’s more work than the single team can handle, it’s time to scale. One team splits into two. Two teams split into three, each team with five to nine people, multiplying like mitosis.

In larger companies that are scaling multi-team tribes, that model remains consistent. The largest tech team I have been responsible for was around 300 engineers, part of a product and tech team of around 450 people. Even at this overall scale, down at the team level it’s dominated by that 5:1 ratio — roughly five engineers and one product manager, usually with unique variations of additional roles, like design, UX, agile coaches, QA and dev ops to make up the numbers.

What makes a?team?

Every product engineering team and every CTO is different. While my personal preference is that product managers know their customers, take responsibility for commercial outcomes, scan the horizon for innovation and work directly with their team to prioritise work, some companies choose to separate product owner and and product manager roles. I prefer agile coaches that work across teams, but some people prefer scrum masters embedded within teams. Some people swear by the importance of testing and QA (quality assurance) as a specific role inside a team, but I prefer to push responsibility to quality for the people building the code and ‘shift-left’ as much as possible. Product design and UX research might be in the team, or provided as a service to the team. There are a thousand configurations with no specific right or wrong answer, and always sensitive to the context of the organisation.

Were I to be asked to create a mythically ‘ideal’ team, I would demand a product lead and a technical lead with equivalent seniority and in creative tension with each other. The importance of these two roles is to represent (and disagree about) their different perspectives. Customer delight. Cost. Benefit. Risk. Value. As I’ll often intone: “Product decides what to build, and tech decides how to build it”.

For the last 20 years or so, since agile methodologies established themselves as the prevalent approach to software development, product engineering teams have usually had a product lead and some number of engineers of varying seniority. In today’s world, an engineer is the de facto unit of product delivery — without engineers turning requirements into outcomes, we don’t ship any products.

Engineers don’t just build new products and features; they’re also responsible for keeping the current products fit and healthy. A great portion of an engineer’s week can be spent maintaining their existing products and systems; fixing bugs, upgrading versions, responding to minor requests for change and addressing security issues. The rest of their week may be spent on more novel problems, building new capabilities or implementing new technical solutions. In addition, most engineering teams have junior engineers requiring time to learn and be trained, and more senior engineers treading water to stay current with the constantly changing face of tech.

The role of a product manager in the team is a critical one. If engineers are our units of value delivery, product managers have equally vital responsibilities; eliciting the customer and stakeholder requirements, helping the team translate challenges into solutions in creative ways, analysing and demonstrating the value of the outcome, prioritising the order in which the value is delivered and then justifying the return on investment of all the effort.

In our team with five engineers writing code, ‘one’ is just about the ideal number of product managers to keep the engineers engaged, productive and well supplied with great work to deliver.

Until now, that is.

The Rise of Generative AI in Product Engineering

Many CEOs I’ve spoken to in the last few months have been focused on how they might crowbar ‘an AI’ into their product offering, with a feverishness somewhere between hysteria and desperation. My recommendation to them is that they should leave the ‘real’ AI work to people with venture capital cash to burn, and simply integrate whatever services OpenAI, Microsoft, Salesforce, Amazon and Google deliver to us in the coming months.

Momentarily putting aside the application of Generative AI (the catch all term I’ll use for GPT, LLMs etc) in new products, I think there is a much more interesting and strategic discussion to be had — what our teams will look like when Generative AI is truly production ready.

In the face of the SAG-AFTRA and WGA actors and writers strike in the US, it’s revelatory to look back at how the mighty McKinsey predicted the impact of AI on the workforce back in 2017. After a great deal of research and analysis, McKinsey rank ordered the professions that would be more or less impacted by AI. McKinsey proposed that the least impacted would be “Creatives,” a small but growing category of artists, performers, and entertainers who will be in demand as rising incomes create more demand for leisure and recreation”. According to McKinsey, entertainers, just like the script writers and actors now on strike, weren’t just going to avoid the impact of AI, but would actively benefit from it. There’s a caution here for anyone making any predictions about the impact of AI (including anything you’re reading in this article).

Of course, McKinsey’s crystal ball is no more effective than anyone else’s and I’m being churlish in referencing six year old research. Even so it’s sobering to see that the impact of AI isn’t going ignored by the world’s largest creative industry, but is actually a key thrust in the creative unions’ complaint against studios and producers. From uproar over the use of generative AI art in the titles of Marvel’s Secret Invasion, to the (disputed) claim that extras may be required to sign over rights to their digital likeness in perpetuity for later use by AI effects, one thing is very clear: creative industries are indisputably going to be impacted by AI in the near future.

Why a scenic detour about the creative industry in an article about product engineering teams? Well, guess who was next on McKinsey’s big list of ‘safe’ professions? According to McKinsey’s, the next least likely group to be impacted by AI would be, “IT professionals and other technology specialists, managers and executives, whose work cannot easily be replaced by machines”

McKinsey’s view, all the way back in the distant, murky past of 2017, was that “automation will have a lesser effect on jobs that involve managing people, applying expertise, and social interactions, where machines are unable to match human performance for now.” [link]. The problem is that GPT has completely upended the reasonable assumption that algorithms would struggle with these things. LLMs in 2023 are extremely capable of giving a convincing appearance of ‘applying expertise’, in a very broad range of topics including the development of software.

In Part 2, we explore:

1. The revolutionary shift in how AI tools like ChatGPT are redefining the way product engineering teams approach coding — from generating user stories to writing the actual code.
2. How tasks often perceived as tedious by developers, like writing tests and documentation, can now be easily and efficiently handled by AI, making the entire coding process more streamlined.
3. A deeper dive into the crucial importance of tests and the potential future of prompt-engineered applications that might begin with insightful test design.
4. A vision of the future where Generative AI tools reshape the roles within product engineering teams and the profound implications for tech teams compared to product management.
5. Stay tuned to see how these developments will impact the world of software engineering and the very essence of what it means to be a developer.

Read Part 2 here

Other articles in this series:

• Part 1
• Part 3

P.S. If you’re enjoying these articles on teams, check out my Teamcraft podcast, in which my co-host Andrew Maclaren and I talk to guests about what makes teams work.