Will ChatGPT take over my job?

Introduction

ChatGPT is the fastest growing consumer app in history having reached 100 million active users in just two months after its release in November 2022.

In this post I wanted to learn if ChatGPT can provide value to some of the fundamental aspects of product development for product managers. Using a real-world test, let’s find out how this global phenomenon performs and if it's capable of taking over my job!

What is ChatGPT?

In simple terms, ChatGPT is a chat bot that uses artificial intelligence trained on 570GB of datasets to generate accurate and human like answers to users’ commands such as a question or task. Given the power of the service, it’s now being integrated into a variety of commercial use cases. For example, Klana has embedded ChatGPT within its own app to offer a highly personalised and intuitive shopping experience by providing curated product recommendations to users who ask the platform for shopping advice, along with links to shop those products via Klarna’s search and compare tool.

The experiment

Product managers love to find customer problems and build novel solutions that generate value for customers and the business. The real-world problem I chose for this experiment is that there are many older people who suffer from loneliness whether it be from the passing of their partner, disability or other factors.

I wanted to test if ChatGPT could deliver across four chosen aspects of the product development lifecycle:

1. Discovery – what is the size of the opportunity?
2. Ideation – what are potential solutions that could help to solve this problem?
3. Requirements – what features would the chosen solution need to be successful?
4. Product launch – what would a successful product launch plan look like?

Discovery

Question: What is the number of lonely elderly people in the UK?

Result: The answer was clear and contained a useful data point that was referenced to a reputable source (pre-pandemic there were 1.2 million chronically lonely older people in the UK according to the charity Age UK).

A drawback is that ChatGPT has limited knowledge of world events after 2021 which may have been why it wasn’t able to provide the most up to date data.

Score: 6/10

Ideation

Question: What products could help solve the problem of chronic loneliness of older people?

Result: I was hugely impressed by the breadth of ideas and summary explanations provided. ChatGPT provided 7 suggestions ranging from technical solutions including social robots and video calling devices to social solutions such as pet therapy and support groups.

It would make an excellent starting point for more in-depth research on the ideas. Top marks could have been awarded if the ideas were given some order of priory e.g., ranked in order of effectiveness.

Score: 8/10

Requirements

Question: What are the product requirements for a social robot mobile app to help lonely older people.

Result: ChatGPT really excelled itself here. In addition to more obvious features such as reminder tools and entertainment, it went further, suggesting an emotional intelligence requirement so that the bot could recognise and respond to the emotional state of users. This shows a real understanding of the problem, use cases and the end user persona.

Although the ability to constantly update the app was suggested, there was a lack of non-functional requirements such as requiring the app to be published on iOS and Android app marketplaces.

Score: 9/10

Product launch

Question: What would a product launch plan be for this app?

Result: ChatGPT quickly suggested a comprehensive product launch framework for a generic mobile app but unfortunately it couldn’t provide specific details for my social robot app.

Score: 5/10

Pros and cons

Pros

• Lightning-fast research tool for initial market discovery providing top level data points and sources
• Superb for ideation for how to solve real-life problems
• An excellent tool to gather a highly relevant starter for ten set of product requirements that can later be enhanced / refined / validated
• Good at detailing templates and frameworks for product launches

Cons

• Data may not always be up to date and difficult to check accuracy
• Responses are not framed in a way that adds greater value e.g., prioritising a list of ideas or requirements by a useful metric
• Non-functional requirements were mostly absent from the initial response however perhaps if I asked ChatGPT to include these, they may have been added
• It can’t produce a detailed and comprehensive launch plan for your specific product

Conclusion

Although this experiment was a bit of fun and not scientific, it did produce some useful findings:

1. Asking the right question is key. The more specificity in your question, the more relevant the answer is likely to be.
2. ChatGPT gives you an incredible speed and quality balance. Responses still need human interpretation and it won’t make those critical product decisions for you.
3. It's valuable to break those blank sheet of paper moments and give you a head start or a framework to work within.

Overall, I’m confident that product managers are safe from ChatGPT.. for now!