LLM AI Considerations for Product Managers
OpenAI's ChatGPT service has captured everybody's imagination and has become an overnight success. It is giving average citizens, corporate leaders, federal regulators, academic researchers, and overzealous entrepreneurs lots to talk about and possibly, a whole lot of anxiety.
Often its text quality is so superior that it can be hard to tell whether it was written by a human or a machine.
Until the arrival of ChatGPT, the typical AI experience for the average consumer was in the form of better search results, finding the right family/friend photos, getting improved recommendations (for all kinds of things), and staying on top of the weather forecast. All of these experiences can be quickly executed, consumed, and are reasonably "objective" (i.e. black 'n white) in terms of interpretation. That is definitely not the case when it comes to ChatGPT, which is an LLM (large learning model) AI product.
An LLM is a deep-learning algorithm trained on enormous amounts of text data.
Generative AI (which ChatGPT is an example of) and LLMs have been around for sometime. Remember Google's LaMDA? Nonetheless, ChatGPT has broken away from the pack and literally defined a new "category" of AI products. Every other LLM AI product is now getting compared and benchmarked against ChatGPT! This new category will drive a whole new wave of investments, both from VCs and in large corporations. We will see a whole new breed of entrepreneurs entering or pivoting towards building or consuming Generative AI services and products.
For the Product Managers, who are brave enough to go where only a few PMs have been, what are the considerations? Implications? Cautionary notes? Fine print? If you look at my "LLM AI PM" visual, PMs have already mastered 3 out 4 quadrants. However, mastering the "LLM Domain Knowledge" quadrant that will pose the great challenge and provide the greatest opportunity.
Here are a few quotes I have read from leading AI voices on the maturity and efficacy of current LLM products -
领英推荐
LLMs don't have common sense.
LLMs are useful but they make stuff up.
ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness.
Clearly, LLMs have a long way to go in terms of explainability and fairness. However, here are my proposed 5 key considerations for PMs as they take on the management of LLM AI products -
The above set of questions are merely a starting point. It will be difficult to assess and make value judgments about something whose full range of capabilities are still unknown. Also, it will be challenging to control possible malicious uses, such as spam, fake news, automated bots, homework cheating, etc. Successful LLM AI PMs will make the time to do their homework on LLM domain knowledge and/or ensure that there are LLM SMEs allocated and available to help guide them through answering all of these important questions.
Having access to some philosophers and social scientists may be handy too.
P.S. This article was written by a human. ??
You articulated this very well, Uday. Thanks for your insights here.
AI Strategist Driving Sustainable Innovation
2 年This an interesting article! By the way, GPTZero thinks a sentence may be written by a machine. But then again, it's a machine telling us about it, not a human. I believe you are more than a machine :-)
Product, Strategy, GTM, Venture Operator | MIT, IIT Bombay - Aerospace Engg | Mentor NASSCOM DeepTechClub
2 年Interesting take. Building LLM model is complex and expensive. Few companies will have the skills and budget for this. More will be able to leverage existing models and services to build their own slide of applications. We are already seeing clients consider using such services to replace their own NLP engine, cataloging engine, possibly content generation. PMs managing products that can intersect with such services need to figure out what to do with this.