How to use AI in an enterprise environment: Personal observations from 5 months of rapid iteration.
The end of a good hype cycle is my favorite part because the marketing pitches are mostly over, the problem space is better defined, and its time for the builders to build real solutions. Now that it has been 2 years, short a month, of Open AI’s original announcement of chatGPT, with investment more deliberate and academia more cautious , it is a great time to take a snapshot of the positives and the pitfalls of implementing AI within a company.
Here are my findings both from internal test projects and external trends. We will start high level with how to think about AI in a business, then keep drilling down to the individual contributor level before closing with some final thoughts.
Disclaimer
First off, these are my personal observations, not sponsored by any company including my current company, not even reflective of company strategy, and definitely not speaking to AI features of the company product, as the focus of this article is on enterprise use.
Second, Generative AI has moved rapidly enough in the last 2 years that this information could quickly become outdated. ChatGPT announced 4o about 2/3 of the way through our testing cycle, which drastically changed our trajectory and the focus of our final report. While I have tried to create some resilience in the timelessness of this information, there is a lot more coming to the field of AI and it could change this guidance.
3 main roles
I think Karim Lakhani’s breakdown of AI roles in business is a great starting point. ?When you use generative AI in this context, it will perform one or more of the following functions:
1.?????? Consumer (and internal) contact: I think of this as communications, internal and external, automated or manual. ChatGPT can draft content in tone, messaging, space, audience, etc. quickly, and this process can be automated (more on that later).
2.?????? Thought partner: I think of this as not only brainstorming and validating ideas, but also creative generation of content, research (glorified search engine), strategy, etc. This may be the easiest category to see gains in quickly.
3.?????? Super assistant to take out drudgery: I hinted at this in my previous article about AI – the vast majority of the work knowledge workers do every day is not in fact novel. It is primarily filled with problems that have already been solved but need a human to apply them to the circumstances. AI is well-suited to eliminate useless cycles that your company spends on problems that have already been solved and get them focused on the 20% (2%?) that makes their role valuable.
Functional areas of use
Mckinsey has gathered some interesting data which aligns well with findings we uncovered during our testing.
They highlight that marketing and sales ?are the areas where AI is most often used. I would attribute this to the combination of the vast amount of research, content generation, and message customization required by these groups to reach their target audiences. Product and service development also came in high, but I am going to not speak too much toward that point because there may be contamination of data between productivity use of AI versus developing AI features, even though generative AI is highly adept as a developer aid.
Here are some high-level areas I’ve noticed are prominent use cases. As you will see later, my caution on both the Mckinsey data and the initial categories below is to consider these as fantastic places to start but once your organization matures in AI, there are a lot of other use cases to uncover. If you want to DM me, I can share some more detailed information on some massive wins we are seeing in special applications and projects.
·???????? Customer or contact research
·???????? Content creation
·???????? Data aggregation and analysis
·???????? Coding, development, or API use.
The three AI categories that compose an enterprise application portfolio
When looking through the eyes of an application portfolio manager, ?IT manager, or business software manager, I see three strategic categories to track.
1.?????? AI-embedded applications
These applications have a main purpose, but have added (embedded) AI features that may assist with that purpose. These can help in limited ways with certain tasks in that application, but the majority of these AI features are unfortunately motivated by either upstream shareholder demand, or downstream sales and marketing requests to generate buzz and interest. As such, many of these are gimmicky and only useful in limited circumstances as they were not focused primarily on creating value for the user, but rather on creating hype for the company. These need to be tracked, particularly to understand how your data is being processed by the 3rd party’s underlying model, but this is the lowest investment and usually the lowest ROI category.
2.?????? AI-centric applications
These applications differ from ai-embedded in that their main value-add is built around an AI use case. The company will refer to themselves as an AI company and if you took out the AI function, the rest of the software would be worth very little.
AI-centric applications excel in taking an industry standard, a well-defined use case for AI and doing the work of defining and refining the use case so you don’t need to. There are solutions that will optimize the sales process, particularly customer interactions, serve customer experience, or internally sort and aggregate data. The advantages are that you can purchase a lot of time and expertise quickly by adopting a pre-made solution. The cost comes in flexibility and in customization. This is the common build vs buy decision. ?when compared to general purpose AI, and the factors for making that decision are changing rapidly as generative AI solutions establish themselves in the market. Currently, if capital is unconstrained, and AI adoption in the company is low, AI-centric solutions offer relatively low investment for a great ROI.
3.?????? General purpose AI applications
General purpose AI applications are going to the source; the base AI model for solutions. This is currently almost always a generative AI model. Generative AI models are trained on truly massive amounts of information and can be extremely useful at the individual, team, and enterprise level. Advantages are that it can be flexibly deployed and used up and down the workstack. Disadvantages are that functions and customization are not pre-programmed: you bear the burden of defining use cases, techniques, customization and automation to get results. This can be resource intensive. The highest investment and highest ROI can be found here when done properly.
领英推荐
The AI-centric vs General Purpose AI buy decisions will be an area I watch closely over the next couple years. I expect the market will resolve into a co-opetition type landscape where each type finds their comparative advantages across business tasks.
The 4 stages of AI maturity of a user
The 4 stages of AI maturity in a user focus almost exclusively on LLM-type AI models.
When first introducing ChatGPT or other llm, there is a tendency for some business leaders to let the user make do on their own, and then judge the outcome based on uncoached, uninvested experience. This is obviously a serious error. We can conceptualize the journey as 4 stages of investment and integration that yield increasingly large ROI.
Stage 1: the untrained beginner.
A user can spend a lot of time on an llm without marked improvement. With some diligence, they will slowly learn what works and what doesn’t but this can be a painful road in terms of getting results, and it depends much on what the user brings to the table with them previously, not their actual capacity to use an llm, to get results.
Please don’t do this. Don’t stop here. It gives you misrepresentative results and puts a bad name on a tool that can remove so much needless toil from workers.
Stage 2: trained and enabled.
With basic training, or a lot of trial and error over time, I roughly estimate that you can get a 3-5% productivity increase generally, and higher increases for certain tasks. Basic training includes prompt engineering like the 6 elements to consider when building a prompt, instruction on fundamental capabilities of a GPT model, and sharing in groups the success through brainstorming and success stories (this is applied, tested knowledge)
Advanced training can accelerate individual contribution considerably. This includes using csv and data models as both input and output, setting rules and guidelines for prompt output, and prompt-stepping to get results better tailored to the user's situation.
Advanced training requires heavy use of company data. It is strongly advised to get a license agreement that protects that data from AI model training. If your users can’t use company data, then you are seriously crippling your upside.
Stage 3: Customized, department-wide results focus.
Stage 3 and stage 4 are interchangeable in order of implementation, but I estimate that stage 3 is easier to obtain for most companies. It also flows more naturally from advanced training.
The keyword of Stage 3 is customGPT. Customization is a bigger picture than just creating a customGPT but this is the tool that best captures what you are doing to make Stage 3 a success. When you create a marketing pitch, you want it in your company’s tone and your company’s voice. You don’t want it in 300 bazillion other companies’ voices all munged together. You want it in your voice but with the talent of 300 bazillion other companies. Customization allows you to specifically train and follow guidelines and guardrails in a replicable manner for a certain type of task. Because it is replicable, it can be developed and used by groups of people with similar needs in your company overtime. In effect, you took your productivity tool and replicated it as many times as you desired across teams, departments, or even divisions. If you do it right, it gets even better. Using the previous example, you crystallized the ideal company spokesperson into a persistent tool that can be used by anyone you give the model to, no matter if they have been in your company 2 weeks or 20 years.
20 years’ experience from the hands of even the newest intern. That’s powerful.
Stage 4: Automation
Stage 4 is under-discussed, but it will become a major point when AI use grows to fill departments with everyday use.
You wouldn’t believe how much time it takes to transfer elements of a prompt into ChatGPT, get the right output, then copy that output to out to the various data entry points you need for your job. When average apps required to do your job hit double digits, the friction of moving focus and content between them is a silent killer.
Stage 4 eliminates that. Automating flows between applications not only lifts the cognitive load of context switching and the manual time wasted of copying and pasting information, it also allows properly implemented AI to handle tasks from multiple sources, and guide both insights coming out and inputs coming in.
Automated, correlated, specialized AI trained correctly that lifts the burden off users is what allows single contributors to generate $3m/year in revenue . How would you like to get that return on each of your employees?
UPDATE: Remember how I said that things can move really rapidly in this space and my info may already be outdated? Well it is. It happened between my first draft yesterday morning, and publishing today. Claude's new computer use feature promises to make stage 4 much more attainable, much faster, for many tasks. While stage 4 also covers system, network, and workflow use, this is a massive leap in the right direction.
Pitfalls in implementation and in use of AI
Final thoughts and conclusions
There’s 5 (anonymized) months of learning from the trenches of rapid experimentation for you. Remember that AI, as with all technology, should be purposed to make us more human. That can be really hard to remember when we have so many ways that technology strips the human element out of it. AI is the knowledge worker’s super power. Enable them so they can make the work happen.
?