The Enterprise AI Layer

Deconstructing How AI Scales Across The Enterprise

For some time, I’ve been working on and thinking about how AI innovation and integration will play out, specifically within the ample enterprise space. As I have illustrated in the past, I still believe we are nearing the peak of the “AI hype cycle,” which, ultimately, will be good for the business world as better and smarter use cases combined with a focus on data strategy and good old-fashioned change management will be the things that move the needle over time. Part of the confusion regarding how we’ll leverage AI within the enterprise stems from some enterprise tech companies, who often compete against each other in an AI Arms race. If you were to stick your finger up in the air at this very moment, you’d see that players such as Microsoft and Salesforce are trying to out-position themselves to own the “Agent” space:

"Microsoft is launching a set of artificial intelligence tools designed to send emails, manage records and take other actions on behalf of business workers, expanding an AI push that intensifies competition with rivals like Salesforce. The Redmond-based software maker said Monday it would roll out 10 “autonomous agents” to complete tasks on behalf of people in areas such as sales, customer support and accounting. The agents will be available in “public preview,” beginning in December and continuing through early 2025."

However, for companies whose customers invest millions and billions of dollars into these large enterprise tech solutions, the responsibility to make sense of this world lies at the organizational level and requires a tech-agnostic approach. To compound the complexities and intense competition happening at the enterprise tech company level, enterprise CIOs are grappling with four key challenges, according to a recent survey of nearly 500 CIOs conducted by Gartner. The challenges outlined in the study are:

1. Uneven productivity gains

2. Spiraling AI costs

3. Governance and employee skepticism

4. Adoption

The Enterprise AI Layer

The above challenges aren’t hugely unique to AI, which is the latest example of arguably one of the most consequential digital transformation initiatives to hit the enterprise. Still, it is the “horizontal” nature of AI and the fact that it can potentially impact multiple functions of an enterprise. To help establish a high-level foundation, I’ve been working on a visual representation of what the layer looks like:

Key to the Enterprise AI Layer is how it breaks down, which may be subject to change given the dynamic nature of the space. Still, to my earlier point of the AI race and the fierce competition to own the “Agent” space, there needs to be some level setting about how the layer may break down. I believe this breakdown is the following:

Assistants: This is closely linked to the now familiar LLM construct that Open AI introduced. Microsoft first introduced an enterprise-friendly version of ChatGPT in the form of Co-Pilot. Working with an AI assistant in the enterprise context is a human-directed experience, augmenting the employee's efforts and enhancing productivity. Assistants do not perform autonomous functions or tasks the way an employee would.

Agents: This less-defined and emerging category is distinctive from assistants in that they are primarily built to accomplish tasks autonomously, taking the human out of the loop as much as possible. Agents are more closely and directly linked to automation because they are task-focused and envisioned as an ecosystem where agents can collaborate and work with other agents (agentic systems). It’s very early days here.

ELMs (Enterprise Language Models):?This is a separate category that, theoretically, could blend aspects of both assistants and agents. I carve this out separately because I believe (strongly) that there are enough compelling reasons for some organizations to invest in building (and owning) a language model that preserves not only the highest level of data security but also control and access to the data, including how users interact with it. All the big tech platforms are learning from what data gets inputted into them and how end users promote the LLMs. This “prompt data,” I believe, will become more and more valuable over time. An ELM can be built to integrate the best parts of an existing third-party language model while retaining access, control, and security over all data (and analytics). McKinsey’s Lilli is an early example of an ELM, and subsequent models will become more robust over time.

Invisibles: AI features are actively incorporated into many software platforms, including those used at the enterprise level. Over time, these AI-powered features, add-ons, etc., will become “invisible”—baked seamlessly into existing software suites. Microsoft and Adobe are already doing this, and in this capacity, many AI features will ultimately become pervasive.

Intelligent Tech Stacks: Beyond the breadth of the horizontal Enterprise AI layer, each function of an organization is or will be building/evolving its “intelligent tech stacks,” i.e., the bespoke and specialized solutions that help the functions do more effective marketing, or accelerate product innovation. Each function has specific needs, and the function level's AI integration must reflect that. For example, custom language models built for particular functions could be part of these tech stacks, given the emerging specifics and unique use cases. In the past, I’ve talked about using language models as part of the communications function, and that vision turned into reality when an executive from the healthcare sector asked to have one built for their team. That tool and others like it become a part of the intelligent tech stack per organizational function.

Thinking of AI horizontally (across the enterprise) and vertically (on the business function level) and coordinating the two is ultimately how AI will scale across the enterprise. It’s also possible that new categories will emerge or be re-defined, i.e., today’s assistants may become tomorrow’s agents, or tomorrow’s agents may become something else entirely. One thing is certain regarding the large enterprise space—the biggest of ships will turn slowly (but steadily).

Visually Yours,