A Bridge Too Far for "AI for Enterprise"

Let us look at 3 emerging stories relating to enterprise adoption of AI. A fourth last story, we will keep for the end of the article.

The first was indicated by Sam Altman (of OpenAI) and others in some recent interviews - LLM architectures’ ability to scale might be reaching the end. The initial thought process was - ‘Keep adding more GPUs, more data, more synthetic data generated by the LLMs, and the models will keep getting better - more knowledgeable, less prone to errors.’ But it seems like the Law of Diminishing Returns is now applicable, well short of the goal of AGI. Altman also worried about the world running out of GPUs and huge delays in fulfilling orders from many large organizations that are racing towards LLM architectures.

The second story is the perspective of Yann Le Cun, Meta’s Chief AI Scientist, shared during an interaction at IIT Madras, that we need to look beyond LLMs to move toward AGI. I want to remind all our readers, though, that he and his boss continue to invest billions of dollars into Meta’s Llama LLM! Le Cun spoke about the use of ‘world models’ and how this approach will be less compute-intensive but will result in better reasoning and decision-making. @techmahindra’s Chief Innovation Officer - and my ex-colleague - also recently spoke about symbolic AI and world models in his latest Linkedin Post . Gary Marcus has been talking about the futility of the LLM Race , and advising tech companies to look beyond it, for the last few years, but no one listened to him till now.

The third story is of the many medium and small size tech companies & startups raising funds (much smaller amounts compared to the investments in LLM) to build bridges that enable enterprises to effectively use these new AI technologies. The bridges could be AI Agent Builders or RAG Vector Database management systems, for instance. There are companies that are building guardrails as front-end filters to control enterprise-to-customer interactions. There are companies that have raised funding to build Hallucination Fixing Tools . They are asking enterprises to document what the truth is in many business scenarios, and they block the LLM’s responses if they are found to be outside of the ‘Truth List’!

A few years ago, a Microsoft chatbot released to the world had to be pulled back in less than 24 hours due to the model learning and using racial slurs and offensive language. I wrote a blog then recommended a ‘negative list’ of unacceptable words and idioms, in many languages and dialects. The Truth List concept seems to be along similar lines. I have also written a piece about an airline’s chatbot offering a discount to a customer outside of the corporate discount policy. The list of discount policies in the Truth List could help the chatbot filter out incorrect discount policies.

I am a big World War II nerd- I have read many books, and have watched almost every Hollywood movie and documentary on the War. There are many stories from WWII that one can pick as parallels for any scenario in today’s world.

The Western Allies landed in Normandy on July 6th, 1944 and there was general talk that the “Boys will be home for Christmas”. However, the war dragged on for 5 more months and victory was finally achieved in May 1945. General Patton was a very aggressive commander and his army was almost at the gates of the German border in early August 1944. He wanted all the supplies such as gasoline, food, medicine, and ammunition to be diverted to him so that he could be in Berlin soon.

However, the Allies’ offensive came to a halt due to supply challenges. The beaches in the Normandy area are not suitable for building large ports; the logistics of getting supplies from large ships parked out in the sea by smaller boats was very inefficient and slow. In fact, the operation Market Garden offensive planned by the Allies in September 1944, was to primarily capture the bigger port in Arnhem, Netherlands to solve the supply problems. The Hollywood movie ‘A Bridge Too Far’ is a great one based on this operation.

So, what has this got to do with the current AI tales?

We have enterprises that are stranded on an island and waiting for rescue! They would like to adopt AI to improve their business outcomes, but they are unable to make meaningful use of it. Massive trillion-dollar ships (LLMs - Llama, Gemini, ChatGPT, and others) have arrived but are unfortunately parked a good distance safely away from the shores. Smaller companies (rescue boats) are working on building makeshift bridges, temporary ports for enterprises to access these massive ships.

Let me narrate another fascinating key event that happened during the early days of WWII. The British expeditionary force, the Belgian army, and the French army were pushed to the small area near the beach of Dunkirk in May 1940. Over three hundred thousand soldiers were stranded in a small strip waiting to be rescued to the safe shores of the United Kingdom not far away. Large ships, even if made available, would have been sitting ducks for the German planes. The United Kingdom pulled off a miracle in a civilian and military joint effort, by mobilizing thousands of small fishing boats and leisure boats. They were able to rescue 3,38,000 soldiers. These soldiers were ready to fight the war in June 1944, when the Allies landed on the shores of Normandy.

Another story is more relevant for the unspoken about, or indeed completely forgotten (but highly relevant) aspects of AI. I am reminded of the old Gulzar song that goes ‘Aap ki khamoshiyan bhi, aap ki aawaaz hain’ or, loosely, ‘Your silences fill my ears, as much as your voice does’. I am referring to the good old predictive learning models for NLP, audio, video, images - these are forgotten, as if LLM will completely replace them.

I am tempted to mention an old Tamil pazhamozhi (proverb) ‘Arasanai nambi purusanai kai vittal’ - about a lady who was enamored by the king's charm during a Janta Darbar, who then left her husband desiring to be with the king, only to find there is no place for her in the palace; stranded neither here, nor there.

We at @aithoughts believe that we must not let go of these ‘traditional’ AI approaches (‘purushans’) under the spell of the LLM’s (‘arasans’) charms!

Let us now tie these stories together and figure out what it means for ‘AI for Enterprise’.

There is nothing wrong with a larger collaborative approach to solving the AI puzzle -

Big tech companies with their bank accounts bulging with billions of dollars can pursue LLM and ‘post-LLM’ (Symbolic AI, World Models) and create massive knowledge systems, perhaps inching towards the Holy Grail i.e. AGI.

Enterprises will use the big models as the first-contact user interface using voice, text, or images. Understanding the customer problem in a customer care application, or customer interest in buying a product in a customer sales application, which takes into account the ‘intent’ of the customer along with certain other key variables such as product, problem description, etc. will come from the biggies. The communication back to the customer will also be done by the LLMs. A key constraint will be the commercial cost of using the LLMs. Companies building LLMs should find ways to offer these to enterprises at more affordable rates, over a period of time.

Most enterprise problems need knowledge of enterprise business processes which are stored in many documents within enterprise firewalls. So, RAG or similar solutions are required to make the models learn the enterprise context and the specific business processes and rules which are applicable to only your enterprise.

And many AI Agents are required to orchestrate this symphony.

Take the example of what we in India call KYC - ‘Know Your Customer’. Who is the user? Which department? Which location? The designation? Various data points about the user will help the AI agents to correctly decide and navigate through the enterprise’s internal and external systems to get the correct solution. An HR agent will need to know all this, combined with all the HR policies, to take accurate action. We also need some sort of ‘state’ information and memory to store these KYC details which can be accessed by various other AI agents.

The orchestrator or the main controller agent, will be semi-autonomous for some time to come. We at @Aithoughts.Org recommend BPM++ configurable engines to automate as much orchestration as possible and not wait for fully autonomous controller agents technology to mature. For example, the agent should know that first action for any customer interaction is to Know the Customer and what systems have that information. The agent orchestrator also need to know how to sign on to the appropriate enterprise system and what APIs to be called to pull the relevant information. We are reminded of the excitement when initial launch of BPM tools two decades ago and how the workflow automation and RPA flourished using this technology.

To illustrate, let us consider the example of placing a purchase order for a supply room in a factory. The customer’s intent of placing a purchase order, the product SKU#, the quantity, etc. will be captured by the front-end LLM. The KYC Agent will identify the designation of the sales employee and obtain order quantity limits for the designation, for the required product SKU. If the order quantity is lower than the limit, the order Agent will place the order. All these steps can be implemented using Agent orchestrator by configuration now and move in to more autonomous auto configurations later.

The guardrail tools, the hallucination filters, can be deployed to review and edit the LLM responses in real time to improve the reliability and compliance of these models.

For IoT and camera-input-based applications, we still believe that video/image recognition models can provide enterprise-grade solutions. PDF-based documents and various legal, contractual, and other documents of the enterprise can still use older NLP models to get good business results.

What we need is smart orchestration of all these approaches to solve enterprise use cases.

The last story to conclude this article. OpenAI has recently announced that by Jan 2025, they will provide AI agent builder tools for Open AI users. The large shipping companies have finally understood that enterprise customers have to reach them; they cannot stay docked a mile away from the shore.

Happy to hear your comments and views.

(In part 2 of this article, we will explore RPA-APA and SaaS turning into Service-as-a-Software.)

AiThoughts.Org #AIagents #AiforEnterprise #AIROI #agenticrag