Our thoughts on GenAI - Part 1: Path beyond initial hype

Foreword

As other enterprise-focused investors, we at Ideaspring Capital are closely watching the GenAI phenomena as it unfolds. Having interacted with numerous founders and practitioners building in GenAI, we are closely observing the themes which founders are building on - best represented by horizontal sales & marketing tools, AI data analysis, AI code-gen, RAG-enablers, AI security and other vertical-focused plays.

As the field of GenAI matures and the early hype subsides, we see clear signals emerge. Well productized applications that go beyond single shot prompting and employ agentic architecture under the hood are quickly emerging as the new standards to adequately solve for enterprise use-cases. Likewise, the ubiquity of AI agents will necessitate improvements in the developer experience of building such applications, and one can reasonably concur that continuous innovations will keep happening in this realm. We are particularly keen on the emerging dev-tools to help equip AI agents with capabilities such as planning, memory & tool-usage. Additionally, tools which can help secure end-user interactions with agentic applications, simplify the development experience and democratize access to agentic AI for wider segments of the market are going to play a significant role going forward.

Having touched upon the dominant themes that we are closely focusing on - we are starting this series of articles to take a closer look at the headwinds that practitioners are currently facing & how it’s being mitigated (focus of the current article), theoretical foundations of agentic AI & it’s use-cases in the enterprise context; which will be focus of our subsequent articles.

2023: A tale of discontinued PoCs

Ever since the release of ChatGPT in late 2022, enterprise leaders & CXOs have been keen to implement the magical technology within their stack to deliver 10x differentiated customer experiences via their products. While much capital has been allocated by enterprises to tinker around with GenAI - the funnel from pilot to production has been particularly narrow. Some stats which help illustrate the problem:

- According to a survey of Enterprise Leaders by Insight Partners, only 27 % of GenAI projects are in the monetization phase (source)

- A Gartner study predicts at least 30% of all Pilot projects in AI will be abandoned by end of 2025

Pain points cited by practitioners

Complexity in handling the infrastructure stack

The visual above is representative of the sheer number of moving pieces that practitioners need to stitch together to power their GenAI applications. Given the relative infancy of the GenAI field, best practices are yet to be established and the cambrian explosion of tools is prompting practitioners to continuously tinker around with new components within their stack thereby unsettling a system which is already precarious

GenAI is not always the right tool for the problem

Many problems don’t require the capabilities of a Large Language Model trained on internet scale corpus data to deliver noteworthy results. While the enthusiasm to tinker around with LLMs and to apply them for problem solving is understandable; leadership teams within enterprises need to construct effective rubrics that guide practitioners to pick the right problems to approach with GenAI. For example, in their insightful blog post Swiggy’s GenAI team mentions how effectively prioritizing the large volume of inbound requests was paramount for successfully tackling relevant problems that actually yielded tangible results for the organization. They also mention an org-wide rubric they developed to scope out problem statements for GenAI focus:

Similarly, every organization needs to develop a well thought out strategy for prioritizing the right problems with GenAI

Reputational damages can be real

We’ve all been frustrated when ChatGPT goes off on its own tangent and does not respond to our prompts satisfactorily. Similarly, we’ve all chuckled when sometimes LLMs cannot seem to solve simple math questions like comparing two numbers and answering which one is larger or when it gets simple addition / subtractions wrong. While such hallucinations though hurtful to the degree of confidence users have in LLMs are still manageable; harmful hallucinations like fabricating legal briefs or spouting factually incorrect scientific answers greatly escalates the gravity of hallucinations. Enterprises are wary of the negative PR generated by incidents like the Air Canada case (an LLM-powered chatbot led a customer to buy another ticket under the false premise that the customer would be refunded later) and understandably want to minimize such reputational risks

Adoption of GenAI point solutions stare at the same brick wall as SaaS

Savvy SaaS operatives have known for a long time that tools which disrupt existing workflows are very hard to get adopted within enterprises and this remains true in the realm of GenAI as well. GenAI as a solution remains greatly effective for generation and synthesis tasks and can greatly aid verticals like Legal / Compliance / Insurance / Sales where document synthesis and constrained content generation tasks are aplenty. Successful implementations should focus on replicating the existing user journeys rather than expecting users to jump around interfaces to interact with their point solutions. Hence, continuing on the learnings of pre-LLM SaaS will be crucial

Buyers still carry SaaS fatigue from the pre-LLM era

The well documented phenomena of SaaS sprawl has made enterprises far more prudent and careful when approached with new GenAI tools. The projected budgetary cuts have made their effects felt on vendors, and many companies are now scrambling to move away from subscription based pricing and shift into outcome based pricing models. Given such headwinds, GenAI developers face an uphill task convincing enterprises to loosen their purse strings

Thus, we can rationalize that Foundational models by OpenAI/ Anthropic/ Meta, etc. though greatly impressive in their own right, cannot alone actualize the promises of great productivity leaps

The journey towards overcoming these hurdles

Developers & policy advocates are focused on mitigating the above discussed challenges and take us towards the much awaited AI-led future; some areas that are currently gathering steam:

- Research and funding for AI Safety and Governance is ramping up across the globe. The UK has dedicated 100M GBP for safe adoption of AI; a new body dedicated towards AI safety has been set-up in the US (source); while the EU has drafted extensive documents and guidelines to aid safe and ethical use of AI. Multiple startups focussed on LLM guardrails (filter for model input & output) and model explainability are cropping up. Further increase in activity in these areas can be anticipated

- Emergence of solutions which are spread across adjacencies & cover more touchpoints across practitioners’ workflows. Products tightly fitted to complex workflows and focused on wider problems within the value-chain are gaining relevance. They help mitigate the problem of SaaS sprawl, allowing enterprises to consolidate their vendors and are the easiest to adopt amongst knowledge workers

- To increase the fidelity of outputs, developers are increasingly coupling foundational models with advanced orchestration techniques to unlock performance gains that are significantly greater when compared to throwing larger models at the same problem. Hence, these orchestration techniques, also known as agentic workflows, are quickly gaining momentum amongst the developer community and the broader ecosystem. It’s becoming apparent that agentic architectures will become synonymous with GenAI applications and these will get established as the new defaults for building post-LLM era software

Agentic experiences offer a glimpse of the AI future and we believe the development of this field will lead to the actualization of AI’s true promise. In the subsequent article; we will delve deeper into the theoretical roots of agentic AI, discuss some seminal papers in the field and develop our intuitive understanding behind AI agents

In the meantime, we would love to hear your thoughts. Please reach out to us over email to further the conversation or to pitch us your newest venture in GenAI - we are available at [email protected]