Break out moment for Generative AI

AI Models

2023 was the breakout year for Generative AI.

Funding for generative AI surged to $25.2 billion in 2023, 8X the 2022 levels. Major players in the generative AI space, including OpenAI, Anthropic, Hugging Face, and Inflection, reported substantial fundraising rounds. Additionally, enterprises have started to get serious about AI and have begun to pour money into Generative AI.

So, where is the funding going?

Industry, academia, and governments are investing in three areas.

1. Frontier AI models
2. Specialised AI Models
3. Generative AI powered AI applications

Let's first understand each of these areas at a high level and double-click on the Frontier models in this article.

Frontier Models

Frontier models are a subset of large language models (LLMs) that exhibit exceptional capabilities and performance beyond their predecessors.

They have a superior architecture, are trained on large data sets, and beat established benchmarks.

These are predominantly general-purpose models.?

They possess vast knowledge (width) and are getting better at reasoning (depth).

Specialised AI Models

Unlike general-purpose models, these models are trained on vast amounts of data specific to their intended purpose, allowing them to perform with greater accuracy, efficiency, and nuance.

Several VC-backed start-ups and enterprises are investing in this space, while tech giants are already in it.

Some examples are medical diagnosis, image recognition, and financial forecasting.

Specialized AI models are artificial intelligence models designed to excel in particular domains or tasks.

Generative AI-based applications

This is a vast space with tremendous excitement across industries.?

Architectural approaches like RAG (Retrieval augmented generation) enable financial services and other highly regulated industries to use Generative AI on their enterprise data.

Anyone can now develop practical applications around the Frontier and Specialised models.

Frontier models

Let's dig deep!

What makes the frontier models unique?

There are at least three aspects to consider.

1. Frontier models often employ innovative architectures and techniques, such as transformer-based models (Google pioneered this approach in 2018) and attention mechanisms, enabling them to process and generate natural language.
2. These models are trained on datasets significantly larger and more diverse than those used for previous generations of LLMs, which allows them to learn complex patterns and relationships in the data.
3. Frontier models consistently outperform previous models on various benchmarks, demonstrating superior capabilities in text generation, translation, summarization, and question-answering tasks.

Which players are leading the charge?

No surprises!

Silicon Valley Tech giants are setting the tone and have invested significantly over the past 24 months. They have access to the best brains on the planet and the necessary infrastructure and knowledge to develop this technology.

According to Stanford University's Artificial Intelligence Index Report 2024, the three tech Giants are currently leading the pack. These are followed by upstarts, creating enormous buzz and investments across the landscape.

Why frontier models are not everyone's cup of tea?

Frontier models are voracious.

Take a look at the graph below. This is just the training and computing cost. Talent and broad-based technical capability are also important in this area which only few institutions can manage.

Frontier models need serious investments, massive computing power, and enormous dedication to move the dial.

How do you know more about a frontier model?

Foundation models can be accessed in different ways.?2023 has been a breakout year, and we saw a doubling of these models in just 12 months.

These models adopt three patterns.

1. No access models, like Google’s PaLM-E, are only accessible to their developers.?
2. Limited access models, like OpenAI’s GPT-4, offer limited access to the models, often through a public API.?
3. Open models, like Meta’s Llama 2, fully release model weights, which means the models can be modified and freely used.?

What can companies do to join the revolution?

In the future, all enterprise applications will have some form of generative AI. Vendors are keen to embed it and showcase it as a differentiator. Departmental heads will clamor to use Generative AI to solve specific business problems. Developers would like to work on Generative AI so they feel included.

Like every wave of new technology in the past three decades, companies need to understand the implications of Generative AI for their business, strategize, and develop a plan.

Here are my 5 takeaways

1. High-quality data will be critical for training the models with organizational data. Organizations need to find a creative way to justify the investment in data infrastructure. A vexing problem that only some companies can truthfully claim to have solved.
2. Data science and engineering capability are prerequisites for maintaining high-quality data infrastructure and building ML capability. IT departments must encourage using Python and similar languages suited for data science. Java and JavaScript dominate enterprises even today due to the speed and portability they offer for application development. Programming languages like Python need to get the mind-share that Java enjoyed when the Internet came of age at the turn of the 20th century.
3. Create machine learning (ML) capability, the backbone of all AI. Companies will be tempted to see Generative AI models as black boxes. This will be a mistake. The coming of age of Generative AI is yet another opportunity for companies to get serious about creating Machine Learning capability.
4. Ethics and Transparency would be critical elements as Generative AI evolves. Vendor solutions or using an LLM in a business workflow must be free from biases and respect data privacy. Additionally, companies must be able to understand and justify the output or outcome in front of a customer or a regulator.
5. Asking for patience from leadership in this ultra-competitive and fragmented world is counterintuitive. However, AI is a new paradigm and is evolving rapidly. Only a lucky few will be successful in the short term.

Reflect on the years it took to master the internet and mobile and the ongoing struggles with digitalization. Set realistic goals, focus, and drive teams to success.

Credit: Cover Image; Research Study: Stanford's Artificial Intelligence Index Report