On the Heaviness of Being AWS

Author’s note: The views in this article are mine alone and do not represent those of any company that I am associated with. Also, I write these articles; they are not written by an AI.

Everyone was wondering if Amazon at ReInvent 2023 would change the GenAI landscape. They invested in OpenAI’s arch-rival Anthropic just 2 months before the event.

On November 28, at Amazon ReInvent, AWS’s big GenAI invention announcement was Q.

Much of the rest was a classic AWS explosion of a vast array of tools for the entire?GenAI technology stack.

Throughout, AWS's new CEO Adam Selipsky repeated the message that Azure was not unseating AWS just because Azure has OpenAI with ChatGPT. Protesting a bit too much?

Is Q the next big thing after ChatGPT, but for enterprises?? And can we make sense of why AWS has to repeatedly say they're still the best?

The Pattern Was Clear Last Summer

In June and July this year, I published four articles to offer a plain language perspective for business leaders on generative AI as ChatGPT captured everyone’s attention. I also wrote one on the impact of GenAI on society.

In the fourth business article, I asserted that the overall market dynamics were pretty clear and that the next checkpoint would be when AWS announced its play at ReInvent 2023. (As expected, we are still waiting on Apple.)

Outside of not predicting the temporary firing of Sam Altman from OpenAI, this assessment was largely correct.?

With AWS ReInvent behind us, let’s see where we are.

One or Maybe Three Uses Cases Above All

My first article argued that the strongest effects of GenAI were on accelerated software development and then on software applications benefiting from new features powered by GenAI inside.

This is correct. The most powerful effect so far has been LLMs’, especially ChatGPT’s, amazing ability to write, fix, and make sense out of software code. At the center of it, Microsoft, by owning two of the most vital developer tools - Visual Studio development environment and GitHub code repository - has taken full advantage of the GPT4-powered copilot strategy.

In effect, every developer is going to have one or more copilots helping them do better work much faster - to make sense of existing code, get new code right, do better annotation, test for security, and build tests and test data, etc - from now on.

Software development is the only area where ChatGPT has reshaped the macro competitive landscape - so far.

In fact, only two other major use cases have dominated:?

One other is generating marketing copy and marketing images--marketing content as it's commonly called. This is going on everywhere. Adobe and Salesforce - already the industry leaders in content marketing and sales-driven marketing - continue to offer AI-differentiated enterprise products.?

More companies are generating more blogs and articles because it’s easier. But this has not changed any other competitive dynamics. Many believe it normalizes not-very-insightful content that swamps clear thinking and exposition

The second other is document summarization. The most visible versions of this are chatbots that have read a company’s entire set of manuals and policies and can answer questions much better than before. This is great for technical products, complex financial products, and legal and medical too.?

Note that ChatGPT's abilities for reasonable summarization have destroyed the idea of book reports and essays in education - that’s a whole other topic.?

There are myriad GenAI startups. People here in the Bay Area are talking about developers moving back to San Francisco so as not to miss the wave.? And there are lots of tantalizing stories about improved drug and treatment discovery in accelerated pipelines on top of what predictive AI is already doing. I hope so.

McKinsey and many others are profiling and quantifying use cases for every function in every vertical.

But outside of Microsoft disrupting Google search and putting AWS on the defensive, so far, we see no major changes to competitive dynamics among big businesses.?

Little Re-Engineering So Far

That first article of mine also argued that the winning companies would go beyond benefiting from GenAI’s individual-productivity boost and figure out how to re-engineer GenAI-affected business processes for increased production and cost reduction that would create real competitive advantage.

This has so far turned out to be the exception.? Microsoft becoming the copilot company is the best example. There are media companies that Nvidia highlights that let GenAI create all the background scenery in animations and films so they can produce a lot more in less time with many fewer people.

The rule, however, is that each empowered worker or computing workflow is improving his/her work and speed by using GenAI for personal productivity within the same business processes as last year. Multiple speakers at ReInvent reported getting more done faster - not differently.

The reengineering effect more broadly will take longer and may require a push from startups as in the media example.? Disruptor companies that are reengineered from the start may come bottom up.

Building GenAI into Every SaaS

In my second article, I mentioned that perhaps leaders would understand the productivity they had at hand, and start asking their organizations to produce beyond what they ever thought possible before - to get beyond personal productivity.

But, I noted that perhaps most organizations would just wait for SaaS tools large and small to appear to do this for them - and for every one of their rivals just the same.

The SaaS option seems much more likely now. We see it in the horizontal functions I mentioned before - marketing and customer service. For the most part, in terms of other business functions, we are all still waiting for new examples that make this clear.

Or maybe, AWS just unleashed the next big thing.

What Has Amazon Done?

In my third article, I mentioned that Salesforce chief Mark Benioff told us all last summer that every CEO had the same idea. Load all my data into an LLM and start asking questions.? Benioff said last summer, “It does not quite work that way.”

This week, AWS introduced Q.?

Now it does work that way. You can watch a very short version here. That’s the giant Q brain in the middle.

This is GenAI doing document summarization on an enterprise scale with enterprise security and controls.?

Benioff’s announcements last summer kicked off a wave of enterprise-grade versions of all the major LLM-powered services. The vendors assure buyers that the enterprise edition doesn’t keep or learn from what users tell it. The goal is to make sure your question and answer sessions don’t provide content to some other company’s questions.??

For us individual users of ChatGPT, we know this is a risk, but we figure our threads are like throwing a few grains of sand on a beach.

Another major development over the fall of 2023 is limiting the LLM to a specific corpus of documents or data sets to summarize when giving answers. 'RAG' and 'vector embeddings' are the terms you will hear. This greatly reduces hallucinations.?

By the start of ReInvent, enterprises had stopped resisting GenAI on the grounds of security and unreliability. They were ready for Q.

The function Q is transforming first is the one that Benioff said all CEOs want:? business intelligence. It takes tasks that took days down to hours, and refinements to charts and slides down from hours to minutes.

Is this process re-engineering that conveys an advantage to those who can do that, or just improved BI??

A potential example was a preview at ReInvent of the ability of an airline to use Q to automatically reschedule all the passengers affected by a weather disruption at a major hub. This involves a huge amount of real-time company and inter-company data and interacting with passengers competently. That would be amazing.

We will be watching eagerly if Q indeed works and if we see examples of companies running away from the crowd the way Microsoft has with copilots.

Or is Amazon Building on a Flawed Foundation?

In my fourth business article, I profiled the rise of Hugging Face spearheading the open-source movement for LLMs.? Enterprises very clearly want open source competition to the natural domination of two or three tech companies that keeps happening when there is no open source.

AWS has a decade-old practice of embracing open-source applications inside AWS services.? This is what they did with GenAI this year at ReInvent.

But their formula may contain some pretty big flaws.

Is Many Better than One That Actually Works?

AWS assured us all at ReInvent that we need a huge range of choices for LLM models. They put many of them into a layer called Bedrock, with all kinds of tools to optimize which ones you use and how you use them. This is exactly what enterprises want to hear and want to be true.

But what if that is wrong??

What if only ChatGPT4 by OpenAI and maybe Claude2 by Anthropic have passed the threshold of being useful for the broad understanding required for services like Q including not hallucinating so much?

Until last week, Bard by Google pretended it was competitive using their best LLM. Then they announced a new LLM, Gemini, to try to reach the threshold.??

As much as we all hope that is true, don’t hold your breath. In a press conference, Sissie Hsiao, vice president of Google’s AI chatbot, Bard, said Google’s most advanced model, Gemini Ultra, beat GPT4 in just one of the eight benchmarks.

That’s it for contenders for OpenAI’s leading position. Nobody but the cloud giants themselves have the resources to create one of these “foundation models” as AWS called them. You might call them models that deliver the goods. And even at that, they can’t quite match OpenAI.

AWS made a big deal of its investment into Anthropic and their late-in-the-game inclusion of Claude2 in Bedrock.

Let’s face it. We’ve all learned that GPT4 is simply better. If you go to an AI developers conference, those who know best will say you have to develop and test on GPT4 to make sure it works even though that is expensive to do. Then you try to make one of the open-source models work for cost reasons by limiting it to those specific tasks in your app where you can limit what data the LLM has to pay attention to.

Let me be clear. There are many well-defined tasks within applications where a lesser LLM can restrict the data it uses to answer questions relevant to that application and do just fine. The future of AI and GenAI-powered features within applications is bright. The many tools that AWS announced will be useful for creating these features.

The question here is whether any LLM besides GPT4 is good enough for the bold vision of Q.

Claude 2 is better than all the rest by a fair margin, but still not as good as GPT4.? Azure has GPT4 and AWS doesn’t.

It’s Only Useful If You Can Use It!

The second potential flaw is that the world of AWS is byzantine. The builders who know how to navigate AWS have access to amazing new tools.

But wait! Amazon Q is only available so far to those who already use Amazon Connect for call centers. Not all AWS users.

It is not for everyone else in the AWS world yet and it is certainly not for end users.?

Just ahead of Reinvent, OpenAI tried to make several moves to appeal to the kinds of builders whom AWS counts on.?

They cut the cost for developers and applications using GPT4 through a program interface (API) - ie in the background of another application - dramatically. They also expanded its input window to be more like Claude2. And they now let you create a persistent thread - your own GPT4 - with the coveted enterprise data security feature included.

They don’t have to do much else. With GPT4, Azure is way ahead as the copilot company. Visual Studio and GitHub co-pilots are fully and amazingly powered by GPT4 and very accessible to all developers. And ChatGPT makes GPT4 available to everyone else.

This is why AWS is nervously insisting they’re still #1.

Conclusion - OpenAI Still Defines GenAI

OpenAI in the hands of Azure remains the shaping force not just in GenAI but in the software world. AWS’s dominance in cloud services and ability to create hundreds of really useful tools is likely not enough to counter Azure’s position.?

Claude2 keeps AWS in the fight. Google has not figured it out yet - we will see with Gemini. (Try Bard this week and see if it still hallucinates prolifically.) None of the other models are there. CIOs and CTOs everywhere are pulling for there to be a real choice; many open-source choices ideally.

These other models might be just right to do particular tasks within particular applications that developers are building. But you cannot trust them to make sense broadly and generally of your whole collection of enterprise data combined with knowledge of everything else - as in Q.

AWS’s Q as an approach is what all businesses want. But it's not ready for everyone to use. This gives Azure time to build and offer the same service - they have all the ingredients - and ChatGPT 4 will be better at it.

Azure’s version of Q and proof points from enterprises that AWS Q actually works are what to watch for next.?

Meanwhile, we are all wondering if GPT5 is going to retain the special advantage of GPT4 and/or be something else altogether.? Sam Altman mentioned GPT5 and the road to artificial General intelligence to conclude his first OpenAI conference in early November. This may have given his board of directors a fatal anxiety attack. For the rest of us, it means we will keep watching OpenAI.