Foundation model debate: Choices, small vs. large, commoditization
Foundational model debates--large language models, small language models, orchestration, enterprise data and choices--are surfacing in ongoing enterprise buyer discussions. The challenge: You may need a crystal ball and architecture savvy to avoid previous mistakes such as lock-in.
In recent days, we have seen the following:
Now that's a lot to talk about considering how enterprises need to plow ahead with generative AI, leverage proprietary data, and pick a still-in-progress model orchestration layer without being boxed in. The dream is that enterprises will be able to swap models as they improve. The reality is that swapping models may be challenging without the right architecture.
Will enterprise software vendors use proprietary models to lock you in? Possibly. There is nothing in enterprise vendor history that would indicate they won't try to lock you in.
The crystal ball says that models are likely to be commoditized at some point. There will be a time when "good enough" is fine as enterprises toggle between cost, speed, and accuracy. Models will be like compute instances where enterprises can simply swap them as needed.
It's too early to say that LLMs will go commodity, but there's no reason to think they won't. Should that commoditization occur, platforms that can create, manage, and orchestrate models will win. However, ?there is a booming market for models for the foreseeable future.
AWS Vice President of AI Matt Wood noted that foundation models today are "disproportionately important because things are moving so quickly." Wood said: "It's important early on with these technologies to have that choice because nobody knows how these models will be used and where their sweet spot is."
Wood said that LLMs will be sustainable because they'll be trained in terms of cost, speed and power efficiency. These models will then be stacked to create an advantage.
Will these various models become a commodity?
"I think foundational models are very unlikely to get commoditized because there's just so much utility for generative AI. There's so much opportunity," said Wood, who noted that LLMs that initially boil the AI ocean are being split into prices and sizes. "You're starting to see a divergence in terms of price per capability. We're talking about task models; I can see industry-focused models; vertically-focused models; models for RAG. There's just so much utility and that's just the baseline for where we're at today."
He added:?
"I doubt these models will become commoditized because we haven't yet built a set of criteria that helps customers evaluate models, which is well understood and broadly distributed. If you're choosing a compute instance, you can look at the amount of memory, number of CPUs, cores and networking. You can make some determination of how that will be useful to you."
In the meantime, your architecture must ensure you aren't boxed in as models leapfrog each other in capabilities. Rapid advances in LLMs mean that you’ll need to hedge your bets.
AWS’ approach to generative AI and “primitive services”
Amazon CEO Andy Jassy delivered his annual shareholder letter and spent a good bit of space on AWS and its approach to generative AI. More interesting may have been Jassy’s talk about delivering “primitive services” and how they guide Amazon’s approach to its business. Primitive services are discrete building blocks that can be composed to solve customer problems.
This primitive services approach gives Amazon some future-proofing since conditions frequently change. Composable services enable you to adapt better. Regarding AWS’ approach to generative AI, Jassy said:
“What we’re building in AWS is not just a compelling app or foundation model. These AWS services, at all three layers of the stack, comprise a set of primitives that democratize this next seminal phase of AI, and will empower internal and external builders to transform virtually every customer experience that we know (and invent altogether new ones as well). We’re optimistic that much of this world-changing AI will be built on top of AWS."
Google Cloud Next 2024 wrap
Google Cloud Next featured a bevy of meaty announcements, but the main takeaways are that Vertex AI is now a platform for model building and orchestration and BigQuery is the data stack. Google Cloud CEO Thomas Kurian walked out a series of Gemini-powered Agents for various business functions and use cases. Meanwhile, Constellation Research Chief R "Ray" Wang roamed the show floor doing interviews.
领英推荐
·? Google Cloud customers we’ve covered include: Uber, Home Depot,?Equifax?and?Wayfair.
From our underwriter:
? ? ?
NOTEBOOK:
?? Meta launched its next-generation training and inferencing processor as it optimizes models for its recommendation and ranking workloads. The second version of the Meta Training and Inference Accelerator (MTIA) highlights how cloud hyperscale players are creating their own processors for large language model (LLM) training and inferencing.
?? Intel launched its Gaudi 3 AI accelerator with availability to OEMs in the second quarter. Intel is banking on Gaudi 3 to grab AI workloads and take some share from Nvidia.
??? Turnitin, which offers academic integrity detection software, launched an AI writing detection feature a year ago and analyzed more than 200 million papers since. More than 22 million, 11% of the papers, have at least 20% AI writing present and 6 million (about 3%) have at least 80% AI present.
??55% of consumers said they are willing to spend more money for a customized experience, according to Twilio's 2024 State of Customer Engagement Report. Almost 7 in 10 Gen Z and Millennials would abandon a brand that doesn't deliver personalized experiences on their preferred channel.
?? JPMorgan Chase CEO Jamie Dimon has economic concerns, but is certain that AI projects pay for themselves. Dimon also outlined the bank's cloud strategy in a shareholder letter.
?? CXOs in Constellation Research's BT150 talked about data lakehouses and small language models vs. large language models.
?? Deloitte Digital expanded its Apple practice and launched Academy for Apple Vision Pro, which will provide a series of one-week courses for engineers, product managers and business leaders to accelerate Apple Vision Pro spatial computing implementations.
Like getting our weekly round-up??Subscribe to Constellation Insights?for consistent news and analysis on technology trends, earnings, enterprise updates and more.
Want to work with Constellation analysts??Our Sales team would love to get in touch!