Wisdom is all you need (AI)

In 2017, a group of Google researchers published a paper titled “Attention is all you need”, which introduced their work on transformers and set the stage for LLM chatbots. Transformers rely on an attention mechanism that is much quicker to train, allows significantly more parallelization, and achieves new levels of translation quality.

This impressive research occurred due to the talent and motivation of the individuals in combination with what is now over two decades of persistent, ever-expanding investment by Google in NLP, ML, language models, and ‘big data’, or data collection, manipulation and repurposing at very large scale.

“Extrapolating the spectacular performance of GPT-3 into the future suggests that the answer to life, the universe and everything is just 4.398 trillion parameters.” – Geoff Hinton.

Hinton has of course subsequently retired from Google to warn the world about the risks of LLMs.

As the Big Techs grew rapidly over the past 25 years, so did their R&D budgets, inevitably reaching a point where most of the world’s R&D spend in advanced IT revolves around the strategic interests of a small group of Big Tech companies. In 2022, for example, Amazon, Alphabet, Meta, Apple and Microsoft collectively spent $2020B in R&D, with Amazon leading at over $73B followed by Alphabet at nearly $40B.

The next five biggest R&D spenders listed on NASDAQ spent only $20B, which includes Nvidia at $7.3B. For perspective, the entire R&D budget for the U.S. Federal Government in 2022 was about $180B across all agencies. 45.7% of the U.S. R&D budget was for national defense. The next largest focus area was health at 24.2%.

So, when people describe Big Techs as sovereigns (as in nations), from an R&D perspective, the tech oligopoly is already larger than the largest national economy (U.S.), and of course they invest in their strategic interests to protect moats, avoid disruption, and continue to expand, despite a scale ceiling and antitrust investigations and lawsuits. ?LLM chatbots are the latest and perhaps greatest example of extreme behavior in strategic investments by Big Tech companies.

With the launch of ChatGPT, and with Microsoft’s enablement, OpenAI triggered an LLM arms race that has taken the scale obsession to new extremes, but scale in language models creates as many or more problems as it solves. Here are a few views on scaling from AI scientists:

“You’re not going to get new capabilities like planning or tool use or agent-like behavior just by scaling existing techniques. It’s not magically going to happen.” – Demis Hassabis

“Most of human knowledge is actually not language so those systems can never reach human-level intelligence — unless you change the architecture.” – Yann LeCun

“Large-scale systems still exhibit significant?factual errors?and?unpredictable behavior?when deployed…. In our opinion, further scaling is unlikely to resolve these reliability issues. The?Inverse Scaling Prize?shows that larger models actually do worse on many well-specified tasks.”?– Yoshua Bengio

Over the long-term, I think LLM performance will improve and costs will decline sufficiently to provide an ROI for chatbots and many more GenAI apps. However, the current cost of high-scale AI is simply too high for the super majority of uses. If we look to venture capital, for example, Sequoia Capital recently estimated that VC firms and Big Techs invested about $50 billion into Nvidia chips, but GenAI startups have only realized about $3 billion in revenue. And this is before paying for copyrighted material.

“You’ve been duped”– Mark Montgomery

Now consider that GenAI margins are much lower than SaaS companies and the challenge begins to reveal itself. This is why I started saying about 15 months ago “you’ve been duped” with regard to LLMs –not just VCs, but also enterprise customers. ?The primary beneficiary for large-scale models to date is Nvidia, followed in a distant second place by cloud providers, but given the foggy accounting it isn’t clear cloud providers will make any money from LLMs.

For example, how much of Microsoft’s recent quarterly report in increased Azure sales is simply Microsoft’s own money being returned from LLM firms like OpenAI Microsoft invested in? Some VC firms who have invested heavily in GenAI ventures have said that up to 80% of their total spend is for cloud computing. Is it any wonder why Microsoft, Nvidia, Google, and AWS are the largest investors in LLM ventures?

Big tech has more discretionary capital than any other sector of our economy, and they spend heavily on consultants, banking, venture capital, M&A, lobbyists, universities, former government employees, law firms, ad buys in media, electricity, water, real estate, regional taxes, and the list goes on and on. The amount of money Big Tech spends buys a lot of loyalty, but it’s not necessarily aligned with the best interests of customers.

Indeed, the larger Big Techs become, the more direct conflicts they will have with customers. It’s the inevitable result of growing too large, resulting in the scale ceiling. As a very early booster to most of the Big Tech companies who has followed closely ever since, I think LLMs provided some Big Techs like Microsoft with what they hoped would be a viable pathway to crashing through the scale ceiling. Since AI could be so disruptive to their cash cows, I suspect Microsoft concluded they didn’t have much choice but to embrace LLMs, even if recklessly premature and inherently high-risk.

What does a wise EAI system look like?

1.???? Executable governance across the EAI system

2.???? Strong security designed-in from inception

3.???? Strengthens and protects sovereignty

4.???? End-to-end precision data management

5.???? Own and control your own data

6.???? Tailor to the needs of each entity by each entity

7.???? Be very selective in use of LLMs and other models

8.???? Prioritize quality over quantity

9.???? Deliver a CALO (Continuously Adaptive Learning Org)

10.? Limit external dependencies as much as possible

We think the wisest path possible in EAI is to adopt the EAI OS in our KOS (we invented), and benefit from nearly three decades of R&D, including more than a decade of direct engagement with many of the world’s leading companies during our product refinement process.

KYield news

We are very pleased to share that Robert Hegbloom (Bob) has joined our board at KYield. Bob retired from Chrysler after 34 years, working from near the bottom of the company all the way to CEO of the Ram Truck brand, which was one of the most successful brands in the auto industry under his leadership. Please join us in welcoming Bob to KYield. Thank you!