2024-26 Board, CEO, CAIO Research AI Insights: AI Risk Management: Corporate Playbook Reset

As AI services like OpenAI's GPT-4 and Microsoft's Copilot revolutionize sectors/industries with millions of users daily, managing AI risks is critical. Today's platforms, built on robust GAI models and massive data foundations, demand strategic oversight to ensure their ethical, safe, transparent, and practical implementation.

"The narrative is that AI is everywhere?all at once, but?the data shows?it's harder to?do than people seem interested in discussing."-?Dr. Kristina McElheran, MIT IDE Visiting Scholar.

AI is reshaping the corporate landscape, offering unprecedented opportunities for growth, efficiency, and innovation. However, these advancements come with significant risks that demand vigilant management. The challenge for Boards, CEOs, and Chief AI Officers is to harness AI's potential while mitigating its threats. This playbook provides a comprehensive roadmap for right-sizing your corporate AI stack, from selecting the right LLMs and applications to optimizing interfaces and domain-specific models. By strategically integrating these components, companies can drive value, foster growth, and achieve a robust ROI.

Here is a GAI LLM and apps—closed or open—ecosystem roadmap for resilient and reliable AI stack options. These risks-based options often use corporate proprietary data, AI LLMs, and open-source apps for data risk mitigation solutions.

Here's what CEOs, CAIOs, CDOs, and CIOs need to know!

CEOs, CAIOs, CXOs Key Takeaways

1. Evaluate Multimodel Foundation Models (LLMs) Adoption, Fit, ROI
2. Select Corporate Data-Specific LLMs for Performance Options
3. Align and Apply LLM Applications Leaders for Interfaces
4. Select LLM Application Leaders for Use Cases
5. Compare Specialized Doman High-Performance LLMs Fit, ROI, Value
6. Leverage AI LLM and Open Source Experiments/Pilots/Adoption.

1. Evaluate Foundation Models (LLMs) Adoption, Fit, ROI

Foundation models are neural networks trained on vast datasets without specific domain optimization, making them versatile for various traditional tasks such as customer and technical support, code generation, or technical error resolutions. These 2024 models include both closed-source and open-source options:

Closed-Source Models:

• OpenAI's GPT-4o
• Microsoft Copilot and Studio 4.0
• Google's Gemini 1.5 Flash, Google AI Studio,?Vertex AI - WIP
• Anthropic:??Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus

Open-Source Models:

• Hugging?Face's BLOOM - Global, 46 natural languages used
• Meta's Llama-2, now Llama-3, is English only, and data includes massive social media (X, Facebook, Reddit) and more. - WIP
• Minstal AI: 8x7B in English, French, Italian, German, Spanish.

Note: Tailored Business LLMs often need highly customized AI solutions. For example, combining Claude 3 and OpenAI models can allow for more finely tuned corporate solutions that better match specific business use cases and performance, such as customer service, data analysis, or content generation. In addition, Using multiple models can also serve as a risk mitigation strategy.

All these LLMs are based on the transformer architecture. Developing new foundation models is challenging due to the massive amount of data, computing power, cloud data centers, and expertise required, leading to a few high-quality options in the AI market.

Takeaways:

• Evaluate leading foundation models like OpenAI GPT4o and APIs and Microsoft Copilot 4.0 against other top open-source models, such as Bloom and Mistral for EU,
• Determine the best corporate AI stack and business platforms and ecosystem, such as Microsoft Azure AI 365 or Google Gemini Workspace ecosystems, that fit their traditional workforce tasks,
• Leverage efficient GAI capabilities and specific LLM app's best-fit use cases.

2. Select Corporate Data-Specific LLM Performance Options

Today, two optimizing foundation models (LLMs) for corporate use are Retrieval-Augmented Generation (RAG) and Fine-Tuning Approaches. Foundation models are versatile but may need to perform optimally for specific tasks. Corporations can consider both approaches to enhance performance for context-specific applications: RAG and Fine-Tuning. Each LLM method has its advantages and disadvantages:

RAG Approach: Retrieves relevant information snippets and appends them to the input prompt for the LLM.

Advantages:

• Easier to implement.
• Lower upfront costs.

Disadvantages:

• Lowers hallucinations and errors only by 17+%
• Increases variable costs due to API usage for each query and
• Limits by the maximum prompt size of the foundation model.

Fine-Tuning Approach: Retrains the LLM with domain-specific corporate data to optimize it for specific tasks.

Advantages:

• Offers potentially better performance for specific contexts and
• More extensive prompts are needed for future queries.

Disadvantages:

• Requires higher upfront computational costs, and
• Retains AI talent with right-sized new hires for greater technical GAI and LLM expertise.

Takeaways:

• Evaluate whether RAG or fine-tuning better suits needs and limitations, and
• RAG is cost-effective initially and more uncomplicated to deploy, while fine-tuning offers long-term efficiency and superior task performance.

3. Align and Apply LLM Applications Leaders for Interfaces

Corporations often rely on LLM application interfaces known as "GPT Wrappers," built on top of leading LLMs, such as OpenAI, Microsoft Copilot, and Google Gemini. Companies leveraging foundation models for their applications face challenges:

• Competitors can easily replicate their functionality.
• Differentiation must occur at the user interface level for corporations to stand out.

Consider GitHub Copilot, an AI industry leader powered by OpenAI's Codex and GPT-4, distributed through GitHub. With a massive 100+M base and growing developers, GitHub holds a significant distributional ecosystem advantage and gains insights for model improvement. Yet, balancing AI model enhancement with corporate customer and user privacy is crucial!

Big Tech leans towards vertical integration, integrating LLM capabilities into their platforms, like Gemini for Google Workspace and Microsoft Copilot for Microsoft 365, through its OpenAI partnership.

Domain-specific incumbents lacking proprietary LLMs can succeed by building tailored applications atop third-party LLMs. Leveraging last-point access, Big Tech can swiftly offer AI-enabled capabilities to customers, posing challenges for new entrants.

CEOs and CAIOs must strategize AI LLM adoption. Incumbents must capitalize on proprietary data for unique value. Startups lacking data access should focus on agility and outmaneuvering incumbents with speed.

Simply put, differentiation at the user interface, leveraging proprietary data, and agility are crucial in navigating the competitive landscape of generative AI adoption.

4. Selecting leading LLM Applications for Corporate Use Cases

The highest layer of the AI stack involves building applications on top of the foundational or custom fine-tuned LLM models for specific use cases. In 2024, the top 10 LLM applications most widely adopted and used by corporations based on specialized needs, by name order, are:

1. Alltius - Technical troubleshooting and customer support.
2. ChatGPT (OpenAI) GPT-4/4o - General-purpose conversational AI for numerous applications.
3. Cohere - Custom LLMs for enterprise-specific needs and applications.
4. Copy.ai - Automated content creation and copywriting.
5. Evisort - Legal contract builds/drafts and management.
6. Gong - Sales conversation analysis and insights.
7. Hugging Face Transformers - A hub for pre-trained models and fine-tuning for specific tasks.
8. Jasper - Marketing and content generation for businesses.
9. Jumpcut - Summarization of books and movie screenplays.
10. Writer - AI-powered writing assistant for business communication.

These LLM apps are often just SaaS apps, with pricing based on monthly usage fees. The costs are mainly associated with secure cloud hosting and API fees from the foundation models each utilize.

Since 2023, AI investments from Magnificent Seven (Mag 7) and VCs with funding stages have fueled the development of numerous new foundations and fine-tuned, task-specific models. This surge has led to thousands of startups creating specialized apps on top of leading LLM models, aiming to gain a competitive advantage. Notably, Mag 7 usually buys needed tech VC startups; however, 90% fail in year one, fewer go to IPOs, and 10% falter and crash in years 2-5.

Takeaways:

• Explore leading LLM applications like the top 10 list for their specific use case needs first.
• Evaluating these options can help integrate advanced AI capabilities efficiently into their operations.

5. Apply Specialized Doman High-Performance LLMs for Fit/Value/ROI

An LLM's performance depends on its neural network architecture and the overall quality and quantity of training data it receives. Transformer models, like BloombergGPT, excel when trained on vast, high-quality datasets tailored to specific domains. For instance, Bloomberg, the top global financial data service leader, leveraged its financial data access to develop BloombergGPT, a 50 billion parameter model surpassing ChatGPT-3.5 on financial tasks despite its smaller size.

However, competitors can quickly replicate these services, diminishing their competitive advantage. Although slower in replication, Big Tech can swiftly dominate the market, securing the lion's share usually in a year.

"Understanding the limitations of AI systems is crucial for businesses to mitigate risks and ensure reliable performance in real-world applications." - Gary Marcus, NYU.

Notably, GPT-4's superior performance in some scenarios highlights the ongoing evolution of generalist models. Corporations in highly regulated domains should "evaluate first" specific proven LLM solutions instead of reinventing the wheel.

Takeaways:

• Potential speed and cost containment of domain-specific data to enhance LLM effectiveness
• Limit the necessity of continual investment in model training and development.
• Sustain competitiveness for fit and value across sectors like aerospace, finance, healthcare, insurance, media, others, and
• Specialize in critical heavy regulation for use cases with specific domain knowledge.

6. Open Source Data Option for AI Experiments/Pilots/Adoption

"Making good use of AI and ML tools often entails having the right data strategy to support them" - Harvard Business Review, 5/2024.

As corporations explore AI's potential to transform their operations, the risk of exposing proprietary data becomes a critical concern. When adopting open-source AI and LLMs, OpenAI GPT4, Microsoft Copilot, and Anthropics Claud, executives must never forget that these LLMs are notorious for biases and inaccuracies related to gender and race.

According to Stanford's Foundation Model Transparency Index (FMTI), primary LLM developers' data remains a significant area of opacity. LLM data access, impact, and trustworthiness are the least transparent subdomains. Notably, most developers need to be more transparent about their data practices, with persistent issues, such as Google at 0%, OpenAI and Meta at 20%, Microsoft Phi-2 at 40%, and only StarCoder at 100%! Despite some improvement in transparency from 20% to 34% in the latest May 2024 FMTI, only BigCode, Hugging Face, and ServiceNow provide clear information on data creators, copyright status, licensing, and personal information. The FMTI shows that most Big Tech LLMs still considerably lag and have much more work ahead of them regarding higher AI and data transparency scores!

However, LLMs offer a path forward, but careful risk management is required to ensure data security and build customer trust. Business-centric custom AI LLM solutions can mitigate data risks for a more controlled approach to corporate AI safety.

Minimizing Open-Source AI Risks is Crucial

Not all LLMs are best for open-source solutions. Models like Bloom, Stable Diffusion-3, Vicuna, and others allow tailored AI solutions without relying on third-party services, keeping sensitive data in-house.

Takeaways:

• Consider Bloom,?Stable Diffusion-3,?and Vicuna models to develop customized AI applications, ensuring data control and security.

Navigate Licensing and Leverage Big Tech for Copyright Risk:

Compliance with AI open-source licensing terms prevents legal and ethical issues, safeguarding the corporation's reputation.

• Understand the specific licenses of each AI model and use resources like the Data Nutrition Project from MIT and Stanford AI research teams to evaluate data sources and score data,
• Seek the legal team to review other corporate data risk source exposures for contract cure.
• Leverage Big Tech AI services with an understanding that their AI data is responsible for any copyright and data risk lawsuits.

Update and Maintain AI Models:

Regularly updating AI models with the latest data releases ensures accuracy and relevance while maintaining system stability.

• Balance updating AI models by ensuring consistent and accurate customer experiences.

Mitigate AI Bias:

Addressing AI biases ensures fair and equitable treatment of all customers, enhancing corporate responsibility and trust.

• Continuously audit AI outputs to monitor and mitigate biases and
• Score and drop LLMs and apps with weak AI data security teams or by primarily using massive social media and ad data for model training.

Build Customer AI Rights and Trust:

Transparency in data usage builds trust and prevents customer backlash over data privacy concerns.

• Clearly communicate timely AI data policies externally and internally and
• Obtain explicit, easy customer consent, especially for private and sensitive data.

Into The Future of Corporate AI LLM Risk Management Strategy

Executives can leverage AI's transformative potential by adopting leading practices to minimize open-source risks. Ensuring data security, compliance with licensing, regular model updates, bias mitigation, and transparent communication will foster responsible AI adoption. This approach protects proprietary information, builds trust, and positions the corporation for sustainable and ethical AI innovation.

By focusing on these strategies, corporations can enhance their data risk posture and protect against privacy threats, bolstering customer loyalty and trust while mitigating potential lawsuits.

CEOs and executives must continuously adapt their strategies to navigate the evolving GenAI landscape. An annual Corporate Strategic AI Playbook reset will keep AI services, operations, and the workforce aligned with advancements and best-of-breed leading practices. Embracing a culture of experimentation, learning, and innovation will maximize ROI and secure a competitive edge in the AI-driven future.

By focusing on ethical and responsible practices, executives can harness AI's power, minimize risks, including "must-have" AI cybersecurity, and build a foundation of trust and reliability for sustained success.

References, Acknowledgements, and Selected Research

Financial Times: Risk Management, "Why risk managers need to fight AI with AI," Nick Huber, May 2, 2024.

Harvard Business Review: Analytics And Data Science, "External Data and AI Are Making Each Other More Valuable," Adam D. Nahari and?Dimitris Bertsimas, February 26, 2024

MIT SMR: MAGAZINE SPRING 2024 ISSUE, "Who Profits the Most From Generative AI?" Kartik Hosanagar and Ramayya Krishnan, March 12, 2024.

Stanford University: CRFM, The Foundation Model Transparency Index (FMTI) v1.1, Advisory Board, May 1, 2024.

Steve Hawald CEO CIO Advisory LLC: Forward-looking Boards & CEOs Newsletter,?"2024-26 Board CEO Research: Chief AI Officer (CAIO) Value," April Issue, April 29, 2024.

The Wharton School; UPenn: Knowledge at Wharton, "Five Myths About Generative AI That Leaders Should Know," Scott A. Snyder and Sophia Velastegui, April 30, 2024.

The Wharton School; UPenn: Knowledge at Wharton, "How Early Adopters of Gen AI Are Gaining Efficiencies," Shankar Parameshwaran, February 20, 2024.

Copyright STEVE HAWALD CEO CIO ADVISORY LLC and Board-CEO Research Insights + Vision? Newsletter? 2017-2024. Copying articles to share/use in any way breaches STEVE HAWALD, CEO CIO Advisory LLC. Our research, newsletters, and articles cited for IP rights cover all copyrights, trademarks, designs, domain names, patents, and all other IP rights worldwide. All other content and IP rights are owned for each researcher and?organization's?content and references cited for their ownership. We reserve all our rights in any IP without prior written approval by the CEO. Disclaimer: These articles are the?author's?opinion without financial payments and engagements.