How good is your AI when your cloud is down?

Ensuring AI Continuity: The Imperative of Multi-Model, and Multi-Cloud Strategies

AI systems are becoming the backbone of business operations and the resilience and reliability of these systems are critical. Recent events (today July 19th, 2024) such as the CrowdStrike security software impacting Microsoft Windows-based systems and disrupting some cloud services across industries, serve as reminders that no IT infrastructure is immune to technical issues. Even with 99.999% uptime guarantees, failures can occur at the most unexpected moments. This raises a critical question: How robust is your AI strategy when your primary cloud provider faces downtime?

The Case for Diversification in AI Deployments

Multi-Model AI Approach

No single AI model can fulfill all requirements. Enterprises are increasingly adopting a multi-model strategy, combining:

1. Proprietary models from major vendors (e.g., OpenAI's GPT-4, Anthropic's Claude)
2. Open-source models (e.g., Mistral, IBM Granite)
3. Custom fine-tuned versions (e.g., domain-specific adaptations of Hugging Face models)

This diversification allows organizations to leverage the strengths of different models for various tasks. Also if the LLM is used as cloud service and that service faces issues, limited operations based on own instance of LLM in on-prem environment are valuable alternative.

Multi-Cloud AI Deployment

To mitigate risks and optimize performance, a multi-cloud strategy for AI deployment is becoming essential. Here are the primary deployment options for AI systems:

• Public Cloud (AWS, Azure, Google Cloud Platform, IBM Cloud, Oracle OCI, etc.)
• Virtual Private Clouds
• Software-as-a-Service (eg. LLMs with APIs from OpenAI, Anthropic, Cohere, etc).
• On-premise (for complete control over AI infrastructure incl. HW/SW/LLM stack)

Each option has its merits, and the choice depends on several factors:

1?? Data sensitivity and regulatory requirements (On-prem and VPC are preferred for AI use cases in highly regulated industries)

2?? Scale and resource requirements (public cloud and SaaS allow to scale AI faster, large enterprises with consistent volumes will find on-prem AI systems - including both hardware and software) more cost-effective)

3?? Existing infrastructure and expertise (those who already invested in own AI infrastructure will lean towards on-prem systems, those starting from no or little IT investment will find SaaS and public cloud more attractive)

4?? Performance and latency requirements (room for edge computing near to end-user)

5?? Budget constraints (SaaS and public cloud have lower staring costs but in long-term may come more expansive)

6?? Customization needs (on-prem and VPC offer full customisation, SaaS is less flexible but easier to maintain)

7?? Geographic distribution (Multi-region public cloud deployments can be ideal for globally distributed teams or applications, if they do not have own data centres on multiple continents).

The Hybrid Cloud Solution for AI Systems

A hybrid cloud approach offers the best of both worlds for AI deployments, allowing organizations to:

• Diversify AI workloads (e.g., training on-premise, inference in the cloud)
• Balance security, cost, and performance needs of different AI applications
• Mitigate vendor lock-in risks for AI infrastructure

While preparing AI applications for multiple cloud environments can be costly, containerisation and orchestration platforms like KUBERNETES offer a compelling solution. Kubernetes enables cloud-agnostic, containerised AI workloads that can be deployed in on-premise, cloud, and edge environments.

Major cloud providers offer managed Kubernetes services ready for AI workloads:

• AWS: Elastic Kubernetes Service (EKS) with integration to SageMaker
• Azure: Azure Kubernetes Service (AKS) with Azure Machine Learning
• Google Cloud: Google Kubernetes Engine (GKE) with Vertex AI
• IBM: IBM Kubernetes Service (IKS) with Watson AI services

Easch of them has unique differences, so question is - Is there an universal option? Yes, for true multi-cloud flexibility in AI deployments, platforms like Red Hat OpenShift (RHOS) stand out - it is available on all major clouds as managed service and is available on prem as well. Open-source code transparency comes as a bonus.

OpenShift's "write once, deploy everywhere" approach minimizes the effort required to move AI workloads between different environments, ensuring consistent performance and security across diverse infrastructures. Especially when emergency requires to redeploy AI applications in different cloud, ability to move quickly to another RHOS environment may be priceless.

Conclusion - not 'if' but 'how'

As AI systems become increasingly central to business operations, ensuring its resilience and availability is crucial. By adopting multi-model, and multi-cloud strategies, organizations can build robust AI systems that remain operational even in the face of infrastructure challenges. The key lies in diversification, flexibility, and leveraging technologies that enable seamless transitions between different AI deployment environments.

In an era where AI downtime can mean significant business disruption, the question isn't WHETHER you should adopt a multi-faceted AI strategy, but HOW QUICKLY you can implement one. The future belongs to organizations that can harness the power of AI consistently, regardless of the underlying infrastructure challenges and sub-system blackouts.