There is more than one way to architect AI

(A variation of "there is more than one way to do GenAI" which was originally published on the Scott Logic Blog).

Is the UK and Europe really behind in AI as some have been claiming on here?

The value from AI doesn’t have to be linked to the brute forced requiring massive data centres approach. Europe isn’t necessarily behind in AI arms race. In fact, the UK and Europe’s constraints and focus on more than just economic return and speculation might well lead to more sustainable approaches. Consider that the applied AI market - including consultancy services could turn out to be much bigger and more lucrative than the Infrastructure?

This article is a follow on to Will Generative AI Implode and Become More Sustainable? from July 2024. It’s purpose is to challenge some of the narratives that the big tech players are pushing out (who might just be a little biased due to their strategy, operating models and investments).

AI needs big massive data centres. Really?

The prevailing narrative suggests that AI requires enormous centralised computing facilities, but let’s examine this assumption more closely:

• Local models are now easier than ever to implement - tools like LM Studio enable personal deployment with just a couple of clicks, democratising access to AI capabilities (but also increase bring your own AI risks?)
• While training large models currently demands significant resources (and federated training remains challenging), inference is a different story entirely
• We can move compute more easily than power - edge and distributed architectures offer compelling advantages that will work alongside or provide alternatives to centralised operating models
• Today’s big infrastructure investments risk becoming tomorrow’s stranded assets. Consider what happened during the last GPU hype cycle for Bitcoin mining - more efficient ASICs eventually replaced GPUs. We’re seeing a similar pattern emerge as models become more efficient and capable of running on consumer-grade hardware (just look at the capabilities of current distilled models)

AI needs to return results right now in real time. Does it?

The assumption that AI must always provide instant responses deserves scrutiny as it is partly what requires the brute force approach:

• AI can operate asynchronously, enabling higher quality processing at lower power consumption
• In enterprise contexts, AI can respond to events through more scalable, sophisticated event-driven architectures that avoid the point-to-point solution integration nightmares of the past

This approach not only reduces resource requirements but often leads to better outcomes through more thoughtful processing.

AI needs to be all powerful and aiming for AGI to be valuable. AI models should be at the centre of solutions. Not necessarily.

The race toward Artificial General Intelligence (AGI) often overshadows more practical applications that don't need hyper scale infrastructure:

• AI simply needs to be good enough for the job - narrow AI (good old Machine Learning) already demonstrates massive value in specific domains
• “Middle AI” that augments human decision-making often proves more pragmatic than attempts to fully automate complex processes
• Smaller models, when orchestrated intelligently, can lead to superior outcomes
• Just as diverse teams typically produce better ideas and implementations, diverse “model zoos” might achieve similar benefits through complementary capabilities
• AI models don't have to be blindly trusted to get integration with data and tools correct on their own
• AI Workflows orchestrated with traditional logic calling models where they are really needed is a far more efficient and lower risk approach. Particularly for highly regulated use cases.
• Agents don't have to be entirely AI powered - they can have a "deterministic spine" using AI models only for component parts of their implementation.

(More on these bullets in a future article).

AI will be chat or end user orientated. Think bigger.

While chatbots dominate current AI discussions, they represent just one pattern among many possibilities - many of these are smart UX and architecture choices that again don't require massive infrastructure. They require change management and end user adoption.

• Chatbots are merely one interface pattern, not the definition of AI interaction
• Human-Computer Interaction (HCI) can be fundamentally reimagined, and we’re only beginning to explore the potential of hyper-personalisation
• Multi-modal AI remains in its infancy, with vast untapped potential
• AI capabilities will increasingly be embedded within other platforms and applications, often invisibly enhancing functionality rather than serving as standalone interfaces
• Agentic AI represents a significant evolution beyond simple chat interfaces.

Sustainable Innovation Through Constraints

Europe’s approach to AI development, often criticised as lagging behind, might actually pioneer more sustainable and ultimately more valuable approaches. The constraints and considerations that shape European AI development – from regulatory frameworks to environmental concerns – could drive innovation in unexpected ways.

Consider how these “limitations” might actually catalyse more efficient solutions:

• Regulatory requirements pushing development toward more transparent, explainable AI systems
• Energy constraints driving innovations in model efficiency
• Privacy concerns leading to better local processing capabilities
• Environmental considerations spurring creative approaches to compute resource utilisation

Learning from Cloud Adoption: The Public and Private AI Journey

The evolution of AI deployment models bears striking similarities to the cloud computing journey. Just as organisations learned to balance public cloud, private cloud, and hybrid approaches, we’re seeing a similar pattern emerge with AI:

• Enterprise Private AI
• Public AI Services
• Hybrid AI Approaches

The lessons learned from cloud adoption can inform AI strategy:

• Not everything belongs in the public cloud
• Cost benefits aren’t always straightforward
• Vendor lock-in concerns are real
• Security and compliance need careful consideration
• The right solution often combines multiple approaches

Looking Forward

The future of AI doesn’t necessarily lie in brute force approaches or massive centralised computing facilities. By challenging these assumptions and exploring alternative federated, distributed and hybrid architectures, we can develop AI systems that are not only more sustainable but potentially more effective at solving real-world problems.

The current AI arms race, with its emphasis on model size and computing power, might be missing the forest for the trees. Sometimes, constraints breed creativity, and Europe’s more measured approach to AI development could lead to innovations that better serve both business and societal needs.

What’s clear is that there’s more than one way to advance applied AI in our organisations, in a way that mitigates the concentration risks of current approaches. As we continue this journey, perhaps it’s time to question whether bigger always means better, and whether the path to truly transformative AI might lie in smarter, more efficient approaches that consider the broader impact on our society and planet. As a result is Europe really behind or just taking a different approach?