All you need is flow - from models to multi-agent AI systems

Millions of internet users interact with AI language models. Starting with popularity of free interactive service ChatGPT in November 2022 (initially using GPT3.5, currently GPT-4o or GPT-4o mini), today there are many alternatives available thru chat interfaces like Anthropic Claude , Microsoft CoPilot , Google Gemini , Perplexity , Cohere Command-R , and more. But they are just starting pointS for AI developers in building advanced apps and AI Systems.

Language Models?

Language models (LLMs, SLMs) and all their complexities of model tuning, prompt engineering, Retrieval Augmented Generation techniques, and ethics involving transparency, trust, guardrails, de-biasing, etc. are just starting components on the way to building AI workflows that can operate with high level of independence transforming inputs and prompts into outcomes - not just data but also actions.?

The language models differentiate in many dimensions and can suit various use cases. Depending on what you need - trustworthiness and fit to enterprise business (Granite by IBM), large context windows (Gemini by Google), popularity (GPT-4 by OpenAI), inference speed (Llama 3.1 by Meta), small size and good latency (Phi-3.5 by Microsoft), or advanced Mixture of Experts features (Mixtral by Mistral). Many models have several editions in the family, with various sizes and specialisation (chat, instruct, code, vision). AI Agents and AI applications can leverage multiple models depending on needs and fit for purpose. It is too early for AI developers to make bet on just one model and have confidence that it will the only one they will ever need. The future of AI is multi-model, multi-modal and multi-cloud (hybrid cloud represents balanced approach to on-prem, in the cloud cloud and edge deployments). Hugging Face and AI platforms from big tech companies provide access to multiple models from other vendors to maintain such flexibility.

AI Agents

AI agents are semi-independent, goal-oriented entities that not only provide answers but also execute tasks. They can talk and they can walk. They leverage the natural language capabilities of underlying models and combine them with decision-making algorithms to perform actions to meet specific objectives. AI Agents can communicate among each other to execute more complex tasks.

AI Applications

AI application represent the user-facing layer of GenAI technology. They package the capabilities of language models and AI agents into intuitive interfaces that allow end-users to apply the?power of AI for specific business processes. They are the closest representation of business processes across industries.

The magic link: Flow

The true magic happens when language models AI agents and AI apps are connected by AI developers to execute business process logic as a FLOW. Sometimes also called WORKFLOW or PIPELINE or CHAIN. It is an orchestrated sequence of operations that connect AI components to achieve complex, multi-step processes. It's the pipeline that channels the capabilities of language models augmented by vector databases, through goal-oriented agents and into user-facing apps, all aimed at reflecting industry business logic and rules.

For example - client's application for a loan at bank is processed by large or small language model, augmented by data from company-own knowledge base in a RAG system which improves quality of output and limits hallucination. Such output is fed to decision support system that decides what action to take: approve the loan application, send email requesting more info or reject and update entries in internal databases for next cycle of model learning.?

Flow toolkits

Several toolkits and platforms have emerged to empower AI application developers simplifying the creation and management of AI flows. Few examples include:

• LangChain : A popular open-source framework to simplify development of applications powered by LLMs. LangChain provides a set of tools and abstractions that make it easier to build complex AI workflows and applications.
• LlamaIndex (formerly known as GPT Index): is also an open-source toolkit focusing on efficient indexing and retrieval of information, which is crucial for creating knowledge-intensive search and retrieval AI applications.
• watsonx.ai Flow Engine : Recent addition to watsonx platform, it simplifies building the flows for RAG or summarisation, using declarative programming language in CLI. It is proprietary tool but can integrate with LangChain which is important for developers with experience in that tool.

There are several new entrants that can change the game with visual tools to simplify and accelerate building of AI workflows, but this is topic for separate article.

The Bigger Picture: AI Systems

The flows connecting models, agents, and apps are part of a larger entity – the AI System. This overarching concept includes not just the AI components and their interactions, but also the supporting infrastructure, governance frameworks, and operational considerations.

Components of a Comprehensive AI System

• Models and Algorithms: The core AI capabilities, including language models and machine learning algorithms.
• Agents and Services: Goal-oriented AI entities and microservices that perform specific functions.
• Applications: User-facing interfaces that make AI capabilities accessible and useful.
• Infrastructure: Systems for storing, processing, and managing the data that feeds AI models, including vector databases. It includes the Compute Resources,?the hardware and cloud services that power AI operations during model training, fine-tuning, and inference. There is also integration layer - APIs, connectors, and middleware that allow different components to communicate and execute the flows.
• AI Systems management: AI Governance, Security measures, policies and tools to protect AI systems from threats, and very fast growing GenAI Ops domain that maintains system efficiency as the scale and complexity of AI operations grow.?

AI becomes the new OS for businesses and will be part of almost every industrial process. Increasingly complex AI systems, require ability to create and manage efficient and scalable flows and data pipelines. Connecting and chaining language models, AI agents, AI apps in cohesive systems will unlock the true potential of AI. The future of AI lies not in isolated smart components, but in intelligently orchestrated systems that can tackle complex, real-world challenges.

In this new era of AI, remember - all you need is flow.