Rethinking the Roadmap: Why a Multi-SLM Strategy Could be Key to True AGI

It seems like daily, I come across internet posts wondering at the seemingly simple yet impressive achievements by Large Language Models (LLMs). Code generations, bug finding, counting the number of 'r's in "strawberry", determining the correct pronunciation of obscure words, or even basic arithmetic operations are presented as remarkable accomplishments. Meanwhile, a significant portion of people express skepticism and doubt about AI's ability to perform even the most mundane tasks. And with good reason - these LLMs require vast computational resources to train and deploy. The cost of training such models can be staggering, running into millions or even tens of millions of dollars. As a result, they often struggle to generalize beyond their narrow domains, leaving many use cases uncovered.

Below, i will explain some concepts (as I have a lot of views for my articles, but minimum interactions, I concluded that, somehow, not everybody understands what I try to explain) , and link them together a possible "better" architecture.

For model compositions, I choose to use:

Pipeline Approach - can set up a series of SLMs where each model performs a specific task in sequence. For example, one SLM might focus on data preprocessing, another on language understanding, and another on generating responses. Each SLM takes the output of the previous model as its input, creating a chained workflow.

Modular Approach - where for more complex interactions, you can assign different SLMs to specialize in subtasks and direct them to pass their output to other models based on task requirements. This is helpful if you're working with very limited model sizes, as it allows you to distribute the workload across several smaller models.

Then, those models need to communicate (through a controller).

Central Controller - a larger "controller" model that acts as a coordinator. It decides which smaller models to activate or consult based on input. The controller routes the requests between SLMs, enabling conditional processing.

Messaging Protocols - by using defined protocols, models send structured messages to each other or to a controller. For instance, if an SLM encounters a task it can’t handle, it can pass a specific message for assistance from another model. This approach I found very useful, because when i was playing with multiple actuals LLMs, everything was "fine" when the language was english, but for romanian/german language, it was katastrophe! Romania has a Research Institute for Artificial Intelligence #racai and it has reached 30 years of activity, unfortunately I couldn't find details about their projects. That's a pitty, because i could re-used some work.

For meta-prompting and context awareness, context-sharing prompts can provide each model with context prompts about what other models are working on, creating a pseudo-awareness. For instance, one model can "know" that another is handling factual verification, and it will focus on generating fluent text rather than checking facts. The structured prompts will use structured prompts that include information about other SLMs' roles. This way, each SLM can tailor its response based on what it knows other models are handling, which reduces overlap and improves task distribution.

On the later step, based on task, a dynamic model selection is needed. For simpler setups, prompt-based role asignment component dynamically assign roles to each SLM through prompting. For example, one prompt might tell a model to "analyze sentiment," while another directs it to "summarize information." This allows flexibility in what each SLM does based on the task context, keeping them adaptable and efficient. Using auto-scaling of the small models, instead of using one single larger model, you can have multiple instances of SLMs that get activated based on demand. For example, if an SLM needs help with language generation complexity, you can activate another instance that specializes in such cases.

1.?Pipeline-Based Interaction (Sequential Processing)

Technical Setup:

• In a pipeline, each SLM has a dedicated role, and data flows through them in a predefined sequence.
• This method is useful when the task can be broken down into smaller steps that SLMs can handle independently.
• Each model processes a part of the task, and the output becomes the input for the next model.
• You can use Python libraries like?transformers?to load multiple models, or leverage frameworks like TensorFlow Extended (TFX) to handle pipeline workflows.

Interaction Example:

Sentiment Analysis + Summary: Suppose you want a system that reads product reviews and outputs a summary sentiment.

• Step 1: SLM 1 analyzes the sentiment of each sentence and tags it as positive, negative, or neutral.
• Step 2: SLM 2 takes these tags and generates a summary sentence based on the majority sentiment.
• Step 3: If specific feedback (e.g., common complaints) is needed, another SLM could highlight frequent keywords, passing them to SLM 2 for more context-aware summarization.

This setup allows each model to specialize and reduces the computational load on any single SLM.

2.?Controller Model with Conditional Routing

Technical Setup:

• A controller model (typically larger and more powerful) oversees and orchestrates the tasks.
• This model dynamically routes requests to appropriate SLMs based on the content and complexity of the input.
• The controller can be designed to detect when certain conditions are met, prompting it to activate specific SLMs for particular sub-tasks.
• To implement this, you can create custom routing functions in a programming language like Python that check conditions and activate relevant models.

Interaction Example:

Customer Support Chatbot: Imagine a chatbot that answers customer questions but relies on multiple SLMs to cover different topics.

• Step 1: When a query comes in, the controller examines keywords to determine the topic (e.g., billing, product details).
• Step 2: Based on the topic, the controller routes the query to the relevant SLM specialized in that domain.
• Step 3: If the controller detects ambiguity, it can request clarification from a follow-up SLM to ask the user for more information before responding.

This setup gives the chatbot flexibility and allows for model specialization.

3.?Meta-Prompting for Task Awareness and Context Sharing

Technical Setup:

• Meta-prompting involves giving models specific context or instructions to ensure they understand their role relative to other models.
• You can achieve this by designing prompts that reference the actions or expected outputs of other models.
• If you have a set of task-specific SLMs, you can prompt each with what other models are expected to do, so they "know" when to hand off their output or refine it.

Interaction Example:

Report Generation Assistant: Suppose you’re creating an assistant to generate comprehensive reports on a topic.

• Step 1: The "data collector" SLM is prompted to find relevant data points, and its output includes a note indicating "data collection complete."
• Step 2: The "synthesis" SLM reads this note and is prompted to combine and interpret the data without needing to collect further information.
• Step 3: The "editor" SLM then checks for readability and structure, prompted to look for synthesis cues, so it understands the output as final text.

Meta-prompting lets each SLM interpret its task relative to the sequence, reducing redundant or conflicting responses.

4.?Dynamic Task Allocation with Auto-Scaling SLM Instances

Technical Setup:

• This setup involves creating and activating multiple SLM instances on demand, depending on the task requirements.
• Cloud platforms like AWS Lambda or Google Cloud Functions allow you to auto-scale small tasks, spinning up instances as needed. - if decided to use internet!
• This is especially beneficial for situations with fluctuating demand, as you avoid overloading any single SLM by distributing requests across multiple copies of a model.

Interaction Example:

Real-Time Social Media Monitoring: Let’s say you’re tracking trending topics.

• Step 1: When a trend spikes, the system activates several SLMs to analyze different aspects, like sentiment, keyword frequency, and related hashtags.
• Step 2: A central collector model then merges these insights, discarding redundant information and combining only the unique insights from each SLM instance.
• Step 3: If the trend dies down, the system scales back, deactivating unneeded SLMs.

This setup optimizes resource use and adapts to real-time demands, perfect for high-traffic scenarios.

5.?Direct Message Passing and Structured Protocols

Technical Setup:

• This involves designing message protocols where each model sends and receives structured data messages.
• Protocols could include metadata like timestamps, the specific task performed, or quality indicators (e.g., confidence scores).
• REST APIs can facilitate communication, allowing models to send HTTP requests and share JSON-formatted messages between each other.

Interaction Example:

Multi-Step Workflow for QA: Suppose you’re using multiple SLMs to verify and improve answers to user questions.

• Step 1: SLM 1 generates an answer to the question and passes it along with a quality score (e.g., 80% confidence) to SLM 2.
• Step 2: SLM 2 reviews and edits the answer if it detects any issues, then forwards it with its own confidence score.
• Step 3: The final answer is either approved by a central controller or passed to an "expert" SLM if scores are low.

This structured message-passing setup lets each model validate and refine outputs, ensuring higher accuracy and quality.

If you read up to this point and decided to enable small models to handle complex tasks by working together efficiently in your company, be aware that the right setup will depend on your application’s specific requirements - (read below for some proposed scenarios), like latency sensitivity, complexity, and scalability needs. Let me know if i can help you further, with your specific setup, through my consulting services!

My diagrams uses Markdown (mermaid) and it can not be aligned perfectly in VS Code.

Therefore, due to the low resolution of LinkedIn pictures, i will split the big diagram into two additional pictures, below.

Using multiple small language models (SLMs) can have advantages over relying on a single large language model (LLM) in several scenarios, particularly around flexibility, efficiency, and specialization. The SLM approach aligns well with modular, task-specific applications where you want to optimize for performance, cost, and flexibility. It allows you to balance your system according to the complexity and scale of each task while keeping maintenance straightforward. By distributing tasks across multiple SLMs, each with its defined role, the system remains agile and responsive even in the absence of internet connectivity. This approach minimizes reliance on cloud services, reduces latency, and allows for more robust performance in isolated or secure environments.

Some key benefits for the proposed model:

1.?Cost Efficiency

• Lower Resource Requirements: LLMs often require extensive computational power, especially for tasks that involve complex or real-time processing. Running multiple SLMs instead can be more cost-effective, as they generally demand fewer resources and can run on standard hardware.
• Reduced Inference Costs: Since each SLM is smaller, they typically require less memory and GPU power to run. For applications with low or moderate user traffic, this can be a huge cost-saving as you only activate models as needed.
• Optimized Cloud Billing: If your setup uses cloud resources, SLMs can take advantage of pay-as-you-go structures where they activate briefly for tasks and turn off when not in use, helping keep costs down.

2.?Scalability and Flexibility

• On-Demand Scaling: Multiple SLMs allow for scalable architectures where individual models can be spun up or down based on demand. This can be more flexible than running an LLM continuously, which can struggle with on-demand scaling due to its larger footprint.
• Specialized SLM Scaling: You can scale only the models needed for specific tasks. For example, if sentiment analysis spikes due to a social media event, only the sentiment analysis model needs to scale, leaving others at lower activity levels.

3.?Task Specialization and Precision

• Domain-Specific Models: Each SLM can be fine-tuned for specific tasks, allowing for higher accuracy and better performance in its specialty area. This means you can deploy task-specific models like a summarization model, a translation model, or a sentiment analysis model, rather than relying on one general-purpose LLM that may perform less optimally across all tasks.
• Reduced Cross-Task Interference: With task-specific SLMs, there’s less risk of "task bleeding," where the general-purpose nature of an LLM might cause it to mix up or prioritize one type of task over another. Each SLM has a clear focus and purpose, which makes them less prone to confusion in complex workflows.

4.?Improved Responsiveness and Real-Time Processing

• Lower Latency: Because SLMs are smaller, they often have lower response times compared to LLMs. For applications requiring real-time interactions or low-latency responses (like customer service), SLMs can offer a better user experience.
• Parallel Processing: You can set up multiple SLMs to handle different tasks in parallel, which can be faster than having a single LLM handle all tasks in sequence. For example, a customer support system can run multiple SLMs in parallel to analyze sentiment, detect entities, and generate responses simultaneously, reducing the time to complete each request.

5.?Easier Debugging and Maintenance

• Component-Level Debugging: With an SLM-based setup, you can troubleshoot issues at the model level, isolating issues in specific SLMs without affecting the whole system. This modular approach makes debugging easier, as you can track errors within a single model rather than sifting through a monolithic LLM.
• Simpler Retraining and Fine-Tuning: If a task changes, you only need to retrain the specific SLM associated with that task, rather than re-fine-tuning an entire LLM. This targeted retraining reduces time and cost, especially for tasks that evolve frequently.

6.?Enhanced Privacy and Security Control

• Localized Data Processing: For data-sensitive tasks, you can run SLMs locally or on-premises, keeping specific data entirely under your control. This is harder with an LLM, as they usually require significant resources that are often available only in the cloud.
• Selective Data Access: You can configure different SLMs to access only the data they need for their specific tasks, reducing the exposure of sensitive information across the entire system. This minimizes the risk of data leaks and allows for a finer-grained approach to data privacy.

7.?Better Customization and Adaptability

• Mix and Match: You can easily add or remove SLMs for different tasks without overhauling the entire system. For instance, if you want to add a new feature (like keyword extraction), you only need to integrate a new SLM for it. This is more challenging with an LLM, as it might require substantial retraining or reconfiguration.
• Adaptability to Niche Applications: By using SLMs, you can develop solutions for niche applications with specific requirements. If a new requirement arises, you can add a specialized SLM without disrupting the rest of the workflow.

Proposed Scenarios:

1.?Home Automation

• Voice Commands Processing: SLMs can be used for interpreting simple voice commands to control home devices like lights, thermostats, and security systems. For example, one SLM could be dedicated to understanding "lights on/off" commands, while another handles climate control commands. Without internet, each SLM can be optimized to interpret a specific set of household tasks accurately, without relying on a cloud-based LLM.
• Security Monitoring: An SLM can process security camera feeds and detect anomalies like unauthorized entry or movement within specified zones. Instead of requiring a continuous internet connection, this model would be trained to recognize a limited set of security-related events locally, sending alerts to the user’s device or triggering alarms.

2.?Agriculture

• Crop Health Monitoring: On a farm with IoT-enabled sensors, multiple SLMs can handle tasks such as monitoring soil moisture, analyzing plant images for signs of disease, and detecting pests. For instance, one SLM could assess soil data from sensors, while another detects color changes in crop images to predict health issues.
• Irrigation Control: An SLM can regulate irrigation based on moisture levels and time of day, adapting to environmental conditions on the spot. By using a localized SLM, farmers avoid dependency on cloud-based solutions, allowing for timely irrigation control even in remote areas.
• Weather Prediction Analysis: In cases where minimal weather data is available from local sensors, an SLM could make short-term predictions (e.g., humidity and temperature changes), guiding decisions on activities like planting or fertilizing without needing a full-scale LLM for complex analysis.

3.?IoT Applications

• Smart Factory Maintenance: In industrial settings, SLMs can monitor equipment status via IoT sensors, processing metrics like vibration, temperature, and usage patterns to predict maintenance needs. One SLM might monitor temperature anomalies, while another tracks unusual vibrations, alerting maintenance personnel as needed. This setup operates entirely offline and locally within the factory, ensuring reliability.
• Energy Optimization: For IoT-enabled homes or businesses, SLMs can optimize energy use by learning and adjusting patterns of device usage. Each SLM could handle a specific subset of appliances (like HVAC, lighting, or kitchen appliances) and adjust power settings or shut off devices when not needed, without relying on an internet connection.
• Vehicle Monitoring: In transportation IoT applications, SLMs can monitor the condition of vehicles, including real-time analysis of fuel usage, brake performance, or tire wear. For instance, an SLM could detect when brakes are overheating or fuel levels are low, allowing for preemptive action without sending data to the cloud.

4.?Drone / Military Applications

• Autonomous Navigation: For drones in remote or combat environments, SLMs can be deployed locally for tasks like obstacle detection, altitude adjustments, and real-time navigation. Each SLM can handle different aspects of flight, allowing for flexible coordination even when GPS or internet connectivity is unavailable.
• Object Recognition and Targeting: Drones can use SLMs for object detection, identifying predefined objects, targets, or hazards (such as vehicles or obstacles) in real time. Each SLM could specialize in recognizing specific classes of objects, enabling quick and focused recognition without the need for large-scale image processing from an LLM.
• Situational Awareness and Alerts: SLMs can analyze environmental data collected by sensors on drones or ground equipment, such as detecting temperature changes, movements, or potential threats. Each SLM is designed for a specific aspect of situational awareness, allowing rapid processing and decision-making on the device itself.

5.?6G and Edge Computing

• Network Performance Optimization: In a 6G network scenario, SLMs at the edge can adjust bandwidth allocation, detect interference, and optimize handover decisions for connected devices. Each SLM could specialize in one performance metric (like latency, signal strength, or bandwidth), making real-time adjustments locally rather than relying on a central server.
• User Behavior Prediction: Edge-based SLMs can analyze local user data to predict behavior patterns, adjusting network resources in response to anticipated needs. For example, one SLM could predict peak data usage times based on previous activity, while another optimizes connection quality.
• Enhanced Security at the Edge: 6G environments will demand advanced security even at the edge. SLMs can detect specific security risks like abnormal traffic patterns or unauthorized access attempts. Instead of depending on cloud connectivity, each SLM acts independently to handle specific security concerns, such as device authentication, data integrity, or network traffic filtering.

I help companies streamline operations, enhance productivity, and innovate with tailored AI-driven solutions—from customer service automation to predictive analytics. Contact me to start your transformation journey!