Using ChatGPT Offline: How Small Language Models Can Aid Healthcare Professionals

By now, you might have come across the term large language models (LLMs), which is a type of generative artificial intelligence (GenAI). If not, you have likely encountered GenAI applications that are based on LLMs. This includes the likes of ChatGPT, Google Bard and Microsoft Copilot. While such models have proved useful, in the healthcare setting, they come with new sets of regulatory, ethical and privacy concerns.

Recently, another type of language model has been gaining attention in the GenAI field: the small language model (SLM). It even holds the promise of addressing some of the challenges with integrating LLMs in healthcare. In this article, we will introduce SLMs, compare them to LLMs and explore their healthcare potential.

What are SLMs?

Like LLMs, SLMs are a type of GenAI and operate in similar ways. This means that they rely on neural networks to learn patterns from language in text to produce new text of their own.?

SLMs are termed as “small” as they are trained on relatively small amounts of data and have a relatively small number of parameters. Parameters here refer to variables that define the model’s structure and behavior.

Emphasis should be paid on relatively as SLMs still involve millions or even billions of parameters. However, this range of parameters in SLM is “small” in comparison to LLMs and we’ll consider the differences between these models in the next section.

SLM and LLM: what’s the difference?

While there is no clear threshold for SLMs, they usually tend to have between a hundred million to tens of billions of parameters. While still large numbers, this range is small compared to the number of parameters LLMs possess which can reach hundreds of billions. Consider OpenAI’s GPT-3: this LLM has 175 billion parameters, and GPT-4 is believed to have about a trillion parameters. In comparison, Microsoft recently introduced Phi-2, an SLM developed by the company’s researchers with 2.7 billion parameters.

This difference in architecture is reflected in the resources required to run the different types of models. LLMs require significant computing resources from servers to storage. Such needs trickle down to the huge costs associated with running such models; and are thus not accessible or even feasible to every organisation. In comparison, SLMs can be small enough to run offline on a phone while bearing significantly less operational costs.

“Small language models can make AI more accessible due to their size and affordability,” says Sebastien Bubeck, who leads the Machine Learning Foundations group at Microsoft Research. “At the same time, we’re discovering new ways to make them as powerful as large language models.”

While a complex model with more parameters can be more powerful, SLMs can still have an edge over LLMs. By being trained on smaller and more specialized datasets, SLMs can be more efficient for specific cases, even if this means having a narrower scope than LLMs.

This can even lead to SLMs to outperform LLMs in certain cases. Microsoft exemplified this with their Phi-2 SLM, which performed better in coding and maths tasks compared to the Llama-2 LLM which is 25 times larger than Phi-2.

SLMs’ potentials in healthcare

By focusing on curated, high-quality data and requiring less computational and financial resources, SLMs are particularly apt for healthcare uses. While GenAI which is based on SLMs has not been publicly released, we can contemplate some of the technology’s potential.

1. Personalised patient journey

By training an SLM-based GenAI on relatively small but high-quality datasets, patients can receive a personalized healthcare experience. This can be achieved by developing a chatbot that focuses on a specific condition and can provide patients with educational materials and recommendations specific to their conditions. With such a tool, each patient could even have a personal, artificial doctor’s assistant that guides them during their patient journey.

2. Affordable generative AI

By requiring fewer resources to train and run, SLMs are more affordable than their LLM counterparts. Such models could be deployed without the need for costly infrastructure such as specialised hardware and cloud services. Through such increased accessibility, more healthcare institutions could benefit from GenAI and further tune the technology to their individual needs, without compromising on efficiency.?

3. Improved AI transparency

Thanks to their simpler architecture, SLM outputs are more interpretable and thus more transparent. Transparency over such AI models is further enhanced by the ability to better control the training data to address biases and be more reflective of the population it is assisting. This can further help in building trust in such AI tools.

While SLM tools have yet to be publicly deployed in healthcare settings, their advantages indicate that it is only a matter of time until this happens. Big tech companies are actively working on such models. Microsoft researchers have developed and released two SLMs, namely Phi and Orca. French AI startup Mistral has released Mixtral-8x7B, which can run on a single computer (with plenty of RAM). Google has Gemini Nano, which can run on smartphones.

However, SLM tools will also bring about their respective concerns when they eventually roll out for healthcare purposes. They will have to adhere to similar regulations we propose for LLMs in order to ensure their safe and ethical applications. As Microsoft lists SLMs as one of the big AI trends to watch in 2024, it might be worthwhile for the healthcare community to do the same.