Large Language Models: an update for the perplexed
Author(s): Ross Burton & ChatGPT???
Editor(s): Joseph Smith & Natalja Kurbatova
Welcome to an enlightening exploration of the ever-evolving world of large language models (LLMs). This article aims to provide a concise yet comprehensive overview of the latest and most thrilling developments in these AI-powered systems.
What is a Large Language Model (LLM)
Before we continue, let me clarify what a Large Language Model (LLM) is: it's an artificial intelligence system trained on extensive text data. It aims to generate text that resembles human language based on the input it receives.
These models operate by predicting the probability of a word given the preceding words in the text. Although they don't comprehend text as humans do, they can produce coherent and contextually relevant sentences by recognizing patterns in their training data.
Several examples of LLMs include Google's BERT, BART, and T5, OpenAI's GPT-3 (the model behind ChatGPT) and GPT-4, as well as Meta's RoBERTa and LLaMa. These models are referred to as "large" due to their billions, or even trillions, of parameters, which enable them to generate high-quality text. The greater the number of parameters in a model, the more complex patterns it can learn from the training data.
LLMs have many applications, such as content creation, translation, and question-answering systems. They consistently demonstrate exceptional performance. In fact, were you aware that almost all the text in this article so far was composed by ChatGPT with minimal assistance?
LLMs represent a significant advancement, especially considering that most groundbreaking research papers on LLMs have emerged within the last three years. This suggests that they are still in the early stages of development. Each day their capabilities continue to expand, making it difficult to keep up to date with this cutting-edge technology. Luckily, here at Zifo, we're monitoring the situation and are here to give a concise overview of the most important changes in recent months.
Recent developments
The open-source explosion
A recent memo leaked from a Google employee titled "We Have No Moat" [1, 2] has sparked debate amongst AI researchers and engineers due to its statements about the state of LLM development. In the memo, the Google engineer states that despite substantial investments by tech giants such as Google, Meta (formerly Facebook), and OpenAI in generative AI chatbots, open-source alternatives could rapidly surpass them. He argues that while Google and OpenAI have been heavily focused on each other, open-source projects have been addressing major AI challenges more quickly and efficiently.
As developers and data scientists working with LLMs daily, we must agree with this statement. Stories from technology news outlets are sharing open-source innovations daily, and a quick search on GitHub reveals that the top trending repositories consist of open-source LLMs, frameworks, worked examples, and LLM-powered full-stack applications.
Open-source large language models such as Alpaca [3], Vicuna [4], GPT4All [5], BLOOM [6], MTB-7B [7], and Dolly [8] are proving to be as effective as closed-source models and offer an additional level of privacy and control for end-users. The open-source nature of these models is also fueling a rapid development cycle. However, as highlighted in the Google memo, the pace of this development raises concerns about the responsible release of AI and potential misuse.
At Zifo, we continue to monitor the landscape of open-source LLM models and consider their application for our customers. We see a significant benefit in the availability of open-source models in the biomedical and pharmaceutical space, as they allow us to control and isolate data flow, securing privacy and reducing third-party risks.
Agents
The current application of LLMs is tightly coupled with a query-response interface with humans in the loop, examples being chatbots, semantic search, and translation. A new paradigm is beginning to arise involving autonomous agents driven by LLMs.
Autonomous AI agents driven by LLMs are designed to interact with environments, understand and generate text, and carry out complex tasks in response. Furthermore, user prompts act as a seed for a complex web of self-prompting that can amalgamate into a fully autonomous process, making them useful for a variety of applications, from customer service to content generation and more.
AgentGPT [9] is one such autonomous AI agent system. An open-source web application, AgentGPT works by chaining language models (agents) to perform a user-defined goal. In a recursive workflow, an agent will predict the best tasks to perform to achieve the goal, execute those tasks, evaluate how it was performed, and continually think of additional tasks.
AgentGPT is not alone, HuggingGPT [10] is another example developed by Microsoft. HuggingGPT presents a collaborative system that consists of an LLM as the controller and numerous expert models (sourced from Hugging Face) as collaborative executors. The result is a powerful autonomous system that can achieve complex tasks such as simultaneous image, audio, and text generation from a single text prompt.
These Autonomous AI agents are still far from perfect and fail when presented with increasingly complex tasks. Additionally, their deployment is complex and often error-prone. Yet the rate of progress in their development is alarming and raises questions about the potential side effects of AI-driven agents. At Zifo, we are keeping a watchful eye on this space and anticipate that autonomous AI agents could have a significant impact on drug discovery and medical manufacturing.
The AI product wars
Whilst developments skyrocket in the open-source domain, a battle has started between the major technology companies to be the sole provider of consumer AI products, business services, and cloud computing platforms.
Amazon, a powerhouse with the weight of Amazon Web Services behind them, announced new EC2 offerings powered by their AWS Trainium accelerators and optimised for network-intensive generative AI models. On top of this, a new service for building and scaling generating AI applications called AWS Bedrock will soon be available for AI developers. They also announced a direct competitor to Microsoft's programming assistant GitHub CoPilot. Amazon CodeWhisperer promises to improve software engineer productivity whilst also scanning code for potential security issues [11].
Microsoft is focusing their efforts on improving the experience of general users with its generative AI products. Power Apps Copilot will allow non-technical staff to build web applications and dashboards with natural language descriptions, and Microsoft 365 Copilot will bring the power of ChatGPT to common tools such as Microsoft Word, Teams, Excel, and PowerBI. On the technical end, Microsoft Azure is now offering OpenAI's GPT and DALL-E models to customers with private networking, regional availability, and responsible AI filtering [12, 13].
Google, once known as the world leader in AI development, now find itself on the back foot fighting off its competitors. However, it was clear from their 2023 annual Google I/O conference that they plan to come back swinging. Google announced the release of PaLM 2, a large language model that comes in an array of sizes. The smallest (aptly named Gecko) promises to be small enough to perform inference on mobile devices, even when offline. They also announced Med-PaLM2, a model fine-tuned on medical knowledge, which they claim is approaching the performance of clinical experts [14].
Google also announced the integration of generative AI into Google Workspace for use with their Google Docs and Gmail platforms, in what appears to be a direct competitor to Microsoft 365 Copilot [14]. For AI developers wishing to deliver bespoke AI products, Google is now offering Vertex AI as part of the Google Cloud Platform. Vertex includes their Generative AI studio, a managed environment for what they call responsible AI development [15].
Zifo consultancy services offer a range of cloud and MLOps expertise that are working hard to keep up to date with these developments. Our expertise is cloud provider agnostic, and we are dedicated to finding the generative AI solution that fits your current cloud provider and business processes.
Public attention
The past few months have seen talk of ChatGPT, generative AI, and the broader impacts enter public debate. Fears around the power of this technology are beginning to boil, with Italy taking the striking move to ban ChatGPT entirely [16] and experts such as Geoffrey Hinton warning us of where this might lead too if left unchecked [17]. Regulatory bodies are also making movements to have strict requirements for LLMs before their use within the EU is approved, raising concerns this could have on open-source models and innovation [18].
In healthcare, biomedical research, and pharmaceuticals, the impact of ChatGPT and models fine-tuned for domain expertise are gaining notice. A recent article in Nature highlights the impact LLMs might have on patient-clinician relationships, communication, and personalised medicine. The article goes on to state that these models are clearly not ready for practical application yet. LLMs require expensive training on expert annotations to achieve acceptable standards of clinical performance and reproducibility. There is also a need for careful consideration and safeguards to protect against potentially dangerous uses, such as bypassing expert medical advice [19].
Several concerns have been raised regarding the application of LLMs in medical research. The impact of trust and credibility could be significant, especially when one considers the issue of authorship and accountability when content is predominantly artificially generated. Data privacy is another concern, as sensitive information inputted into LLM training may be at risk of unauthorised access or dissemination [20, 21].
The promise of this technology for biomedical research is equally recognised, however. Powerful AI models could help identify gaps in existing knowledge, help summarise and understand unfamiliar topics, and streamline research by finding the most relevant references, protocols, and data for expert scientists [22]. Generative AI models have also proved useful for direct drug discovery, such as Nvidia's BioNeMo service [23]. Through this tool, scientists can train LLMs to capture information about molecular structure, protein solubility and more, aiding in the discovery and design of therapeutics. Organisations such as AstraZeneca, MIT, and Harvard are using BioNeMo for protein design and predicting molecular structures [24].
领英推荐
At Zifo, we expect to see the public debate on these technologies continue, and we will stay up to date with the latest developments in AI regulation and how they affect its application in biomedical research.
What to look out for next
By now, it should be clear how impactful these new technologies are and how quickly they are beginning to develop and influence our daily business activities. We are confident that more models and applications will arise in the coming months. Developments likely include an increase in multi-model models that accept multiple data types as input (text, images, and audio) and can generate multiple data types as output. We also expect the ecosystem of open-source models to continue to increase and mature, especially now that privacy concerns are at the forefront of public debate. The deployment and application of LLMs will also continue to develop. A quiet but significant battle is taking place right now between the likes of Deepset's Haystack [22], Sidekick [23], and LangChain [24] to be the dominant framework in which LLMs are developed and deployed. For an example of Haystack in action, see our previous posts about our LLM demo (https://www.dhirubhai.net/pulse/get-semantically-similar-documents-from-unstructured, https://www.dhirubhai.net/pulse/zifo-semantic-search-service-technical-details-zifo-data-science). Finally, LLMs will require a new focus on MLOps to ensure reliable and timely delivery of LLM-powered services. The application of MLOps to LLMs is being popularised by the acronym "LLMOps", encapsulating the challenging task of training, evaluating, and deploying these models in a continuous lifecycle.
We hope you have enjoyed this short update on the most recent developments in the world of large language models. The dawn of a new age in AI development and engineering appears to be upon us, and Zifo is committed to keeping ahead for our customers. If you are interested in how Zifo's Data Science team can support the integration of LLMs into your business processes, please contact our team directly at [email protected]. We are here to help you solve your data integration and information search challenges.
References
[1] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
[2] https://www.economist.com/leaders/2023/05/11/what-does-a-leaked-google-memo-reveal-about-the-future-of-ai
[3] https://crfm.stanford.edu/2023/03/13/alpaca.html
[4] https://lmsys.org/blog/2023-03-30-vicuna/
[5] https://gpt4all.io/index.html
[6] https://bigscience.huggingface.co/blog/bloom
[7] https://www.mosaicml.com/blog/mpt-7b
[8] https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm
[9] https://agentgpt.reworkd.ai/
[10] https://arxiv.org/abs/2303.17580?
[11] https://aws.amazon.com/blogs/machine-learning/announcing-new-tools-for-building-with-generative-ai-on-aws/
[12] https://news.microsoft.com/ai/
[13] https://848.co/blogs-and-insights/microsofts-ai-announcements-in-2023-so-far/
[14] https://blog.google/technology/ai/google-io-2023-keynote-sundar-pichai/#labs-generative-search-experience
[15] https://cloud.google.com/vertex-ai
[16] https://www.bbc.co.uk/news/technology-65139406
[17] https://www.bbc.co.uk/news/world-us-canada-65452940
[18] https://technomancers.ai/eu-ai-act-to-target-us-open-source-software
[19] https://www.nature.com/articles/s41591-023-02289-5
[20] https://www.thelancet.com/journals/landig/article/PIIS2589-7500(23)00083-3/fulltext
[21] https://www.thelancet.com/journals/ebiom/article/PIIS2352-3964(23)00077-4/fulltext?ref=dedataverbinders.nl#secsectitle0040
[22] https://www.nature.com/articles/s41551-023-01012-6
[23] https://haystack.deepset.ai/?
[24] https://www.getsidekick.ai/
[25] https://python.langchain.com/en/latest/index.html