Open-Source LLMs for Legal Applications

Artificial Intelligence is a technology capable of revolutionizing nearly any business sector, including law. Or rather, especially in law. This is due to the fact that any activity in the legal field involves processing vast amounts of data.

Large language models (LLMs) are ideal for this task. They leverage deep learning techniques to process textual data, and I must say, they do so with impressive efficiency.

What open-source LLMs for legal applications are available on the market? How have leading players in the legal sector integrated this technology into their processes, and should you do the same? I'll explain further.

Top Open-Source LLMs for Transforming Legal Processes

The modern market offers a sufficient number of large language models that are open-source and provide impressive capabilities.

I present to you those that are most suitable for building legal systems.

#1. OpenChatKit

I ranked this model first in my subjective rating because it offers a versatile search system. Developers can enhance bot responses with data gathered from various sources, such as document repositories, APIs, and more.

Using this LLM grants AI systems access to external data sources and allows them to provide users with comprehensive, informative answers.

#2. Falcon

This is a multilingual LLM designed for inference tasks. It can quickly and efficiently generate text, perform translations, and answer user questions.

Its application is especially relevant in fields like law. Legal professionals often refer to international research and legal documents written in foreign languages.

#3. SauLM-7B

This model is specifically designed for legal applications. It is trained on a vast amount of specialized texts and allows users to get answers to a wide range of industry-specific questions, analyze contracts, and summarize documents.

#4. GPT-NeoXT-Chat-Base-20B

This model is based on GPT-NeoX by EleutherAI. It was trained to follow instructions and participate in conversations. Thanks to this specificity, this LLM can be used to create chatbots and virtual assistants in the legal field.

A Success Story From Global Practice: The Complex Combination of Technologies in Westlaw AI

The Westlaw platform is a tool for legal research and an impressive database for legal professionals. Its goal, like that of other similar solutions, is to analyze vast amounts of legal data to generate answers to various user queries.

Given the complexity of the legal field, the implementation of large language models was essential for realizing its functionality.

The creators of Westlaw, Thomson Reuters, do not disclose the specific LLM used in their software product. I can assume that the company has developed its own industry-specific models.The only publicly available information concerns the company's experiments with the now-popular BERT model.

They used the basic version and the one released by Google. The latter was trained on an impressive dataset, including Wikipedia (2.5 billion words) and the Toronto Book Corpus (0.8 billion words). And that's not all – the company further refined it with their own legal data. Thus, the model was adapted to the specific nuances of legal language and concepts.

In addition, the developers used another innovative technology stack:

• The Amazon SageMaker engine, which allows training and deploying the model in production with literally one click.
• The Open Arena corporate platform to facilitate experiments with different LLMs.
• AWS Serverless Components for managing workflows on the platform. AWS DevOps Services for continuous integration and continuous delivery (CI/CD).
• The AI Platform data service, which frees the user from the need to gather information, allowing them to focus on analysis and model development.

The thorough approach of the Thomson Reuters team in selecting technologies made Westlaw the number one choice for thousands of legal companies.

Want to adopt their successful experience? See how to create an AI-based system similar to this solution.

Development of an AI-powered Legal Application: 5 Steps to Success

Here are the key stages that are indispensable when creating a legal digital solution aiming to lead its industry:

#1. Collecting legal data. The effectiveness of the model depends on the quality of the data it is trained on. Therefore, the first step should be collecting data. Different sources must be used for this, including case law, legislative acts, legal journals, and more.

That’s not all. Now, the collected data needs to be processed and structured. For example, it may be necessary to remove irrelevant information or standardize its format.

#2. Choosing and configuring a large language model. Now it’s time to choose the LLM that best fits your field of work (the available options were mentioned earlier). Afterward, you need to configure it, i.e., train it on the pre-prepared data. This will allow the model to better understand legal terminology, legal concepts, and other industry nuances.

#3. Developing a Reliable Architecture. It is important to keep in mind that the architecture of such software must handle a large volume of legal data and complex user queries. An excellent example is the technology stack used to create the Westlaw AI system, which I mentioned earlier.

#4. Ensuring a positive user experience. Prioritizing the development of an intuitive user interface is crucial, enabling users to ask questions in simple language and receive well-structured, clear responses. Additionally, incorporating extra features such as summarization, highlighting, and links to the original sources is recommended.

#5. Ongoing monitoring and improvement. It is crucial to integrate ongoing performance monitoring mechanisms into the product. Additionally, maintaining up-to-date data is vital for providing precise and relevant responses. Human review of the results for accuracy and feedback for improving the quality of the output is also very effective.

Here are a few more important considerations that AI-based legal application developers should not forget:

1. It is important to timely address biases and prejudices in legal datasets.
2. Software testing should not be ignored to ensure its accuracy and reliability.
3. Special attention should be given to the security of confidential data.
4. Focus should be placed on fairness, transparency, and accountability in the collection, storage, and processing of information via LLMs.
5. It is recommended to use explainable (whitebox) AI. After all, only such models can provide not only an answer to the user’s query but also the algorithm behind it.

Want to join the global experience of using large language models in the legal field?

Share your experience (or plans) of integrating AI technologies into your law firm's infrastructure in the comments.

P.S. At AdvantISS, we develop AI-driven legal tech solutions using open-source LLMs and automation tools. If you're interested in legal AI solutions, contact me on LinkedIn or find more details on our website.