Part III: Getting Started with ollama

Welcome back to our blog series on local Large Language Models (LLMs). In this post, we’ll introduce you to ????????????, a versatile platform designed to simplify the deployment and management of LLMs. We’ll guide you through the process of installing and running ???????????? locally using ???????????? and demonstrate how to use existing models. By the end of this post, you’ll be ready to leverage ????????????’s capabilities for your own projects.

What is ?????????????

???????????? is an innovative platform that provides a streamlined interface for deploying and managing large language models. Designed for ease of use and flexibility, ???????????? supports various LLM architectures, including those compatible with llama.cpp. It allows you to run LLMs locally, ensuring data privacy, reduced latency, and cost efficiency.

Key Features of ????????????

• User-Friendly Interface: Simplifies the process of managing LLMs.
• ???????????? Integration: Facilitates easy local deployment using ???????????? containers.
• Flexibility: Supports multiple model architectures and customization options.
• Scalability: Capable of handling large-scale data and complex tasks.

Installing and Running ???????????? with ????????????

To get started with ????????????, you’ll need ???????????? installed on your machine. Follow these steps to install and run ???????????? locally using ????????????.

Step 1: Install ????????????

If you don’t already have ???????????? installed, download and install it from the official Docker website.

Step 2: Pull the ???????????? ???????????? Image

Once Docker is installed, pull the ???????????? ???????????? image from Docker Hub in the command line:

Step 3: Run ???????????? Container

Run the ???????????? ???????????? container:

This command starts the ???????????? container in detached mode, mapping port 8000 on your local machine to port 8000 in the container.

Using Existing Models with ????????????

With ???????????? up and running, let’s explore how to use existing pre-trained language models.

Step 1: Browse Available Models

???????????? provides access to a repository of pre-trained models. Navigate to the library of models on the ollama websiteand have a look. Deploying locally, we want to make sure we are using some of the smaller models, such as ``????????????:????, ????????????:???? or ??????????????:????.

Step 2: Download and Deploy a Model

Select a model from the repository that suits your needs. For this example, we'll use ????????????:????.

1. Open a terminal.
2. Type ???????????? run ????????????.

This will automatically download the ???????????? model and weights. Once completed, you should see the following in your terminal:

Step 3: Use the Deployed Model

There are a few ways of interacting with the ???????????? and the model installed. The first we have already seen, where we simply run it in the command line:

Another way is to use ???????????? via its API endpoint for interacting with the deployed models. Here’s an example of how to use the model via a simple Python script.

Create a new Python file, ??????_??????????.????, and add the following code:

Run the script:

You should see the output generated by the model based on the input prompt.

Note: of course you can also run this in a jupyter notebook.

The third we’ll cover is using the ???????????? python package:

Install via ?????? ?????????????? ????????????, then

Practical Applications of ????????????

???????????? can be used for a wide range of applications, including:

• Chatbots and Virtual Assistants: Create intelligent conversational agents that understand and respond to user queries.
• Content Generation: Automatically generate articles, reports, and other written content.
• Sentiment Analysis: Analyze the sentiment of text data for market research or social media monitoring.
• Research and Development: Experiment with advanced NLP techniques and model architectures.

And of course, as a tool at the center of Retrieval Augmented Generation, to use it for summarising scientific articles and help us speed up the way we keep up to date with the flood of literature being released every day.

Conclusion

In this post, we introduced ????????????, a powerful platform for deploying and managing large language models locally. We covered the installation process using ???????????? and demonstrated how to use existing pre-trained models. With ????????????, you can take advantage of on-premise AI to enhance your projects with advanced language processing capabilities.

Stay tuned for our next post, where we’ll explore ???????? ?????????? and show you how to set up multiple models with ???????????? and ???????? ??????????. Don’t forget to subscribe to our blog and leave your comments and questions below!

Happy experimenting!

Bonus: Running the API call in R