Run LLaMA2 with Ollama

With Ollama application, we can run the Large language models on local machine.

Ollama application is available for MacOs and Linux and can be installed like any package on the macOS.

1. Download the Ollama app from https://ollama.ai/ and install it.
2. Run this command to download the Meta LLaMA2 Large language model and it will run and share prompt as shown in your local PC.

ollama run llama2        

3. it will display the prompt and we can query the like ChatGPT UI.

4. For programming interface, we can interact with model by interacting with Ollama port 11434

curl -X POST https://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why Pluto is not a planet?"
 }'        

5. We can use langchain library to interact with Ollama in python and do the queries to any PDFs or datasets by creating the embeddings using Chroma.

In my test environment, asking the questions to large PDF by LLama2 model is very slow and model took 30-40 seconds from 120 pages company report files.

Ollama and LLama 2 definitely open lot of opportunities for application developers to start consuming the large language models in their business applications.