Part 2: Introduction to Ollama

Ollama is an innovative framework designed for running, managing, and building applications using large language models (LLMs) directly on local machines. It provides developers with the tools to integrate AI capabilities into their applications while maintaining control over the models and data. It Utilizes a?Mixture-of-Experts (MoE)?architecture, which allows it to handle nuanced tasks such as reasoning and inference effectively. This design enables it to maintain high performance with smaller models, making it resource-efficient and accessible for individual users and smaller organizations.?

?Key Features of Ollama

·???????? Local Model execution

·???????? Open-Source Models

·???????? Easy Integration

·???????? Model Customization

·???????? Performance Optimization

In this section we will demo how to deploy Ollama as a Docker container in AWS ec2 instance and use the same with llama3 and phi4. We will also install a Hugging face model to demo usage of models not natively supported by Ollama.

We will not be covering the steps to create ec2 instance. we will assume this is already available we will start with deploying docker, docker-compose and Ollama with Open Web-UI service and validating the same from our laptop/desktop.

The EC2 instance is running.

?Steps to install Docker.

?

Steps to Install Docker compose.

?

Ollama-compose file to create both Ollama and OpenWebUI as Service.

?

Let us connect to the instance via putty.

Let us install Docker and docker-compose

Next docker-compose.

Next , we will create the ollama.yaml file.

?

Now let us deploy this by running the command

docker-compose -f ollama.yaml up -d

?

?

We can validate it by running the command.

docker ps

?

Now we can open chrome browser and test it,

https:// 18.209.164.240:3000 where “18.209.164.240” is the Public IP of the ec2 instance on which the service is running.

?

Press “Get Started”.

Enter the details and press “Create Admin Account”.

Press “Okey, Lets go”.

Now we are ready to use. Go to url https://ollama.com/library to get list of models available in ollama.

For experiment Let us Load the Llama 3.2 model

In our OpenWebUI browser type

And Press on pull “llama3.2:1b” from ollama.com.

Wait for it to complete.

Now its complete. Let us use it and ask questions.

Next, let us test the reasoning of llama3.2 model.

Not very good at reasoning.

Next Let us Load phi4 and try the same question and see how the reasoning works with phi4.

Press Pull “vanilj/Phi-4” from Ollama.com. Let us switch to Phi-4 and ask the same question and see the answer.

Phi-4 is very good at reasoning. Now let us try a code generation as well.

?

Next, we will create HTML CSS code from Wire frame Diagram using Llama3.2 vision model.

Press “ Pull llama3.2-vision:11b” from Ollama.com . It ready now. Select the Vision Model

Press “ on the “+” sign to upload the image.

Press “Upload files”.

Press “Enter now”.

?

?

Next, we will Load a model from Hugging face Hub which is not natively available in ollama. We can load only GGUF models.

?

Let us use this model with ollama now.

?

?Cleaning the installation

Next, we will bring down the service and remove the docker images.

Let us manually remove the images.

If we run “docker ps -a” we should not see any service running.

We can logout and shutdown ec2 instance.? In the Next Part we will integrate all this and create an AI Search engine.

?

?

?

?

?