Run Large Language Models on Your Local Machine - No Coding Experience Required

?? The Secret Most Miss: You don't need a computer science degree to run advanced AI. You just need the right guidance and the willingness to learn.

By following the steps in this guide, you're not just downloading a model - you're entering the next generation of personal AI exploration.

Learning to run and finetune Large Language Models by yourself isn't for the purpose of learning, it is one of the requirement to push yourself forward in the impact-delivery chain in the new workforce.

I understand that majority of my subscribers have no technical background and might be afraid of the technical jargons involved - While I promise to keep things really simple, you will have to learn some jargons alongside as they all add up to the skill you are building.

In this guide, I will show you how to install open source large language models on your local machine starting with the newly released Google gemma3 models but the process is the same for other models.

What makes Gemma 3 revolutionary?

• 128,000 token context window (that's ~100 book pages)
• Comprehends 140+ languages fluently
• Fully multimodal (handles text AND images)
• Just 27B parameters yet matches performance with models 25X larger
• Open source - completely free to use

Most people miss: The 1B parameter version runs on almost ANY laptop with 8GB RAM - performance that was impossible just months ago.

Step-by-Step Deployment Guide

Watch this video for the full guide and follow along

1. Set Up Ollama (Your Local AI Engine)

First, verify if Ollama is installed by opening your command prompt and typing:

To open command prompt on your windows device, press

• Windows Key + R
• Then type cmd
• Press Enter

ollama        

If not recognized, download and install Ollama from their official site. Ensure you have version 0.6.0+ as earlier versions can't run Gemma3's architecture.

Pro tip: Check your version with ollama --version

2. Choose Your Model Size Based on Hardware

Your RAM dictates which model variant to use:

• 8GB RAM → Gemma 3 1B
• 16GB RAM → Gemma 4B
• 32GB RAM → Gemma 12B
• 64GB+ RAM → Gemma 27B (full version)

3. Install Your Model

Simply run this command (example for 1B version):

ollama pull gemma3:1b        

The secret is: You can verify installed models anytime with ollama list

4. Add a ChatGPT-Like Interface (Optional but Recommended)

Install Docker Desktop (scroll to the end for the link to download and install Docker desktop), then run:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main        

This creates a beautiful web interface accessible at https://localhost:3000

5. Start Chatting!

Command line: ollama run gemma3:1b

Web UI: Open https://localhost:3000, login once, and select your model

But What About More Advanced Features?

You're not limited to just basic chat. With the same setup, you can:

• Upload documents for analysis
• Process images (with multimodal models)
• Use code interpreter capabilities
• Run multiple models and compare outputs
• Build custom applications on top

Here's why this matters: The democratization of AI is happening RIGHT NOW. The tools that were exclusive to tech giants just months ago are running on consumer hardware today.

What Will You Build?

I'm diving deep into local AI deployment and automation - no technical background required. If I can do this, you absolutely can too.

What applications are you excited to build with locally-running AI? Drop your ideas in the comments!

Link to resources

• Resources OLLAMA - https://ollama.com/
• OLLAMA Models - https://ollama.com/search
• Docker Download - https://www.docker.com/products/docker-desktop/
• OpenWeb UI - https://docs.openwebui.com/

Up Next

How to finetune open source large language models.