Zero to One: Build your first LLM Ai

This time around, we get down to some serious business - moving away from the theoretical ground work and opinion on how the landscape will shape up. We are going to build something useful that will let you touch and feel the power of Large Language Models first hand. Let us also challenge ourselves a bit more, and ensure that the entire thing runs on our local machine - with no dependency on any third-party cloud service or a wrapper around an Ai API service like that of ChatGPT or Gemini. Alright, let's dive right in!

Before we begin

Download and install Ollama - head over to www.ollama.com and find your way to the download section. You should be able to download on all operating systems including macOS, Linux and Windows.

Why do we need Ollama? We don't, but - it is the quickest way to get an open source base large language model like the Llama 3 or Mistral running on your local machine. Not only will this get the model running locally, but also spin up all of the services around the model for us to run local inference endpoints.

Next, make sure you have some sort of a Python scripting environment set up, preferrably with Jupyter Notebooks. Should be fairly easy. Alright then, we are set to roll!

What will our Ai do?

For this experiment, let us address a simple problem - we have LinkedIn, after all that is where you are reading this article, so let us explore a relevant problem. We fancy connecting with good people on LinkedIn - maybe for business or maybe for job prospects. Let us presume we have the URLs to their LinkedIn profiles. How about our Ai visits their LinkedIn profiles, reads through their page, and drafts an interesting introduction message to which they are highly likely to respond. The best part is, we don't need to go into the details of how Ai will come up with an interesting introduction - we will just let it figure it out (hence the Ai :-)

Sounds good for a first project? Ok then!

Building out our Ai Agent

Getting the "base" LLM

Time for us to hit the terminal. Now that you have Ollama installed and the ollama command available on your CLI, type in the following:

ollama run llama3        

If you haven't gathered, we have chosen Llama 3 as our base LLM and will write our agent on top of it. The above command will first download the model file to your local and then run it locally for your inferences to run against it. You have a command line looking like the one below:

How does it feel speaking to "your" silicon for the first time? This wasn't some third company's Ai that you interacted with - it was in all reality your own physical hardware that generated the response. In essence, intelligence ran on your machine... for the first time ??! Ok you get it... next things next.

Some quick and dirty python code to fetch a LinkedIn profile

Take the below code into your Jupyter notebook and given you have the Chrome Driver setup good, it should fetch my linkedin profile and extract all readable text from it. Code is commented for your easy understanding, but you should be able to tweak it to your needs. Notice how we install langchain and langchain community. You could do without langchain but it does make it easy for you to build Ai agents as you will see in your many future LLM adventures.

! pip install selenium bs4 lxml langchain langchain_community

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from bs4 import BeautifulSoup
import time

# Replace with the path to your chromedriver executable
chromedriver_path = "/usr/local/bin/chromedriver"  

# LinkedIn profile URL stored in a variable
linkedin_url = "https://www.dhirubhai.net/in/abi-chatterjee"

# Initialize Chrome webdriver
try:
  service = Service(executable_path=chromedriver_path)
  driver = webdriver.Chrome(service=service)
  # Open the LinkedIn URL
  driver.get(linkedin_url)
except Exception as e:
  print("Error creating session:", e)
  # Optional: log the exception for further analysis

# Wait for the page to load (adjust wait time as needed)
time.sleep(5)

# Get the HTML content
html_content = driver.page_source

# Close the browser window
driver.quit()

# Parse the HTML with BeautifulSoup
soup = BeautifulSoup(html_content, "html.parser")

# Extract all text content
text = ""
for element in soup.find_all(text=True):
  text += element.strip() + "\n"

# Print the extracted text
print(text)        

The last line of code should print out a text dump of everything from the profile. It is not the prettiest sight to behold but it will get our job done. In the next step, we will craft a nice primed prompt for the LLM to process and produce a creative introduction message.

from langchain_community.llms import Ollama

llm = Ollama(model="llama3")

prompt = f"""
You are an expert marketing bot and are wonderful at crafting creative introductions.
Below is a LinkedIn profile extract with some user activities.
----------------------
{text}
----------------------

Create a creative introduction message to this user.
"""

response = llm.invoke(prompt)

response        

The output of the above should look like something like the screenshot below.

That is darn phenomenal isn't it? We have been able to craft a hyper-personalised message for connecting with a LinkedIn connection. In this example we simply let the LLM do the magic for us with no extra frills and fancies. Now let us write some quick code to automate this connection process.

email = "<your_email>"
pwd = "<password>"

service = Service(executable_path=chromedriver_path)
driver = webdriver.Chrome(service=service)
driver.get('https://www.dhirubhai.net/home')
time.sleep(5)

username = driver.find_element(By.ID, 'session_key')
username.send_keys(email)

password = driver.find_element(By.ID, 'session_password')
password.send_keys(pwd)

sign_in_button = driver.find_element(By.CLASS_NAME, "sign-in-form__submit-btn--full-width")
sign_in_button.click()

time.sleep(10)

driver.get(linkedin_url)

# ... remaining code to click the connect button and send the personally crafted connection message
        

Now there are some nuances to automating such processes with legalities and such but hope this gets you primed enough to get creative and build more such interesting automations.

And to conclude

We have today dipped our toes into the world of LLM based Ai automations and seen the power of being able to run them locally. This is just the start of magic. In the next edition, let us do some finer things like fine tune a model or introduce memory or retrieval augmentated generation... we'll see. Stay tuned!