Geek Out Time: Play with LangChain 2- locally with Gemma
In “Geek Out Time: Play with Langchain”, I realized that I had reached my OpenAI usage limit and needed to arrange for an upgrade. Running the Large Language Model (LLM) locally, where LangChain can access it, will save some money. Google released Gemma a few weeks back . Let’s give it a try.
ollama run gemma:2b
After successful installation, we can verify it by posing questions internally to see Gemma’s responses.
3. Install LangChain
pip install langchain
Run the command to verify the installation
? python -c "import langchain; print('LangChain version:', langchain.__version__)"
LangChain version: 0.1.11
4. Invoke Geme in the testing code
from langchain_community.llms import Ollama
import logging
# Configure basic logging
logging.basicConfig(level=logging.INFO)
try:
llm = Ollama(model="gemma:2b")
# It's good practice to ensure prompts are well-defined. Adjust based on the model's capabilities.
prompt = ("Who are you? "
"Are you better than Mistral? "
"Can you share a detailed comparison?")
response = llm.invoke(prompt)
print(response)
except ImportError:
logging.error("Failed to import Ollama from langchain_community. Is the package installed?")
except Exception as e:
logging.error(f"An unexpected error occurred: {e}")
and you will see the reply,
You can also call Gemma through a RESTful API locally:
curl https://localhost:11434/api/chat -d '{
"model": "gemma:2b",
"messages": [
{ "role": "user", "content": "hi, who are you?" }
]
}'
It takes 2–3 seconds to receive a reply, and Gemma seems quite impressive. Now, we are ready to port over more complicated LangChain works to our local large language model (LLM) while saving money. Have fun!