Local LLM Messenger: Chat with GenAI on Your iPhone

Imagine this: You need a quick code snippet or some help brainstorming solutions to coding problems. With LLMs integrated into your messaging app, you can chat with your AI assistant directly within the familiar interface to generate creative ideas or get help brainstorming solutions. No more complex commands or clunky interfaces — just a natural conversation to unlock the power of AI.

Integrating with messaging platforms can be a time-consuming task, especially for macOS users. That’s where Local LLM Messenger (LoLLMM) steps in, offering a streamlined solution for connecting with your AI via iMessage.?

What makes LoLLM Messenger unique?

The following demo, which was submitted to the AI/ML Hackathon, provides an overview of LoLLM Messenger (Figure 1).

The LoLLM Messenger bot allows you to send iMessages to Generative AI (GenAI) models running directly on your computer. This approach eliminates the need for complex setups and cloud services, making it easier for developers to experiment with LLMs locally.

Key features of LoLLM Messenger

LoLLM Messenger includes impressive features that make it a standout among similar projects, such as:

• Local execution: Runs on your computer, eliminating the need for cloud-based services and ensuring data privacy.
• Scalability: Handles multiple AI models simultaneously, allowing users to experiment with different models and switch between them easily.
• User-friendly interface: Offers a simple and intuitive interface, making it accessible to users of all skill levels.
• Integration with Sendblue: Integrates seamlessly with Sendblue, enabling users to send iMessages to the bot and receive responses directly in their inbox.
• Support for ChatGPT: Supports the GPT-3.5 Turbo and DALL-E 2 models, providing users with access to powerful AI capabilities.
• Customization: Allows users to customize the bot’s behavior by modifying the available commands and integrating their own AI models.

How does it work?

The architecture diagram shown provides a high-level overview of the components and interactions within the LoLLM Messenger project. It illustrates how the main application, AI models, messaging platform, and external APIs work together to enable users to send iMessages to AI models running on their computers.

By leveraging Docker, Sendblue, and Ollama, LoLLM Messenger offers a seamless and efficient solution for those seeking to explore AI models without the need for cloud-based services. LoLLM Messenger utilizes Docker Compose to manage the required services.?

Docker Compose simplifies the process by handling the setup and configuration of multiple containers, including the main application, ngrok (for creating a secure tunnel), and Ollama (a server that bridges the gap between messaging apps and AI models).

Technical stack

The LoLLM Messenger tech stack includes:

• Lollmm service: This service is responsible for running the main application. It handles incoming iMessages, processing user requests, and interacting with the AI models. The lollmm service communicates with the Ollama model, which is a powerful AI model for text and image generation.
• Ngrok: This service is used to expose the main application’s port 8000 to the internet using ngrok. It runs in the Alpine image and forwards traffic from port 8000 to the ngrok tunnel. The service is set to run in the host network mode.
• Ollama: This service runs the Ollama model, which is a powerful AI model for text and image generation. It listens on port 11434 and mounts a volume from ./run/ollama to /home/ollama. The service is set to deploy with GPU resources, ensuring that it can utilize an NVIDIA GPU if available.
• Sendblue: The project integrates with Sendblue to handle iMessages. You can set up Sendblue by adding your API Key and API Secret in the app/.env file and adding your phone number as a Sendblue contact.

Read the entire article at Docker

Ajeet Singh Raina is a developer advocate at Docker. He is a founder of Collabnix. He leads a Collabnix Slack community of 10K members. He is a Docker Community Leader and leads the Docker Bangalore community of 15K+ members. His community blogging site attracts millions of DevOps engineers every year and has more than 750+ blogs on Docker, Kubernetes and Cloud. Follow him on Twitter, Slack and Discord.