Open WebUI - Advanced LLM GUI

Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI designed to operate entirely offline.?

In this article, we'll explore how to set up and run a ChatGPT-like interface on your local machine using Open WebUI. This tool allows you to utilize large language models like Llama or dolphin-mixtral, as well as connect to OpenAI's API.

Open WebUI can optionally be set up on a server for multiple user access and has a rich set of admin and user controls.

Cost effectiveness

Using Open WebUI presents a cost-effective solution for managing various chatbot interactions. This versatility stems from its ability to switch between free, open-source models and more powerful, paid APIs based on the complexity of the task at hand.

For routine inquiries and standard interactions, the free models offer sufficient capabilities, effectively reducing operational costs. However, when faced with more complex requests that demand higher accuracy and nuanced understanding, users can seamlessly switch to paid APIs.

This flexibility allows for a more economical use of resources, ensuring that you only pay for the advanced services when absolutely necessary, while still maintaining high-quality responses across all types of queries.

Installation

1. Install Docker:

• For Windows and Mac: Download Docker from the Docker website.
• For Linux: Use your package manager to install Docker.

2. Install Ollama:

• Visit the Ollama.ai website and follow the installation instructions or view my Ollama tutorial.

For windows users I recommend the one click Docker install option.

https://docs.docker.com/desktop/install/windows-install/

Open windows terminal

Check docker is running - docker --version

Run this command

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main        

Accessing Open WebUI

1. After running the Docker container, you can access the Web UI.
2. Open your browser and navigate to https://localhost:3000.
3. The interface should resemble ChatGPT.

To read more or for troubleshooting tips.

https://github.com/open-webui/open-webui

The first time you run Open WebUI sign up for account. This will create an administrator account. Standard user accounts may optionally be created later.

Light/Dark Mode

Optionally change theme to dark theme found under the settings menu at the top right of the window.

Interacting with the Model

1. Choose a model from the drop-down menu (e.g., dolphin-mistral:latest).
2. For this example I will select dolphin-mixtral.
3. When using Ollama the models must first be installed using Ollama before they will show up in as a selection in this dropdown menu.
4. Enter a prompt, like "hello" or a question about a specific topic.
5. The first prompt may take longer as the model loads. Subsequent prompts will receive faster responses.

Note: If you add an API for commercially available LLMs, for example OpenAI's ChatGPT then the model selection dropdown will include these options.

Using Models and OpenAI API

1. You can switch between local models and OpenAI's GPT models.
2. To use OpenAI's models, you need to enter your OpenAI API key in the settings under the 'external' section.
3. After entering the key, you can select GPT models (like GPT-4) and interact in the same way as with local models.
4. Remember, using OpenAI's API will incur costs per token for input and output and does not offer the same level of privacy and security as open-source LLMs running on your own computer.

Running Code and Complex Queries

1. With models like dolphin-mixtral, you can run more complex queries, such as coding requests.
2. However, remember that large models require significant system resources (VRAM, RAM, CPU).
3. If your system is powerful enough, you can run these models for advanced tasks.

Notable features

1. Local and Remote RAG Integration

• What it is: Experience advanced chat interactions with Retrieval Augmented Generation (RAG).
• How to use: Integrate content from both local sources and the web into your chats by starting your prompt with a # symbol.
• Where to learn more: Detailed guidance is available at the RAG Integration Tutorial.

2. Prompt Preset Support

• What it offers: Quick access to preset prompts for streamlined interactions.
• Usage: Use the / command in the chat input to load conversation starters instantly.
• Extra resources: Import prompts easily through the Open WebUI Community.

3. Multiple Model Support

• Functionality: Switch between various chat models to diversify your interactions.

4. Many Models Conversations

• Advantages: Engage with multiple models at the same time, taking advantage of their unique capabilities for richer responses.
• Application: Enhance your experience by interacting with a variety of models concurrently.

5. Voice Input Support

• Experience: Interact with your model using voice input for a more convenient and engaging conversation.

6. Image Generation Integration

• Integration: Incorporate image generation features seamlessly.
• Specifics: Utilize the AUTOMATIC1111 API for local image generation capabilities.

Note: Automatic1111 must first be installed on your computer.

Setting Up Multi-User Environments

1. Connecting to an External Server

• Purpose: This feature lets you connect to a server hosted at a different location.
• How to Set Up: Configure the environment variable to link your system to the external server. This creates a seamless connection for users.

2. Load Balancing Across Multiple Open WebUI Instances

• Advantage: Improves performance and reliability by spreading chat requests over several Open WebUI instances.
• Implementation: Set up multiple instances of Open WebUI and configure them to distribute chat requests evenly. This ensures smoother operation even under high demand.

3. Managing Multiple Users

• Functionality: Offers an efficient way to manage a multi-user setup.
• Management Tool: Utilize the user-friendly admin panel to oversee and control user access and activities. This streamlines the process of managing numerous users within your system.

These features are designed to optimize multi-user setups, enhancing the overall performance and management of the system for all users.

For a full list of features check out the documents page.

Conclusion and Tips

This setup is ideal for those who want to run large language models locally or use OpenAI's API more cost-effectively.

Remember to check your system's compatibility and resource availability, especially for larger models.

Experiment with different models and queries to fully leverage the capabilities of Ollama Web UI.

Paul Hankin is the author of:

AI Adoption: A Practical Guide for Business

and

AI and Law: Navigating the Future