Jan: Turn your computer into an AI computer

Artificial Intelligence is rapidly evolving, and so are the tools that make it accessible. One such groundbreaking tool is Jan, an open-source application designed to run SLMs locally on your device. As someone who recently experimented with running the SLM called Phi-mini on my MacBook Air M3, I can attest to the ease and efficiency of using Jan. This blog post will explore how Jan enables seamless AI interactions without compromising data privacy or performance. Jan’s primary advantage lies in its local-first approach, ensuring that all interactions and data remain on your device. This means you have complete control over your data.

You can choose to download a specific model of choice and adjust inference parameters as needed. You can also replace a default assistant message and customize as you wish.

One of Jan’s standout features is its extensive model selection. With an app store-like interface, you can easily choose from various models tailored to different needs, such as coding assistance or general queries. I started with Phi-mini, a model known for its balanced performance and efficiency and can be easily run on Mac’s M-series chips. The model ran smoothly, delivering prompt and accurate responses. The nice feature of the app is that it shows you which models you have enough RAM for to run locally.

Beyond just running models, Jan offers customization and integration features that make it a versatile tool for various applications. You can customize the behavior of the models through third-party extensions and APIs, enabling a tailored AI experience. For developers, Jan’s ability to function as a local OpenAI equivalent API is a game-changer, allowing integration into other applications seamlessly. This flexibility makes Jan not just a tool but a robust platform.

RAG example by uploading 45 pages of PDF document from ITF about rules of tennis. Jan running Phi-mini was able to extract and summarize information about The Rules of Tennis Committee for me:

Running OpenAI compatible local HTTP server:

In conclusion, Jan represents a significant leap towards democratizing AI by making powerful models accessible and controllable locally. Running the Phi-mini model on my MacBook Air M3 showcased Jan’s ease of use, speed, and versatility. Whether you’re a developer looking to integrate AI into your applications or an individual seeking a private and efficient AI tool, Jan offers a compelling solution.