Star History Monthly Pick | Llama 2 and Ecosystem Edition
Bytebase - Database CI/CD and Security at Scale
Leading open source database DevSecOps solution for Developer, Security, DBA, and Platform Engineering team.
On July 18th, Meta released Llama 2, the next generation of Llama. It can be freely used for research and commercial purposes and supports private deployment.
Therefore, we have located a few open-source projects to help you quickly get started with Llama 2 on your own machine, regardless of what it is!
Llama
Llama itself is an open-source Large Language Model (LLM), trained with publicly available data. It was officially open-sourced earlier this February and five months later, a new generation was released.
Compared to the original version, Llama 2 was trained on 2 trillion tokens, has double the context length than Llama 1, and comes with three different parameter sizes: 7B, 13B, and 70B. The difference in parameters allows you to choose between a smaller and faster model or a more accurate one based on your preferences.
llama.cpp
llama.cpp is one of the achievements of the community mentioned in Meta's official announcement. It has rewritten Llama's inference code in C++, and through various optimizations, it has challenged our understanding: it can run large-scale LLMs quickly on ordinary hardware. For example:
This project is so successful that the author, Georgi Gerganov, established his side project as a startup called ggml.ai (a tensor library for machine learning, powering both llama.cpp and whisper.cpp).
Ollama
Ollama is designed to run, create, and share LLMs easily. It was originally designed for macOS, (with Windows and Linux coming soon, as per their website).
Ollama's author previously worked at Docker, and the rise of open-source language models inspired him that LLMs could use something similar. This led to the idea of providing pre-compiled packages with adjustable parameters.
领英推荐
Once you have downloaded Ollama on your Mac, you can start chatting with Llama 2 by simply running ollama run llama2.
MLC LLM
MLC LLM aims to enable you to develop, optimize, and deploy AI models on any device. You can natively deploy any LLM on a diverse set of hardware backends and native applications (supported devices include mobile phones, tablets, computers, and web browsers) without the need for server support. You can also further optimize the model performance to suit your own use cases.
MLC Chat already launched on the Apple App Store and now supports the Llama-2-7b model. It is simple and super easy to get started with, although my iPhone got really after 3 questions ?? (Side note: looks like Llama 2 still has a lot of room for growth tho, is any of these SQL Editors real?).
LlamaGPT
LlamaGPT has proven that the AI tide is still at its highest, as it has already gained 6.6K stars on GitHub just five days after being open-sourced.
It is a self-hosted chatbot that offers a similar experience to ChatGPT but does not transmit any data to external devices. Currently, all three models of Llama are supported, and llama.cpp is utilized in the backend (all hail open source).
Compared to the aforementioned tools, LlamaGPT is a more complete application with a UI and does not require manual configuration or optimizing parameters. This makes it the most friendly for non-technical users to get started with Llama 2.
Last but not least
As an open-source, free, and commercially available LLM, Llama has brought AI closer to us. Although it may not be as advanced as other paid models, just like Meta mentioned in the press release, "We have experienced the benefits of open source, such as React and PyTorch, which are now commonly used infrastructure for the entire technology industry. We believe that openly sharing today’s large language models will support the development of helpful and safer generative AI too." With the power of the community, Llama and its ecosystem will surely continue to iterate (quickly).
But of course, there are many other ways to start using Llama 2, via Homebrew, Poe, etc. For some further reading: