MLX on Apple silicon

"MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.

The overarching goal of MLX is to be user-friendly yet efficient for training and deploying models. Its design is intentionally straightforward to encourage researchers to easily extend and innovate within the framework. MLX draws inspiration from established frameworks like NumPy, PyTorch, Jax, and ArrayFire."

MLX can be used to enable LLM's, StableDiffusion, Whisper etc. on your Apple silicon ?? ??

How to setup?

These are the different steps I used to install MLX on my Mac Studio (Apple M1 Ultra). You'll need to install Xcode and have conda.

Build Requirements

• A C++ compiler with C++17 support (e.g. Clang >= 5.0)
• cmake – version 3.24 or later, and make
• Xcode >= 14.3 (Xcode >= 15.0 for MacOS 14 and above)

$ clone https://github.com/ml-explore/mlx
$ cd mlx
$ mkdir -p build && cd build

$ conda create -n mlx python=3.10
$ conda activate mlx
$ conda install pybind11
$ conda install pytorch torchvision torchaudio -c pytorch

$ env CMAKE_BUILD_PARALLEL_LEVEL="" pip install -e .

$ cmake .. && make -j
$ make test
$ make install        

Full details ???? https://ml-explore.github.io/mlx/build/html/install.html

MLX Examples

Once the above is installed you can start playing with some MLX examples

$ git clone https://github.com/ml-explore/mlx-examples.git 
$ cd mlx-examples/mistral
$ pip install -r requirements.txt        

Now download the mistral model and tokenizer

$ curl -O https://files.mistral-7b-v0-1.mistral.ai/mistral-7B-v0.1.tar
$ tar -xf mistral-7B-v0.1.tar        

Once downloaded convert the weights with

$ python convert.py         

Now you can use the LLM as follows

$ python mistral.py --prompt "It is a truth universally acknowledged,"  --temp 0

[INFO] Loading model from disk.
[INFO] Starting generation...
It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.

So begins Pride and Prejudice, one of the most famous novels in the English language.

The story of the Bennet family, five unmarried daughters and their mother, is a classic tale of love and misunderstanding.

The novel was first published in 1813, and has been adapted for film and television many times.

The most recent
------        

Whisper

There's also a Whisper example

$ cd ../whisper        

Create a python file which loads an mp3 some spoken audio and transform it to text

import whisper
speech_file="test.mp3"
text = whisper.transcribe(speech_file)["text"]
print(text)        

Now run this python code and see the transcript appear in your terminal ??

python audio.py        

StableDiffusion 2.1 in MLX

The first time you run this it will download the stableDiffusion weights provided by Stability AI on Huggingface Hub. This does mean you need to have a HuggingFace token installed.

Now you can generate images as follows

$ python txt2image.py "A green apple on Mars"        

Good times!

Peace,

Stephan