LM Studio

Meta Llama 3.2 running on NPU ?? We’ve been working closely with the ONNX Runtime team + Qualcomm to bring NPU support to LM Studio! H/t to Matthew Clayton and Natalie Kershaw for pulling this off in record time. Thanks Manoj Khilnani Devang Aggarwal Manish Sirdeshmukh for the partnership. In the picture: Snapdragon NPU doing the hard work during a session demo at Microsoft Ignite in Chicago this evening. It’s currently an engineering build, and the team is working to get ready it for release soon, stay tuned.

?? It’s crazy what you can achieve with a MacBook, LM Studio and Apple MLX running Mistral AI Pixtral while on a plane!!!

Huge news for local coding LLMs ??: Qwen 2.5 Coder 0.5B, 1.5B, 3B, 14B, and 32B are here! The 32B version scores higher than GPT-4o on Aider's code editing benchmark. Get it from the model page: https://lnkd.in/ey6ZCXFy Congrats to Junyang Lin and the rest of the Qwen team!

???? Are you a cracked JavaScript / NodeJS Systems Engineer in NYC? Join us to build LM Studio and our local LLM dev SDKs. Apply here: https://lnkd.in/eXr--RG6

Apple featuring LM Studio in the M4 MacBook Pro announcement makes so much sense Local LLMs are going to be the major driver for local compute needs for consumers/prosumers. > With support for up to 128GB of unified memory, developers can easily interact with LLMs that have nearly 200 billion parameters. My predictions for 2025: ? Local LLMs are going to be seamlessly integrated into our OSes and will 10x our productivity ? We will rely on data-center compute and GPU clusters much less hat/tip Yagil Burowski and team

Since May, I've been working with LM Studio to support shipping complex Python components for local execution in desktop environments in a way that readily supports sharing of large framework components like PyTorch and CUDA. LM Studio users have seen the benefits of that with the recent inclusion of Apple MLX support, and today I'm excited to share the open source publication of "venvstacks", the underlying utility that enabled that support: https://lnkd.in/gpGTrGw5

Such an awesome moment! LM Studio shown in Apple's new M4 MacBook pro announcement ?? > With support for up to 128GB of unified memory, developers can easily interact with LLMs that have nearly 200 billion parameters.

Don't let data intensive LLMs slow you down. ?? ??? Applications like LM Studio allow users with less available GPU memory to accelerate the most demanding LLMs with GPU-offloading. Learn more about this RTX acceleration in this week's #AIDecoded ?? https://nvda.ws/3YxkW5P ??

?? LM Studio v0.3.5 is out now with Headless Mode! Use LM Studio as your local server for powering local web apps, code editor or web browser extensions, and much more. Read the blog post: https://lnkd.in/dG6Cbc7K

This is game-changer for my local ML workflows: LM Studio 0.3.4 ships with Apple MLX ???? Run on-device LLMs super fast, 100% locally and offline on your Apple Silicon Mac. Includes: > run Llama 3.2 1B at ~250 tok/sec (!) on M3 > enforce structured JSON responses (via .txt's outlines ??) > use via chat UI, or from your own code > run multiple models simultaneously > download any model from Hugging Face The video below is at 1x speed. via Yagil Burowski and LM Studio based on terrific work from Awni Hannun and the community! ??