Nexa AI

We are thrilled to announce that Nexa AI’s GitHub repo has reached 3000 stars!? ? ? This is a huge milestone for us, and we’re incredibly grateful to our amazing community of developers and supporters. ?? Big updates are coming soon ?? ?? —make sure to star our GitHub and stay in the loop: github.com/NexaAI/nexa-sdk. Your feedback, requests, and contributions are always welcome! ?? ?? Check out some well-loved features and example projects we released earlier: - Executables for easier model setup - Evaluation tools integrated with the SDK - Support for popular models, including models from Hugging Face - Streamlit support for better visualization ?? https://lnkd.in/gm69DCyG ?? Project Examples: - Chat with Local PDF: https://lnkd.in/g7qyUeCb - Local File Organization: https://lnkd.in/gKG5F7fE ?? Small Language Model Leaderboard: nexa.ai/leaderboard

?? Excited to announce our partnership with PIN AI to bring Nexa’s on-device AI models and local inference framework to mobile! PIN AI is building a visionary personal AI OS for mobile, enabling a powerful, private, and responsive AI experience right in the palm of your hand. This partnership is a leap toward a future where personal AI on mobile is secure, fast, and easily accessible. Stay tuned! ?? https://lnkd.in/d7SaRaPd

?? We just launched an On-Device Small Language Model (SLM) Leaderboard for all you local LLM fans! ?? Compare quantized versions of popular models like Llama3.2, Qwen2.5, Phi-3.5 based on quality (ifEval) and performance (response time, output speed, prefill speed, and power consumption). ?? Check it out & contribute: nexa.ai/leaderboard Want a specific model added? Join our Discord Server at https://lnkd.in/dUMJAXCU or submit a GitHub issue at github.com/NexaAI/nexa-sdk! ???

Thank you, Howie Xu, for hosting such a great discussion! Exciting things are on the way—we’re gearing up to release a series of multimodal models, and Nexa SDK will see major updates soon! Follow us at https://nexa.ai to stay updated on our journey in on-device AI!

?? Day 2 at TechCrunch Disrupt 2024 was on fire! We showcased our fresh Octopus on iOS demo, and the energy here has been incredible. ??? Don’t miss our pitch today at the SaaS, Enterprise + Productivity Stage. We can’t wait to share our vision for the future of on-device AI with you! ?? There’s still time to swing by Nexa AI's booth at L8 for more demos—snap a photo and tag us! ? Follow our GitHub repo and give us a star to be among the first to try Local AI on your mobile: github.com/NexaAI/nexa-sdk. Stay tuned, exciting things are on the way! ?? #TechCrunchDisrupt2024 #NexaAI #OnDeviceAI #LocalAI #SaaS #EnterpriseAI

We’ve got an amazing lineup of AI experts and innovators ready to share their expertise at #ODSCWest 2024! These seasoned professionals will dive into the latest advancements in topics like on-device AI, scaling deep learning with PyTorch, bridging the DevOps gap for data teams, and customizing AI with synthetic data. Get to know a few of them below! ??Zack Li is the CTO and co-founder of NexaAI, specializing in on-device AI. He co-authored the Octopus model, a top-ranked AI model, and previously worked at Google and Amazon Lab126. Tutorial: On-Device Multimodal Model Development and Inference Acceleration Zack Li will present innovations in improving large language model function calling using the Octopus model's functional tokens. He’ll also cover optimization techniques that enhance the speed and efficiency of on-device AI across various modalities like text, images, and speech. ??Shagun Sodhani is a Tech Lead at FAIR, Meta’s AI Research division, where he leads a team focused on training large-scale foundation models for multimodal data. His research aims to develop lifelong learning agents capable of continuous improvement through real-world experience. Tutorial: Scaling Deep Learning Training with Fully Sharded Data Parallelism in PyTorch Shagun will explore Fully Sharded Data Parallel (FSDP) in PyTorch, demonstrating how it distributes training across multiple GPUs to scale large models efficiently, even with hardware limitations. ??Elad Hirsch is a Tech Lead in the CTO Office at TeraSky, where he specializes in bridging the gap between data operations and DevOps practices. Tutorial: Introduction to Prompt Engineering and AWS Bedrock for Backend and Data Engineers In this session, you will learn the basics of LLMs, explore key techniques for crafting effective prompts, and discover how AWS Bedrock enables seamless model deployment and fine-tuning. ??Salma Mayorquin is the CEO and Co-Founder of Remyx AI, with a strong background in ML infrastructure from her time at Databricks.? ??Terry Rodriguez, also a Co-Founder, brings over a decade of experience as a Data Scientist and Machine Learning engineer, having led AI projects in industries like healthcare, robotics, and gaming. Tutorial: Customizing AI with Synthetic Data: Techniques and Real-World Applications Salma and Terry will explore the creation and use of synthetic data to enhance AI models. The session will cover techniques and real-world applications, such as training LLMs, customizing chatbots, and optimizing RAG-based systems. Only a few weeks remain to secure your in-person or virtual pass to ODSC West, the leading technical AI conference, where you’ll master new AI skills and make meaningful connections with AI practitioners, experts, and founders. ? Don’t miss out—register now: https://lnkd.in/drjNivMk

Integrating and deploying GenAI can be challenging, but at the?GenAI X Summit, our world-class AI experts will teach you how to make it easier. Dive deep into topics like model security, quality, and scaling during insightful talks and panels. Meet a few of our attending experts below! ??Andy Hock, PhD, Senior Vice President, Product & Strategy, Cerebras Systems Andy Hock is the SVP of Product and Strategy at Cerebras Systems. Previously, he served as Senior Director of Advanced Technologies at Skybox Imaging, which was acquired by Google for $500M. After the acquisition, Andy continued as Product Manager at Google before joining Cerebras. With a PhD in Geophysics and Space Physics from UCLA, Andy brings a wealth of expertise in advanced technologies and AI. Talk:?Developing and Deploying State-of-the-Art AI: The Power of Wafer-Scale Engines ??Rehgan Bleile (Avon), Co-Founder & CEO, AlignAI | Founder, Women in Analytics (WIA) Rehgan Bleile is the Co-Founder and CEO of AlignAI, an AI Adoption and Governance Platform helping enterprises quickly and safely unlock AI’s value. With a decade of experience, she has designed AI solutions for highly regulated industries. Rehgan is also the Founder of Women in Analytics, a global community of over 7,000 members, aimed at elevating diverse voices in data and AI. She is a sought-after speaker, having keynoted at over 50 conferences. Talk:?Quantifying the Value of AI: Going Beyond Cost Savings ??Alex (Wei) Chen, PhD, Founder & CEO, Nexa AI Alex Chen is the Founder and CEO of Nexa AI and holds a PhD from Stanford. He is the creator of Octopus v2, an on-device AI model that is 4x faster than GPT-4o and 70x more energy-efficient. Octopus v2 ranked 2nd on Hugging Face and was featured at Google I/O. Nexa AI focuses on building on-device models and toolkits for developers and enterprises. Talk:?Getting Started with On-Device AI: Models and Local Inference SDK It’s time to take your GenAI models from production to deployment. Don’t miss this chance to build the necessary skills and knowledge! ?Register now:?https://lnkd.in/ec89ATfC

?? HELLO from Day 1 at TechCrunch Disrupt 2024! We’re thrilled to be here and can’t wait to meet you! Swing by Nexa AI's booth at L8 (in the SaaS, Enterprise + Productivity section) to explore our latest on-device AI models and demos. Let’s chat about your AI journey! ?? Snap a photo at our booth and tag us—we’d love to see it! ??? Catch our pitch on Day 3, October 30th, at the SaaS, Enterprise + Productivity Stage. See you there! #TechCrunchDisrupt2024 #NexaAI #OnDeviceAI #SaaS #AIInnovation #EnterpriseAI

Update: We are #1 trending at r/LocalLLaMA ?? https://lnkd.in/gptq7THg ?? What could local RAG system with 1B/3B LLM really do in real world? David Qian and I have been exploring what small LLMs can actually do with local document processing. It's one of the most wanted features for the local AI community - chatting with documents we can't/won't upload to Claude or ChatGPT. We built a local RAG system (you won’t need any API or Internet connection to run it) with Nomic AI embedding model + AI at Meta Llama 3.2 3B + LangChain RAG workflow + Nexa AI SDK ?? Here is what we found: Simple Q&A works surprisingly well - we tested with Nvidia's 9-page financial report and every direct QA is accurate and fast. Except the small language model struggles with the reasoning in more complex questions. ?? LoRA could push the limit of small language model: Since training a search-optimized model takes lots of time, we trained two LoRA adapters for generating pie/column charts as proof of concept. We also used Octopus_v2 action model (https://lnkd.in/gg6Qb8B5) to route different models based on user's intents?? ?? When the model sees <pdf> tag → RAG search ?? When the model sees "pie chart" or "column chart" → generate graphs ?? The project opens interesting possibilities for us: we could have one small 3B model with different LoRA "plugins" for specific tasks. Like a lightweight model wearing different hats ??. The project is open-sourced if you want to try ?? https://lnkd.in/gwhRXKS2

We all ?? open-source models, but inconsistent benchmarks often make it hard to choose the best quantized version for your task and device. With the latest release of Nexa SDK, you can now ?? benchmark GGUF models with ONE line of code — with ease, speed, and support for multiprocessing and 8 types of evaluation tasks. ?? Full docs: https://lnkd.in/gzyanH6b ?? Follow the example and try for yourself: Benchmarking AI at Meta's Llama3.2-1B-Instruct Q4_K_M quant on the "ifeval" dataset for general language understanding in 80 minutes on a 4090. ?? Steps: 1. Install Nexa SDK ? https://lnkd.in/g-By9MfX 2. Install Nexa Eval package ?? pip install 'nexaai[eval]' 3. Choose the GGUF model from ?? https://nexa.ai/models 4. Run one line of code ?? nexa eval Llama3.2-1B-Instruct:q4_K_M --tasks ifeval --num_workers 4 ?? Follow us and give us a star: https://lnkd.in/gX2E7CiT — more updates are on the way! ?? Pin us for any feedback and requests: https://lnkd.in/dUMJAXCU.