Jason Murray is speaking 9am Tuesday AND 10:40am Tuesday on the Green Mountain panel with our friends at JCP and Stord. You can find us at booth # 408. We can't wait to see you!
Heading to Parcel Forum today. Very excited to share more about our real world ML tech powering time in transit models for hundreds of millions of shipments. Hope to see you at 9am tomorrow. For a preview, you can visit map.shipium.com/tnt
[News] The dstack Sky service connects users with GPUs at competitive prices The dstack team has introduced dstack Sky, a managed service enabling users' access to GPUs from a wide selection of providers at competitive rates and without the hassle of managing individual provider accounts. https://buff.ly/3wQD3Zc ↓ Are you interested in AI? Check out Data Phoenix (https://buff.ly/4a5WeN2) - the global AI and Data community of 8000+ Engineers, Executives, and Founders. #DataPhoenix #AI #ML #MachineLearning #DataScience #ArtificialIntelligence #News
Connecting Event Professionals with the University of Cambridge Colleges for Unforgettable Events
#Developer pepes, you seen this? Gian Marco has collab'd on a blog bringing together insights on Meta's Llama 3.2 models & Arm's KleidiAI. If Gian Marco is involved (have you read his book #tinyML Cook book https://lnkd.in/eWJHwMdz?), you know there's going to be some really useful info. Check it out ??
It's a big day for small models! We're excited to celebrate 2 exciting new updates that bring generative AI enhancements to the Arm compute platform: ?? The Beta release of ExecuTorch ?? New quantized Llama 3.2 1B and 3B models By combining the ExecuTorch framework with the pervasive Arm compute platform, we're optimizing AI performance for faster, real-world generative AI workloads at the edge. We've put it to the test on 2 demos using text summarization and generation running entirely on the device thanks to Arm Kleidi's integration with the ExecuTorch framework. ?? Combine this with AI at Meta's new quantized model scheme, designed with ExecuTorch and the Arm CPU in mind, and we're empowering our global ecosystem of 20 million developers to build intelligent AI-based applications quicker at scale. https://lnkd.in/eSPGPgt4
Awesome work and thanks for your leadership Digant, Gian Marco, Alan and the #ExecuTorch, #KleidiAI and #XNNPACK teams at large! The open and modular ExecuTorch architecture is critical to respond to the hardware heterogeneity and diversity found in devices to make on-device AI a reality from Ray-Ban Meta stories smart glasses and Orion AR glasses to iOS and Android phones and tablets and the Meta Quest. XNNPACK is a perfect integration point for many common application libraries and it's heart to see KleidiAI integrated into XNNPACK. Personally, I've been dying to build an LLM that will run on an Apple Watch - ostensibly it's just another iOS flavor, but it comes with super stringent application size limitations. That being said, the advances brought by Llama 3.2 quantized models may be getting us ever closer to running LLMs basically even on smart watches! So head over to ExecuTorch on GitHub https://lnkd.in/gVaKEAzw and the blog https://lnkd.in/gFxPBRQJ jointly authored by our teams, then share your experience and how you're planning to use ExecuTorch with KleidiAI below, or send an ExecuTorch pull request for your exciting example/demo! #PyTorch #ExecuTorch #KleidiAI #XNNPACK #AcceleratedAI #AcceleratedLLM #Llama #ARM #Google #MetaAI #Meta
It's a big day for small models! We're excited to celebrate 2 exciting new updates that bring generative AI enhancements to the Arm compute platform: ?? The Beta release of ExecuTorch ?? New quantized Llama 3.2 1B and 3B models By combining the ExecuTorch framework with the pervasive Arm compute platform, we're optimizing AI performance for faster, real-world generative AI workloads at the edge. We've put it to the test on 2 demos using text summarization and generation running entirely on the device thanks to Arm Kleidi's integration with the ExecuTorch framework. ?? Combine this with AI at Meta's new quantized model scheme, designed with ExecuTorch and the Arm CPU in mind, and we're empowering our global ecosystem of 20 million developers to build intelligent AI-based applications quicker at scale. https://lnkd.in/eSPGPgt4
It's a big day for small models! We're excited to celebrate 2 exciting new updates that bring generative AI enhancements to the Arm compute platform: ?? The Beta release of ExecuTorch ?? New quantized Llama 3.2 1B and 3B models By combining the ExecuTorch framework with the pervasive Arm compute platform, we're optimizing AI performance for faster, real-world generative AI workloads at the edge. We've put it to the test on 2 demos using text summarization and generation running entirely on the device thanks to Arm Kleidi's integration with the ExecuTorch framework. ?? Combine this with AI at Meta's new quantized model scheme, designed with ExecuTorch and the Arm CPU in mind, and we're empowering our global ecosystem of 20 million developers to build intelligent AI-based applications quicker at scale. https://lnkd.in/eSPGPgt4
??How Gintonic AI Empowers Data Scientists A financial firm's data science team uses Gintonic to streamline ??stock market trend predictions without managing GPUs ??or building AI workflows from scratch. The platform provides ready-to-use time-series models for fast deployment, with on-demand GPU rentals for intensive experiments. This setup boosts productivity, enabling efficient data experiments while paying only for essential GPU usage. Join the waitlist: https://www.gintonic.ai/# ??
The "cost frontier" keeps shifting: With this speedup on Google TPUs, cost per million tokens of this e.g. 70B param model, is $1.77. The cost for 70B param models on Groq looks like it's at about half that, at ~$0.80/megaoutputtoken (in comparison, GPT4o is $10/M ($5 for the batch api) and Claude Sonnet is $15 (and as of today: also half-off [$7.50/M] for batch)
How can you efficiently serve open LLMs from Hugging Face on Google TPUs? Hex-LLM is a new LLM serving framework designed for TPUs, offering low-cost, high-throughput deployment for open models from Hugging Face. ?? TL;DR: ?? Token-based continuous batching, XLA-optimized PagedAttention kernels, tensor parallelism ?? Direct integration with Hugging Face Hub for model loading ?? Achieves 1510 output tokens per second for Llama 2 70B in int8 on a single TPU v5e-8 (~9.6$/h) with a latency of 26ms/token. ?? Supports Dynamic LoRA Adapters, Quantization with BNB/AWQ ???Optimized for open Models based on Meta Llama, Google DeepMind Gemma, and Mistral/Mixtral ???Available inside Vertex AI Model Garden for open Models ??Hex-LLM is currently not open source Example: https://lnkd.in/eeKq7zNW Blog: https://lnkd.in/edeeATpB
Couple of announcements in last few days from tech heavyweights Apple and Elon Musk's?x.ai?who are making an entry into LLMs but could potentially alter the future landscape On March 14th,?Apple released a paper on MM1?( https://lnkd.in/gq_gTD56 ), a multimodal large language model. MM stands for multimodal and paper mentions a family of multimodal models up to 30B parameters, consisting of both dense models and mixture-of-experts (MoE) variants. Today,?x.ai?released weights & architecture of Grok-1 ; a 314 B parameters MoE model. Its a base model (i.e. not fine tuned for any task like chat) & released under Apache 2.0. See https://x.ai/blog/grok-os
Interesting digital event on GenAI with Amazon Bedrock coming March 27th! If you are interested in learning more about Bedrock, the newly available models (Mistral AI and Claude 3 family), give it a shot!
Helping businesses in the UK innovate and grow with Salesforce solutions | ex-AWS | ex-Microsoft | ex-IBM | MBA
Unlock the Power of GenAI with Amazon Bedrock! ?? Join me on this exclusive virtual Generative AI event session I’m organising for our EMEA Startup customers. When? ?? On March 27th from 11-12:30pm GMT Where? ?? Startup Loft - Online Session - Register here: https://lnkd.in/dYBNGbMW Ayman Emam, one of our AI Solutions Architects will be demoing Amazon Bedrock, AWS's fully managed service for building next-gen AI apps powered by Foundation Models, such as Amazon's Titan, Anthropic's Claude, Meta's Llama 2 and more?? In this complimentary 1.5-hours session, you'll get: - Intro to GenAI key concepts, use cases and techniques ?? - Overview of Bedrock and integrations with models like Claude and Stability AI ?? - Live demo of Bedrock in action ?? - Plus, ways on how AWS and our Partner DoiT International can help you kick-starting your AI projects ?? Whether you're (still) exploring AI for your business (or even want to de-risk your current AI strategy by diversifying), this is an event you don't want to miss. Secure your spot now to see Bedrock's capabilities first-hand. ?? #Bedrock #AWS #awsbedrock #DoIT #GenAI #FondationalModels #Claude cc: Graeme Stewart
Run Mixtral 8x7B Hands On Google Colab for FREE | End to End GenAI Hands-on Project". Discover how Mixtral 8x7B, a cutting-edge sparse mixture-of-experts model, outshines the 70B parameter Llama 2 model in most benchmarks with 6x faster inference speeds. This video is a game-changer for anyone looking to push the boundaries of their GenAI projects with one of the most powerful open-source models available today. Watch now and take your GenAI projects to the next level! ?? https://lnkd.in/g-SN9Gnr #analyticsvidhya #datascience #generativeai
Discover the benefits of the Cerebras AI Model studio: - Train 1-175 billion parameter models 8x faster than most other publicly available GPU instances - Enable higher-performing models with longer sequence lengths (up to 50,000!) - Train your models from scratch or fine-tune open-source models with your data Learn more about our partnership with Cerebras Systems: https://hubs.la/Q02tw4KY0
