Web AI Monthly #19: Meta's 8B Llama3 LLM in browser, let's meet IRL, and a new Web AI website with the Chrome team

May the 4th be with you! Yes, May is upon us, and many exciting events are lined up. Maybe we will even cross paths IRL this year? Read on below to find out where we shall be speaking and demoing! In this edition we have some seriously cool new models and resources for you to check out.

If you enjoy the content, please do give us a share with friends, colleagues, family - everyone is welcome.? I aim to centralize the community with all the amazing work being produced in this space and bring light to our most awesome creations. Tag me (Jason Mayes) if you make something noteworthy for future editions so I can help get eyes on your Web AI work - many readers work for top global tech companies or high growth startups. We have subscribers ranging from decision makers (think C-level, VPs, and Directors) to folk on the frontlines using this stuff day to day (SWEs, web engineers and researchers). You never know who may see your creations. Alright, lets go!

Another key LLM gets ported to Web AI: Meta's 8B Llama3

Breaking news, in case you didn't have enough LLMs to choose from already, the amazing folk at MLC (hat tip to Charlie Ruan ) ported Meta 's newest 8B parameter LLM to run entirely in the browser, client side (no cloud needed for inference) with WebGPU on the same day AI at Meta launched! Incredible efficiency by the community there - well played.

This 8 billion parameter model is a really nice size that even runs at a great speed on my very old NVIDIA 1070 GPU. For me this represents a serious win as many gamers or such are still sitting on 1070/1080 cards from way back when, so this means even more folk can use this great tech on device with a single GPU. See it in action - captured in real time, below:

I guess you want to try it out eh? Well head on over to https://webllm.mlc.ai/ to do that. Be sure to select the right model from the drop down before you send a chat message - after which it will download and cache the selected model. Be warned this is several GB in size, but once loaded, it will load much faster on the 2nd page load.

AI and the Web - a new website

I'm also pleased to announce a collaboration with the Chrome team (shout outs to Alexandra K. , Maud Nalpas , André Bandarra , Paul Kinlan ) who are helping to create a wonderful centralized resource for Web Devs who use AI.

This brand new website will be different to all the existing Python biased AI docs online, and instead be from the perspective, needs, and wants of a web engineer / developer - actually putting the 70% of engineers who are JS developers top of mind with an AI lens focusing on tasks and processes that matter to us:

It's clear that web folk have different requirements and expectations when using AI client side in the browser vs folk who may be working on the server side with near infinite resources at their disposal by contrast. The models we may chose may not be the same as a researcher looking to get an extra percentage point in accuracy when training a new state of the art model.

I hope that longer term we can try and narrow down some of the complexity for you all when choosing a model to use for various on device scenarios for a given target vertical or use case.

After speaking to many of you it's clear that documentation and benchmarking of AI models is not obvious or consistent, and it would be nice to have some sort of opinion / comparison for popular model types with production use cases in mind instead of research, all in one place that you can pick from, knowing with some confidence it would fair well in the web browser environment.

So go check out the beginnings of this site, but remember this is just the start, not the end - so we welcome feedback for what you want to see more of on such a site:

https://web.dev/explore/ai

Bookmark and check back regularly as we continue to add articles throughout the year touching on new topics that matter in the space.

?? Music to my ears with MusicGen model

As stated by Joshua Lochner :

"The competition in AI music generation is heating up, with Suno and Udio leading the way. Unfortunately, neither are open source... ?? Luckily, MusicGen is! ?? The quality is amazing and you can even run it locally in your browser with Transformers.js!"

Well, I couldn't put it any better, so quoted his tweet above, which you should totally check out to hear the beats this model can produce right in the browser all locally!

Joshua used the prompt "slow lo-fi music with a soothing melody, chill with organic samples and rain in the background" and ended up with the result you can hear on his tweet here: (volume up!) https://twitter.com/xenovacom/status/1780253106460143995

Even better you can try it yourself using this link:

https://huggingface.co/spaces/Xenova/musicgen-web

Already front end engineers are making really sweet interfaces to this sort of model, like this one by Enzo Esteves .

Try this version for yourself here: https://huggingface.co/spaces/enzostvs/ai-jukebox

Web AI global speaking tour begins - let's meet!

So I hear you wanted to meet IRL? It was a pleasure meeting many of you already at Google Cloud Next back in April. Well, I hope to cross paths with even more of you this year if you're able to make any of the following events:

Google IO - USA

14th/15th May 2024, Mountain View, USA.

I will be speaking live at our annual Google IO event - the largest event of the year for 谷歌 where we launch all the coolest tech. This year I'll have 2 talks for your viewing pleasure:

1. Web AI: New models, tools, and APIs for your next web app - my annual updates for the year! Google's Web AI libraries and models have crossed 1 billion downloads - learn what's new with the state of Web AI at Google and discover Gen AI models you can run client-side in the browser, along with new tooling and APIs for your company's next web app. I'll cover key web ML updates from Google Chrome , TensorFlow.js, MediaPipe Web, Core ML, Visual Blocks, and beyond. Suitable for everyone.
2. Visual Blocks: Bring AI ideas to life with custom nodes for your APIs - Visual Blocks ML is a powerful low-code solution that provides more than 70 nodes you can arrange or connect together to bring any AI powered-idea to production faster. This year, we launched support for defining your own custom nodes that can potentially integrate with any service, model, or code you can dream up. Learn how to make your own nodes that work seamlessly with existing ones and your own products and services in this workshop, opening up creative workflows that can be shared with the community or used with your internal teams. Suitable for intermediate / advanced JS developers who want to enable their wider team / company to work better together.

Learn more / register: https://io.google/2024/ (search for my name to find my talks above - Jason Mayes).

JS Nation - Amsterdam

13th - 17th June 2024, Amsterdam, Netherlands.

Super pumped to be speaking at a huge JS conference this year - JSNation ! With 1500 attendees in person, and about 10K online, should be a great show. Get a little teaser from me here in their LinkedIn post.

You can add my talk to your calendar using this link if you're going!

Google IO Connect - Netherlands, India, China

This year there will be 3 IO connects that I shall hopefully be speaking / demoing at too. See if any of the following align with your calendars:

27th June 2024, Berlin, Germany

Register for Berlin here

17th July, Bengaluru, India

Register for Bengaluru here

7th - 8th August, Beijing, China

Register for Beijing here (coming soon)

I hope to meet more of you IRL this year to help answer your questions and show you some of our latest demos that we launched at IO this year. Do tag me in your posts if you are going to any of these so we can arrange a group meetup!

See you next time!

If you're new to this space and want to learn Web AI, you can get started fast with my free Google Developers course here (no background in AI needed, just a love for JavaScript and curiosity for AI). Or get inspired via our Show & Tell - I got you either way!

See you next time with even more great content and please do tag me (Jason Mayes ) if you make or find something for future editions - I need your help lovely #WebAI community as things are moving so fast!

Cheers!

Jason Mayes