Web AI Monthly #20: A collaboration with Hugging Face, global talks, and Microsoft / Intel launch WebNN / WebGPU playgrounds

An ask to readers: If you enjoy the content, help me help you, by giving us a share. Let's centralize the community with all the amazing work being produced in this space and shine light on your most awesome creations so people who care can see them.

Tag me (Jason Mayes ) if you make something noteworthy for future editions so I can help get eyes on your Web AI work - many of my readers work for top global tech companies or high growth startups. We have subscribers ranging from decision makers (think C-level, VPs, and Directors) to folk on the frontlines using this stuff day to day (SWEs, web engineers and researchers). You never know who may see your creations. Alright, lets go!

WebNN Developer Preview

Great news - 微软 created a website to explore the emerging WebNN (Web Neural Network) web standard. Run ONNX models in the browser - such as diffusion models, segmentation, or speech recognition - all accelerated by WebNN.

This developer preview unlocks interactive machine learning on the web that benefits from reduced latency, enhanced privacy and security, and GPU acceleration from DirectML. To run the samples on their site, ensure the WebNN flag is enabled in about:flags for your browser. So what are you waiting for?

Try the demos: https://microsoft.github.io/webnn-developer-preview/

Contribute via Github: https://github.com/microsoft/webnn-developer-preview

Web AI Showcase

You are going to want to bookmark this one for the future too! In a similar nature, 英特尔 also created a #WebAI showcase for hot demos that leverage not just WebNN, but also WebGPU, and Web Assembly.

It's really great to see other companies creating centralized areas we can go to to find the hottest demos they've found for the space too.

View the website: https://intel.github.io/web-ai-showcase/index.html

Contribute via Github here: https://github.com/intel/web-ai-showcase

Hugging Face Visual Blocks collaboration

I am pleased to announce that this year I collaborated with Hugging Face to bring 16 new nodes to the #VisualBlocks ecosystem.

For those of you new to Visual Blocks you can learn more about what that is in this video . TLDR: Ridiculously rapid ML/AI prototyping that allows you to go from idea to working demo faster - we even use it at 谷歌 internally.

So what new nodes were added?

Great question. Let's walk you through them - all of which you can try live in your browser with a click of the link - it will open in Visual Blocks to try immediately! So cool.

Client Side Nodes (Web AI powered):

All of the following nodes work entirely on your device - no cloud needed after the download! I dare you to click one of the following links - its a lot of fun:

1. Image Segmentation - click on an image and segment the object you clicked on. Even better you can click multiple areas to combine and refine segmentations!
2. Text Classification - perform tasks like sentiment analysis on a text input.
3. Object Detection - find objects in images with their bounding boxes displayed.
4. Image Classification - classify what is the main object in an image but no bounding boxes returned.
5. Depth Estimation - for any given 2D image, convert it to a 3D one by predicting the depth of every pixel in that image. 3D photos here we go!
6. Token Classification - for a string of text, extract meaningful "tokens" such as names of people, places, companies etc.
7. Translation - as you may expect, with this you can translate a string of text in one language to a different language.
8. Background Removal - for a given image, remove the background from the foreground object. Works on things beyond humans too.

Server Side Nodes (Hybrid powered - could compliment a Web AI implementation)

Hugging Face also created 8 server side powered nodes too that are "task based" - meaning it opens up thousands of server side models from Hugging Face to be usable within Visual Blocks even though they have not yet been converted to Web AI formats. This is a great idea so we as JS devs can at least play and explore these models with existing pipelines that we may have. Learn more on their website for what tasks are supported.

Learn more and try yourself

So, some really amazing work by Joshua Lochner and Radamés Ajna - it was a pleasure working with you on this! Also kudos to the Visual Blocks team who helped fix some bugs we found - Matthew Soulanille , Jing Jin , Na Li .

Check their blog post here: https://huggingface.co/blog/radames/hugging-face-google-visual-blocks

Learn more about the nodes here: https://huggingface.co/hf-vb

New Web AI talks to catch up on

Wondered why there was no June newsletter? Well, it was a rather busy month meeting many of you IRL across several talks in Europe and USA. Missed them? Fear not - I have links for you to catch up with the state of Web AI below!

JS Nation - Amsterdam

It was a pleasure meeting over 1,500 JavaScript developers at this year's JSNation over in Amsterdam. From a keynote on Web AI, to a roundtable on AI on the web generally, and even a 1-1 interview with the GitNation organizers that will come out soon - it was a truly incredible experience and I highly recommend attending next year if you live and breath Web / JavaScript.

Keynote talk: Web Apps of the future with Web AI

For those of you who could not make it this year, a full recording of the talk I gave is now online to view.

Please do leave any feedback in a comment, or DM if you prefer. This year is *the year* to expand your toolkit with a splash of AI no matter what industry you are working within before you get left behind! Thank you Lera Chizhik (Zaitseva) for inviting me and putting on such a great show.

Watch the talk now: https://portal.gitnation.org/contents/web-apps-of-the-future-with-web-ai

Google IO - USA and Germany

We also made our way to USA (Mountain View) and Germany (Berlin) to give a more in depth tech talk (and demos throughout the day!) on the latest updates from Google for 2024 at IO and IO Connect.

From showcasing Visual Blocks custom nodes with our Hugging Face collaboration to running 2B and 7B Gemma LLM models entirely client side in the browser, we covered a lot of great updates. If you missed both, check the links below to catch up.

Web AI: The future of web apps

Google's web AI libraries and models just exceeded 1 billion downloads in the past 2 years. Learn what's new with the state of Web AI at Google and discover Gen AI models you can run client-side in the browser, along with new tooling and APIs for your company's next web app. I'll cover key web ML updates from Chrome, TensorFlow.js, MediaPipe Web, Core ML, Visual Blocks, and beyond.

Grab a coffee (and a friend) and then hit the play button on the link below.

Watch now: https://www.youtube.com/watch?v=PJm8WNajZtw

Visual Blocks: Bring AI ideas to life with custom nodes for your APIs

Join me in this workshop that teaches you step by step how to make your own custom nodes that work within the #VisualBlocks ecosystem. By doing this, you can leverage and reuse the work of others on your team faster than ever, in a way that is easy to share.

Want to learn how to make your own nodes like Hugging Face did? Yeah, I thought so - just check my workshop below.

Watch workshop now: https://www.youtube.com/watch?v=zeFxny7KSlY

See you next time!

If you're new to this space and want to learn Web AI, you can get started fast with my free Google Developers course here (no background in AI needed, just a love for JavaScript and curiosity for AI). Or get inspired via my Show & Tell - I got you either way!

See you next time with even more great content and please do tag me (Jason Mayes ) if you make or find something for future editions - I need your help lovely #WebAI community as things are moving so fast!

Cheers!

Jason Mayes (that Web AI guy)