TensorFlow.js Monthly #7: RoboFlow.js, Coral Edge TPU acceleration for Node.js, and OCR recognition in the browser

Hello Tensors!?In this 7th installment of TensorFlow.js monthly we have some great new advances and community contributions to discuss! Remember, if you enjoy this newsletter?consider letting your colleagues / friends know to sign up for the future! A little love goes a long way when its just one guy and his ramblings on something he loves talking about. Thanks.

RoboFlow enables thousands of developers to use computer vision with TensorFlow.js

Roboflow?lets developers build their own computer vision applications, from data preparation and model training, to deployment and active learning.?Over 100,000 developers build with Roboflow’s tools and TensorFlow.js makes up a core part of?Roboflow's deployment stack?that has?now powered over 10,000 projects?created by developers around the world.

In order to provide the best user experience, they needed to be able to run users' models directly in their web browser instead of requiring a round-trip to their servers. The three primary concerns that motivated this decision were latency, bandwidth, and cost.

Example: Magic: The Gathering card recognition

Roboflow powers?SpellTable's?Codex?feature which uses a computer vision model to identify?Magic: The Gathering?cards as shown below:

Example 2: Label Assist

Label Assist is a?model-assisted image labeling tool?that lets developers use their previous model's predictions as the starting point for annotating additional images.

One way Roboflow users leverage Label Assist is in-browser predictions as shown:

The easiest way to try Roboflow.js is by using "Preview" on?Roboflow Universe, where they host over 7,000 pre-trained models that users have shared. Any of these models can be readily built into your applications for things like?seeing playing cards,?counting surfers,?reading license plates, and?spotting bacteria under microscope, and more.

On the Deployment tab of?any project with a trained model, you can drop a video or use your webcam to run inference right in your browser. To see a live in-browser example, give this community created?mask detector?a try by clicking the “Webcam” icon:

Pretty cool right? What are you waiting for - go try it out right now or read their amazing blog post write up they just published on the TensorFlow blog (and then come back to my lovely newsletter for more).

Use Coral Edge TPUs to run TFlite models in Node on a Raspberry Pi with TensorFlow.js

Our latest TensorFlow.js codelab by Matthew Soulanille from the TensorFlow.js team here at Google shows his amazing work that enables you to use the popular USB Edge TPU devices (Tensor Processing Units - like a GPU but aimed at accelerating machine learning models instead of computer graphics) on a Raspberry Pi to execute TFlite models via Node.js to get faster performance. Check this demo out which goes from 18 FPS to almost 50 FPS at the tap of a button:

If you love JavaScript and want to stay in Node.js on a Raspberry Pi instead of Python then you know what to do: go ahead and try his codelab out right now. Although the harder thing to get your hands on right now is an Edge TPU to use which are in short supply - but if you get one, go have at it.

Mindee enables OCR in the browser using TensorFlow.js

Optical Character Recognition (OCR) refers to technologies capable of capturing text elements from images or documents and converting them into a machine-readable text format.

Mindee, have developed an open-source Python-based OCR called?DocTR, however they also wanted to deploy it in the browser to ensure that it was accessible to all developers - especially as?~70% developers choose to use JavaScript.

They managed to achieve this using the?TensorFlow.js API, that resulted in?a web demo? you can now try for yourself using images of your own:

Want to learn more? Read their full blog post write up over on the TensorFlow blog and get started fast.

New Book Alert: Deep Learning with TensorFlow.js Projects

I have not had the pleasure of checking out this new book yet, but I was recently tagged in Umang Sharma 's new post announcing his latest book that brings another educational resource to our community. A quote from Umang is as follows:

"The book is for everyone, beginners to intermediates, and even experts. Each chapter consists of a concept and a web-app that uses it. Its been made to make it fun for the reader. You will learn end to end Deep learning with the book"

If you have a moment to check it out - feel free to take a look and let me know what you think. It may only be available on Amazon India right now but let me know if you find it in other locations too.

On a related note I am keeping track of all published TensorFlow.js books (we have quite a few now) over on my Discord channel - if you need an invite to join, use this link.

New #MadeWithTFJS videos are up!

BrainChop - 3D MRI segmentation

Learn how Mohamed Masoud created a 3D MRI brain segmentation tool powered by TensorFlow.js right in the browser for anyone to use. Amazing. This is different to the MedSeg.ai system I featured in a previous newsletter so check out this awesome video interview!

Visual spell check for designers

Also see how JooHyung Park is researching how to create a "spell check for designers" via a Figma plugin using JavaScript - early stages but amazing work and excited to see where it goes. If you can get involved do reach out to him.

New York, New York!

Finally it was a great pleasure heading over to New York this last month to meet with a whole bunch of folk from creative agencies, speaking at API Days, to an amazing meetup for New Yorkers to come ask me questions about Web ML live in person. I even gave a rather in the moment speech in a public square due to popular demand and gathered a reasonable size crowd who had many questions after! Great meeting you all and hope to come back soon :-)

Seen something cool? Send me your finest TensorFlow.js finds

Any help pointing me to amazing TensorFlow.js finds you see on your travels is a great help. If you have made something cool or seen a demo out in the wild, be sure to tag it with?#MadeWithTFJS or #WebML?on LinkedIn / Twitter / social so we can find it for a chance to be in our newsletter, future events, or even our?YouTube show and tell!

See you next month with even more great #WebML content that's #MadeWithTFJS. Cheers!

Jason