Machine Learning in Edge AI

The AI hype curve

Last year’s we have heard a lot of AI (Artificial Intelligence), ML (Machine Learning), NLP (Natural Language Processing). It is still quite high in the hype curve and is going to stay there for quite long time. The reason why it is hot now even it was invented already in 60's is that the technology is now ready realize it in practice. For sure we are still far away from the human brain capability, but it is already very useful in very many applications.

Edge is having its limitations regarding ML

If you think about it generally it is easy already in the cloud-based applications, as you have all the CPU, GPU & accelerator and memory capacity available in the world. With 5G low latency and other advanced connection methods you can extend you are able to extend the capabilities you can enjoy some of the cloud benefits in the edge as well. There is still big but if you think your devices in the edge. The CPU, accelerators. memory etc. are limited. Your power envelope is not without restrictions. Most likely you have some limit on the cost of the edge device as well and it is not free to use cloud and use the 5G network either.

GPU computing is the best option for edge ML?

Quite many are utilizing currently Nvidia's GPU -based ML or deep learning as it is very effective to use it example to train models from video or picture data due to parallel computing capabilities and GPU -based mathematics. It also very well supported with software libraries and cuda GPU -programming. But in most cases, you can't afford to have this kind of high power, expensive and not rugged solution in the embedded application.

What to look at first?

First start to think what you are aiming to achieve. What is going to improve your product and business? Are you example going to count something, steer things with voice, detect abnormalities, having predictive maintenance, geo fencing, recognize users etc.? After you have figured out what is important for you and your application it is time to look at suitable hardware & software combinations to realize your goals.

Plenty of options

It helps if you have some accelerator HW in your system to speed up the processing, but it is not a must to have. You can example use Intel Open Vino toolkit in any Intel based HW or use Intel Movidius accelerator cards as well. There is as well plenty of machine learning frameworks https://hackr.io/blog/machine-learning-frameworks (tools and libraries) available to make development of ML easier. When stepping down in power consumption and price you can as well select example one of the NXP family CPUs like i.mx8PLUS or i.mx8ULP where you find accelerators build in the CPU. Today you can do the basic ML in any device example with software from

Edge Impulse.

Future is here today!

We are going to showcase some nice examples based on NXP i.mx8PLUS Smarc platform in Teknologia 2022 and Embedded world later in June 2022. The show cases are giving a demo how to recognize user by face and how to steer crane with voice. As well we are demonstrating object tracking and geo fencing. These are real examples how you can make your device more intelligent and exploit capabilities in the software and hardware. The future is here today, and it is better to be onboard than left to a station.

Vision solution outo of the box