The Other Machine Learning Revolution
While the world has been focused on the advancements in Large Language Models such as chat GPT, there is a much quieter revolution happening in the world of smart sensors and IoT devices. The introduction of Machine Learning (ML) algorithms into microcontroller software is changing the game for these devices, making them even smarter and more efficient. In this article, I would like to give a brief overview of the key advantages of adding ML algorithms to small embedded systems and a review of the chief design considerations.
Today microcontrollers are at the heart of the vast majority of electronic devices. Increasingly many of these devices are connected to cloud servers as part of an Internet of Things (IoT) network. While it is possible to transfer data to a cloud server where it can be used for ML inference there are considerable advantages to performing the ML inference on the device itself. These include real-time inference, lower overall power consumption, enhanced privacy and greater reliability.
Machine learning algorithms can enable real-time decision-making within the microcontroller. This is particularly useful in applications where quick responses are required, such as robotics, autonomous systems, or smart devices. The microcontroller can process data locally and make decisions without relying on external systems, reducing latency and improving overall performance.
Power consumption cuts two ways. Firstly a typical ML algorithm performs a large number of integer or floating point calculations, while this can be mitigated through hardware support it will take additional energy and compute time. However if we are sending data to a cloud server as in the case of a typical IoT system, then the amount of data transmitted will be greatly reduced as we will only need to send a few bytes of classification data rather than Kilobytes of image data. This means that the overall power budget will end up being lower.
Another big plus is that by running the ML algorithms directly on the microcontroller we can improve privacy and security. Data can be processed locally, eliminating or reducing the need to send sensitive information to the cloud or external servers. This reduces the risk of data breaches, ensures confidentiality, and protects user privacy.
Finally, with machine learning algorithms on the microcontroller, it can continue to function even when disconnected from the internet or other external systems. This offline operation is particularly valuable in remote or isolated environments where connectivity is unreliable or not available. The microcontroller can still perform its tasks and make decisions based on the learned models, ensuring uninterrupted operation.
So how can we enter this brave new world??Here we must consider four different factors, low-cost microcontroller hardware which is powerful enough to support the use of an ML algorithm, a suitable design methodology which can target such a constrained system. We also need to collect, clean and label a representative data set to train the ML algorithm. Finally, we must consider device security to protect the ML model from malicious attacks.
To consider our first-factor hardware, the last 20 years have seen huge gains in the performance of typical microcontroller processors. In ?2004, Arm introduced the Cortex-M 32-bit processor family. This helped create a powerful new generation of low-cost microcontrollers. Since then the Cortex-M processor family has been wildly successful and has become established as the dominant MCU processor architecture. As new processor variants have been introduced there have been several updates to the Cortex-M series that have greatly increased its maths processing power. The Cortex-M4 processor introduced hardware floating point support and the ability to perform multiple integer calculations in parallel with the “Single Instruction Multiple Data”(SIMD) Instructions”. With the latest architectural revision, this SIMD support now extends to the floating point capability with the addition of the Helium Vector extension and additional instruction set support for low overhead looping. Together these provide roughly a 5 times performance boost for ML and DSP algorithms over the earlier Cortex-M4. This has made it easier to perform complex calculations on microcontrollers, which is essential for machine learning algorithms.
One of the most exciting developments in the world of microcontroller software is the introduction of the Ethos Neural Processing Unit (NPU). This unit acts as a dedicated accelerator for machine learning algorithms, making it possible to run these algorithms on even the smallest microcontrollers with around a 480 times performance boost. This means that IoT devices can now perform complex tasks such as image recognition and natural language processing without relying on cloud-based solutions. Today we are seeing the first microcontrollers based on Cortex-M55 with Helium and Ethos NPU. This new generation of devices will provide an ideal platform for future embedded ML algorithms.
Secondly, our design process will have an extra layer where the ML algorithm is designed and trained in a Python or ‘R’ language-based framework often running in a cloud service. There are several open-source frameworks available but by far the most widely used design tool is Tensorflow. Originally designed by Google for internal research work and then released under an open-source Apache license. TensorFlow provides direct support for microcontrollers with TensorFlow Lite Micro (TFLu or TFLM). TFLu allows developers to design a complete machine-learning system in Python and export it to a C++ framework, with a C API if necessary. This makes it easy for developers to create complex machine-learning models without needing to learn a new programming language. ?Arm also provide a Fast Model simulator for each of their Cortex-M processors. This allows you to easily prototype and test the performance of an algorithm running on a range of processors before deciding on a final target. Each of the fast models is configured with a large amount of code and data memory, this allows you to test the performance of an algorithm before optimising it to run or a real-world constrained microcontroller.
领英推荐
The third critical factor is to have enough real-world data to train an ML algorithm, generally, as much as we can get our hands on. This data also has to be labelled so it can be used in supervised training. This can be a costly and time-consuming process. Fortunately, there are a couple of shortcuts available. It is possible to take an existing model and transfer-train it to fit your application. This uses the first few layers of the existing model and adds a new ‘head’ layer which can then be trained with a much smaller labelled data set. Similarly, an autoencoder can be used to pre-train the initial layers of an Artificial Neural Network (ANN) or Convolutional Neural Network (CNN) using unlabelled data to create a “Latent Representation” and then an output layer can be added and trained with a much smaller labelled data set.
Once deployed it is possible to continue logging data to further refine the model. This allows your design to improve and even adjust to changing conditions over time. As your system becomes more refined your ML model becomes quite a valuable asset and needs to be protected from malicious attackers, particularly in an IoT system. Consequently, it is important to base your design on a strong security model such as the Arm Platform Security Architecture (PSA). which protects your investment from software and network attacks. For any connected device, an appropriate security model is not optional and must be added at the start of every project as it becomes more difficult and costly to add later on.
In conclusion, the introduction of machine learning algorithms into microcontroller software is changing the game for smart sensors and IoT devices. The Cortex-M series provides a powerful platform for microcontroller software development, while the Ethos Neural processing unit acts as an accelerator for machine learning algorithms. TensorFlow Lite Micro makes it easy for developers to create complex machine-learning models, while also allowing for real-time decision-making and reducing the need for cloud-based solutions. Now we have reached the point where incorporating ML algorithms into microcontroller applications offers the benefits of enhanced intelligence, real-time decision-making capabilities, and optimized resource utilization, enabling efficient and autonomous functionality in edge devices.
?
Want to learn more?
Join or webinar on 6th June 2023 for an overview of ML on Arm MCU
We are also presenting a short course on Practical ML for Arm MCU over the following dates:
For more information visit our website: www.hitex.co.uk or get in touch: [email protected]. You can also connect with us: @hitexuk (Facebook & Twitter)