How do you train and update AI models on edge devices?

One of the best use cases for Federated Learning is the healthcare sector. One of the main issues when doing data science and ML in this industry is the limitation in terms of data sharing. It's challenging to obtain healthcare data due to various privacy and security limitations (e.g. HIPAA). Doctors and hospitals are particularly resistant to sharing patient data despite privacy measures put in place because of the potential risks. Federated Learning, instead, allows moving models, instead of data, around. Each model can be trained on a single hospital's patients and only the coefficients are shared to generate a single model. This allows to account for the entire population, mitigating bias, while ensuring patients' data are safe.

Useful methodology in any industry where collaborative processing of sensitive data distributed across multiple entities is needed: for example in healthcare, where patient data is highly sensitive and subject to strict privacy regulations (such as HIPAA), or in the industrial and energy by enabling secure data sharing spaces between organizations. Edge AI tools such as Barbara allow training models directly on edge devices while preserving data privacy. In my experience, FL proved invaluable in all types of organizations by enabling training without the need for centralized storage or processing and its ability to leverage local data, minimizing data transfer and ensuring compliance with regulatory requirements.

While incremental learning presents a more efficient way to update AI models, it also requires careful management of model complexity, memory, and stability. It is crucial to strike a balance between adapting new data and retaining previous knowledge, so the models can provide accurate and reliable predictions or classifications.

A water treatment customer with plants around the world implemented incremental learning on their edge devices using our Edge application orchestration platform, enabling their predictive models to continuously adapt to new data and improve their performance over time. In my experience, incremental learning has been instrumental in keeping AI solutions up to date in dynamic environments where data distributions evolve frequently. One utility I have found at the professional level with incremental learning is its efficiency in updating models on edge devices, reducing the need for extensive retraining and speeding deployment time for new AI applications.

Knowledge Distillation is a kind of transfer learning that can help edge devices use the knowledge of a teacher in which trained by a large dataset. In the scope of healthcare, it also helps privacy and security issues. Teacher transfers acquired knowledge to the edge devices, not the data itself. So, students can recognize unseen local cases with the knowledge of teacher.

Leveraging transfer learning to streamline the deployment of AI solutions in our edge infrastructure allows us to capitalize on pre-trained models and customize them for specific tasks or domains. In my experience, transfer learning has significantly reduced the computational resources required to train models on edge devices, enabling rapid iteration and adaptation to changing business needs. One utility we have found with transfer learning is its flexibility in reusing existing models for new applications, which facilitates scalability and cost-effectiveness in deploying AI at the edge.

These frameworks, such as TensorFlow Lite, PyTorch Mobile, ONNX Runtime, and Edge Impulse, offer features like model compression, optimization, conversion, inference, and monitoring to improve efficiency, compatibility, and robustness of AI models on edge devices. By using edge AI frameworks, developers and machine learning engineers can train and update AI models on edge devices, facilitating more portable and self-contained technology. Furthermore, these frameworks often come with development tools, pretrained models, and cloud-native suites of AI and data analytics software, making it easier to integrate AI applications into edge computing environments.

Having state-of-the-art edge AI frameworks such as Tensorflow, Pytorch, ONNX... allows us to streamline the development and deployment of AI applications in our distributed edge network. In my experience, Edge AI frameworks have provided us with the necessary tools to optimize model performance, and resource utilization on edge devices and accelerate deployment to production. One utility I have found at the professional level with edge AI frameworks is their seamless integration with the Barbara platform, which enables end-to-end management of AI solutions from development to deployment and maintenance. Allowing companies to focus on the development of new models and use cases and not worry about the underlying infrastructure.

When implementing best practices for training and updating AI models on edge devices in your company, it is necessary to include robust data management procedures and regular performance monitoring to ensure the reliability and effectiveness of our AI solutions. In my experience, implementing a best practice protocol is critical to maintaining the quality and integrity of AI models deployed at the edge, minimizing downtime and maximizing operational efficiency. One usefulness I have found at a professional level with best practices is their role in mitigating risks associated with deploying models on edge devices, such as data leaks or algorithmic biases, ensuring compliance with industry standards and regulations.

Edge-Cloud Collaboration: In some cases, it might be feasible to offload part of the training process to more powerful cloud-based servers. The edge device can send data to the cloud for model training and receive updated models in return. This approach reduces the computational burden on the edge device while still allowing it to benefit from more sophisticated models. On-Device Data Augmentation: Data augmentation techniques, such as random cropping, rotation, and flipping, can be applied directly on the edge device during training. By generating augmented data on-the-fly, the need to store and transfer large amounts of augmented data can be avoided.

It's worth noting that the specific approach to training and updating AI models on edge devices can vary depending on factors such as the device's computational capabilities, connectivity, and specific application requirements. Different techniques and optimizations may be employed to strike a balance between model performance, resource constraints, and energy efficiency on edge devices.