How do you tackle complex edge AI scenarios?

Based on my experience, this is a very delicate decision to make. Only certain applications should be deployed as with AI at Edge devices, where necessity requires it, others should deploy AI in the cloud. All too often, edge devices are not capable of properly running AI models due to limited resources, often leading to compromises in the AI architecture which in turn leads to biased and poor performing AI models. Typically the model is constrained in its resource footprint which means the AI can not properly fit the training data that developers are attempting to train it with. In these cases, you're better off running the AI model in the cloud which offers scalability of its resources, and room to grow as the AI model evolves.

Having been in the trenches of cybersecurity for decades, I can't emphasize enough the importance of understanding your edge devices. Often, these devices are seen as mere tools to deliver results, but they are also potential vulnerabilities. It is crucial to know the device's manufacturer, understand the supply chain risks, and be aware of its firmware and update frequency. Organizations must pay more attention and thoroughly inventory their edge devices. No matter how insignificant, each device represents a point of entry or a potential weak link in your security posture. Before deploying any AI solution, a comprehensive risk assessment should be conducted to understand each device's potential vulnerabilities and threat vectors.

In the realm of wound care, understanding edge devices is crucial. These devices, which gather and process data directly at the source, can play a pivotal role in real-time wound assessment and management. Instead of sending data to a central location for processing, the immediacy of edge computing allows for rapid response, be it in assessing wound progression or identifying potential complications. For practitioners, this means quicker, more informed decisions that could directly influence patient outcomes. Leveraging edge devices effectively not only streamlines the care process but also harnesses the power of AI to make wound care more proactive and patient-centric.

In my role at Intel edge computing systems, I've led initiatives tackling complex edge AI scenarios using a structured approach. First, we define clear objectives, aligning them with business goals and customer needs. Then, we focus on data collection and preparation, ensuring high-quality data inputs for our AI models. Next, we leverage Intel's cutting-edge technologies to select and optimize AI models suitable for edge deployment. We prioritize model efficiency, balancing inference speed and accuracy while considering the resource constraints of edge devices and tailoring them to our business segment use cases. Integration is key. We seamlessly integrate AI models with Intel-powered edge devices, ensuring compatibility and scalability.

Processing Power: Evaluate the CPU and GPU performance to gauge their processing potential for AI workloads. Memory: Analyze the available RAM and storage on your edge devices. Storage: Determine the storage capacity of your devices. Interoperability: Understand how your edge devices interact with each other and with cloud services. Update and Maintenance Requirements: Determine the frequency and method of updates and maintenance needed for your devices. AI Models: Choose AI models that are tailored to your edge devices' capabilities. Data Collection: Design data collection strategies that consider device limitations. Data Management: Develop data management protocols that align with your storage capacity. .

I've often seen the allure of using the latest and greatest AI models. However, edge devices demand pragmatism. The most advanced model is not always the most suitable, especially if it requires significant computational power or memory. It's also worth noting that while techniques like quantization and pruning can make models more efficient, they can sometimes introduce vulnerabilities or make existing vulnerabilities more exploitable. For instance, reduced precision might make the model more susceptible to adversarial attacks. Always consider the security implications of any optimization technique.

SOLVE THE PROBLEM DON’T BE COOL (aka use a fancy technique that might not be appropriate). In order to tackle this step you need to, — fully under the necessity: what is the task that AI should solve? — convert the previous step in a simple question that guides your development — define KPIs, how would I measure the quality of my work? — start simple, if possible, maybe with a statistics approach to verify your approach and to onboard non technical stakeholders — keep in mind that you need to develop a solution that answers to the question at point 2 and not something new and cool — scan for web and scientific papers for inspiration and solutions, if needed — keep everything on Git or generally tracked

Choosing the right frameworks and models is fundamental for optimal performance in edge AI scenarios, especially within specialized fields like wound care. The correct framework can simplify development and deployment, while the appropriate model can determine the accuracy and efficiency of data processing. For wound care practitioners, this choice can mean the difference between an almost instant and accurate assessment and one that’s sub-optimal. As edge devices gain traction in healthcare, ensuring they're powered by the best-suited frameworks and models becomes paramount to deliver timely and precise care.

In my experience, this is where “Knowledge Distillation” makes sense. Generally, we train a SOTA model, then use Student-Teacher training to train a smaller network to learn the distribution of the large model. Now this model can be a pruned & quantised version of the original model. We can make them even smaller with “weight clustering”. Weight clustering involves grouping similar weights of the model into clusters and representing them with cluster centroids.

When choosing AI frameworks and models for edge AI, balance performance, accuracy, and efficiency. Opt for edge-optimized frameworks like TensorFlow Lite or PyTorch Mobile. Select simpler models like CNNs or RNNs for quick inference. Apply techniques like quantization or pruning to reduce model size without compromising quality. Consider real-time needs, customization, and scalability. Test thoroughly on edge devices, plan for updates, and maintenance. Striking this balance ensures effective AI deployment on resource-constrained edge devices.

Ensuring high data quality and diversity is paramount for effective edge AI applications, particularly in specialized healthcare fields like wound care. Quality data directly impacts the model's accuracy and reliability, while diverse data ensures the model is robust and inclusive. In wound care, where patient demographics and wound characteristics vary widely, capturing a diverse range of data ensures AI models can accurately diagnose and suggest treatments for a broad spectrum of patients. As edge devices in healthcare become more prevalent, rigorous management of data quality and diversity will be pivotal in achieving consistent, high-quality patient outcomes.

A mistake that a lot of organizations make is to only train their models on relatively normal and standard data. Ideally, a dataset should be extensively diverse, including edge cases and attempts to break the AI. It is inevitable that once you release an AI, people will try to break it. Get ahead of the game and try to break-fix on your own before others do! That's the purpose of high quality, diverse training and testing data. Note that you MUST use separate data for training and for testing, or you will get poor quality models that ONLY address the training data.

Monitoring of production data and model's performance is crucial in ensuring correct downstream operations, as an effective monitoring scheme would trigger timely updates to the model. However, being at the edge can bring about challenges in monitoring mechanisms. A typical human based monitoring would consider a large bandwidth to pipe back data for this purpose, defeating the purpose of having the model in the edge in the first place. One could consider automating the monitoring mechanisms that would reduce the need for human intervention. This is unfortunately hard to achieve today considering the monitoring technique currently available, which requires some form of human interpretation to be correctly utilized.

Data quality and diversity are essential aspects of any data-driven project throughout the data lifecycle. Data diversity greatly enhances the robustness, representativeness, and generalizability of the data analysis. Data diversity helps in training the model to detect more anomalies thus playing the key role in increasing the efficiency of the model.

Representative, relevant, and reliable data are essential for effective edge AI applications. Employing augmentation techniques like cropping and flipping can bolster data variety. Validation methods such as checksums and filters aid in detecting and rectifying errors. Moreover, utilizing data federation approaches like federated learning and differential privacy promotes secure collaboration among edge devices. Overcoming these challenges is pivotal for maximizing the accuracy and reliability of edge AI systems.

Deploying and updating AI models efficiently is critical, especially in dynamic fields like healthcare. In wound care, where timely intervention can drastically impact patient outcomes, having the most up-to-date AI models ensures that diagnoses and treatment recommendations are based on the latest data and research. This continuous model evolution not only improves accuracy but also allows for adaptation to new wound care techniques and technologies. Efficient deployment also minimizes disruptions, ensuring that healthcare professionals have uninterrupted access to AI tools when making clinical decisions. Streamlined deployment and updating mechanisms will be key in delivering consistent, cutting-edge care.

Model deployment and updates are areas ripe for potential security breaches. Over-the-air updates, while convenient, can be intercepted or tampered with if not adequately secured. And in my years as a Security Architect, I've seen many instances where outdated models with known vulnerabilities were still running because of the inertia around updates. A principle I always advocate for is "Trust, but Verify." Even when deploying updates from trusted sources, always have mechanisms in place to verify the integrity of the updates. Implementing signatures and a secure update process can prevent many potential pitfalls.

Efficient model deployment and updates are vital in edge AI. Ensure compatibility with conversion tools like TensorFlow Lite Converter or PyTorch Mobile Converter. Use management tools (e.g., TensorFlow Hub, PyTorch Hub) for consistent model storage and distribution. Keep models current with update tools like TensorFlow Lite for Microcontrollers or PyTorch Mobile Updater, enabling over-the-air or incremental updates. Rigorously test updated models, maintain version control, and bolster security measures. Optimize distribution for timely updates. This approach ensures models are compatible, consistent, and up-to-date for successful edge AI.

My view is that appropriate governance structures should be in place to support model deployment and update in BAU, especially considering the complexities that arise from the distributed nature of edge AI. As with common MLOps best practice, maintenance and performance preservation procedures should be operationalised in a way that is manageable and compatible with the operating model of the organisation.

How data is gathered & connected together with a data governance framework is a must! We need to have a clear idea on how the different data from the different data sources flow and how they are processed and analysed.

In my experience, we need to strike a balance between cost, performance, latency of the AI applications while deploying AI model/application in edge computing. There is no one size fits all, some models need to be light weight to be deployed on mobile and require less latency to serve the users , that’s ideal use case of chasing edge computing. Having board level alignment in terms of cost associated with it at pretty early stage helps successful model deployment at edges.

Edge AI is a domain where the convergence of AI and cybersecurity becomes extremely pronounced. The attack surface expands as we push more processing to the edge. Pragmatism, thoroughness, and a security-first mindset are critical in navigating the challenges of Edge AI.

Energy Efficiency: Optimize algorithms for energy efficiency, crucial for battery-powered devices. Device Security: Implement robust security measures to protect AI models and data. User Experience: Ensure AI enhances usability without complexity. Monitoring and Maintenance: Proactively address issues for long-term reliability. Ethical Considerations: Address bias, fairness, and transparency in AI decision-making. Collaborative Partnerships: Seek collaborations for innovation and best practices. Skill Development: Invest in team skills to stay current in edge AI. Cost-Benefit Analysis: Assess ROI for edge AI investments. Scalability: Plan for growth to accommodate increased workloads.

Choosing the right edge device: There are many edge devices available in the market from different vendors. Some vendors offer end to end deployment and maintenance services and some just offer devices. We need to understand what problems we are trying to solve and what type of device can help to solve it in an optimal way.