The EU AI Act made simple: WHY Risk & Quality Management matter

Last week I introduced the EU AI Act, breaking down its core values and risk-based approach to AI regulation. While understanding the framework is essential, I wanted to take it a step further and further enrich my knowledge on how do these rules actually work in practice? What do companies need to do to ensure compliance, and how does this regulation translate into real-world applications?

This week, I’m focusing on two crucial aspects of the AI Act that apply to high-risk AI systems: Risk Management Systems (RMS) and Quality Management Systems (QMS). These are mandatory for all AI applications classified as high-risk, ensuring they operate safely, ethically, and in compliance with EU regulations.

To make this discussion more concrete, I’ll break down what RMS and QMS entail and illustrate how they apply to a real-world use case.

Risk Management Systems (RMS)

An RMS is a comprehensive process that identifies, evaluates, and mitigates potential risks associated with an AI system throughout its entire lifecycle. This means that from the initial design phase to deployment and ongoing use, developers must continuously assess and address any risks that could harm individuals or violate their rights. The AI Act emphasizes that this process should be ongoing and adaptable, incorporating new information and feedback to ensure the AI system remains safe and effective.

Quality Management Systems (QMS)

A QMS is a structured framework that ensures an AI system meets consistent quality standards. For high-risk AI systems, the AI Act requires the establishment of a QMS that covers various aspects, including:

• Pre-Market activities: this involves strategies for regulatory compliance, design control, verification, testing, and validation of the AI system.
• Post-Market activities: after the AI system is deployed, there must be processes for quality control, reporting of serious incidents, and a system for monitoring the AI's performance in the real world.
• Continuous Activities: throughout the AI system's lifecycle, there should be data management procedures, ongoing risk management, communication with authorities, documentation, resource management, and an accountability framework.

For AI systems that continue to learn and evolve, the QMS must include technical solutions to ensure they remain compliant with the AI Act's requirements.

A practical example: AI in Healthcare

Consider an AI system used in healthcare to assist in diagnosing medical conditions. Such a system is classified as high-risk because incorrect diagnoses could have serious consequences for patients' health.

• Risk Management: developers must identify potential risks, such as misdiagnosis due to biased data. They need to evaluate how likely these risks are and implement measures to mitigate them, like using diverse and representative datasets to train the AI. Continuous testing is essential to ensure the AI performs accurately across different patient groups.
• Quality Management: a QMS would ensure that the AI system is designed following strict quality standards, thoroughly tested before deployment, and monitored after it's in use. If any issues arise, there should be clear procedures to address them promptly.

By implementing robust RMS and QMS frameworks, developers can ensure that high-risk AI systems operate safely, effectively, and ethically, aligning with the EU's commitment to trustworthy AI.

But this is just part of the story. Starting February 2025, the first enforceable provisions of the AI Act come into effect. These initial rules focus on transparency obligations for general-purpose AI models, requiring AI providers to disclose essential information about their systems. This includes details on training data, potential biases, and energy efficiency, helping users and regulators better understand how AI operates.

In my next and final (maybe) article of this series, I’ll discuss these newly enforced requirements, breaking down what they mean for businesses, AI developers, and end-users. Stay tuned! ??