Newsletter #11: AI Act and Transparency Requirements

Ensuring Explainable and Accountable AI Systems

Author Henrik Engel

Introduction

Transparency is a cornerstone of the AI Act, aimed at regulating the use of artificial intelligence and ensuring it is both accountable and understandable. But what does transparency mean in practice, and how can we ensure AI systems meet these requirements?

This article explores the specific transparency requirements in the AI Act and their implications for organizations and users.

What does the AI Act require regarding transparency?

The AI Act establishes clear requirements to ensure that users and stakeholders have access to the necessary information about how AI systems work. This includes:

• Identification of AI systems: Organizations must clearly inform users when they are interacting with an AI system, such as via chatbots or voice assistants. Users should know that they are not interacting with a human.
• Documentation and explainability: Manufacturers must provide technical documentation describing how the system works, and what data and algorithms are used. The documentation must be verifiable by supervisory authorities.
• Transparency in decision-making processes: For AI systems that affect individual rights or high-risk decisions, it must be possible to explain how the system reached its conclusion.

How do we ensure that AI systems are explainable and understandable?

Explainability is about making complex AI models understandable to humans, which can often be challenging. Here are some methods to ensure explainable AI systems:

• Development of simple models: Where possible, organizations should use models that are easier to explain, such as linear regressions or decision trees, instead of complex neural networks.
• Use of XAI (Explainable AI): Tools like SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) can help explain which factors contributed to a given decision in the system.
• User-oriented explanations: The information should be presented in a language and format that is understandable to the user, regardless of technical background. For example, using visualizations and simple descriptions.
• Continuous evaluation and updating: Explainability should be evaluated and improved over time based on user feedback and new technologies.

The relationship between transparency and data protection

Transparency and data protection are closely linked, as both are about protecting users' rights. Some key aspects of this connection are:

• Informed decisions: Transparency requirements ensure that users understand how their data is used, which strengthens their ability to give informed consent.
• Risk assessment and impact analyses: Organizations must conduct impact assessments (DPIAs) to ensure that AI systems' use of data is in accordance with GDPR. Transparency requirements can help identify potential risks.
• Right of access: GDPR's Article 15 gives users the right to access how their data is processed. The AI Act's transparency requirements support this by making it easier to explain how AI systems use data.
• Compliance with ISO standards: ISO 27701, which focuses on privacy management, and ISO 42001, which deals with AI governance, offer frameworks for achieving both transparency and data protection. These standards help organizations document and comply with the requirements.

What do the transparency requirements demand of the organization?

The transparency requirements in the AI Act place significant demands on organizations that develop, implement, or use AI systems. Specific organizational obligations include:

• Governance structures: Organizations should establish clear governance processes to ensure responsible use of AI systems.
• Risk management: Continuous risk assessments must be performed to identify and mitigate potential problems related to transparency and explainability.
• Internal training: Organizations must train employees to understand and implement the transparency requirements in the AI Act and comply with the AI Act's requirements for general AI skills.
• Communication: Internal and external communication strategies should be developed to explain the functionality and limitations of AI systems.
• Compliance reports: Organizations must be able to document how they comply with the transparency requirements and ensure that this documentation is available to supervisory authorities.

Conclusion

The transparency requirements in the AI Act are essential to build trust in AI systems and ensure they are used responsibly. By combining technological solutions with clear documentation processes, organizations can meet the requirements while protecting users' rights.

Questions for the reader: How does your organization work to make AI systems more transparent and explainable? Do you have challenges balancing transparency and data protection? Feel free to share your experiences in the comments section!

Sources and inspiration:

• AI Act official documentation
• ISO 27701: Privacy management
• ISO 42001: AI Governance
• Tools for explainable AI: SHAP, LIME.