Build Trust with a Transparent ML Supply Chain

In an era where artificial intelligence (AI) reshapes industries at an unprecedented pace, trust remains the paramount challenge and necessity for widespread adoption, which is particularly crucial in a world where AI integration is pervasive and impactful across various sectors. But trust in AI systems isn't just about technical robustness—it's about integrating ethical, transparent, and responsible practices into the very fabric of AI development and deployment. Leveraging the power of open source is at the core of Red Hat strategy to enhance trust in AI applications. This is why we emphasise transparency throughout the ML supply chain, ensuring that every phase of AI development—from data collection to model deployment—is open and accountable. This approach not only mitigates risks but also enhances the reliability and ethical stature of AI solutions.

Dimensions of Trust in an AI System

Recently I have been reflecting on the six dimensions of trust in AI systems as detailed in this great article from Charles Vardeman, who works in the Laboratory for Assured AI Applications Development (LA3D). It makes a lot of sense to assess our strategy to build trust in AI across these dimensions which are not just theoretical ideals but practical realities. These dimensions include:

1. Explainability: Ensuring that AI behaviours and decisions are understandable by humans.
2. Safety and Robustness: Focusing on the robustness of AI systems under varying conditions.
3. Non-discrimination and Fairness: Actively preventing AI systems from perpetuating biases.
4. Privacy and Data Governance: Safeguarding personal data against unauthorised access and ensuring the right to privacy.
5. Sustainability: Assessing and managing the impact of running AI on computing resources, including power and carbon usage and optimisation.
6. Accountability: Establishing clear accountability for AI actions and a mechanism for correcting mistakes.

By embedding these trust dimensions into our platforms, we help organizations deploy AI solutions that are not only powerful and efficient but also aligned with ethical standards and societal values.

The Importance of Open Source in Trustworthy AI

Open source is more than a development model; it's a commitment to collaborative innovation and transparency that builds inherent trust. By developing AI in an open source environment, we ensure that every aspect of AI—from the algorithms used to the data processed—is accessible, auditable, and amendable by a community that spans industries, sectors, and geographies.

This approach aligns with the six dimensions of trust as outlined above. Here’s how Red Hat's open source approach directly impacts each dimension:

1. Explainability: Tools such as TrustyAI, an integral part of Red Hat’s open source AI strategy, enhance the transparency and explainability of AI systems by providing clear insights into how algorithms make decisions, ensuring these processes are transparent and comprehensible. The way this works is TrustyAI allows users to visualise and manipulate input variables to see how they affect outcomes, thereby identifying and correcting biases. The work done by the TrustyAI community therefore supports our commitment to building trustworthy AI by ensuring all processes are understandable and auditable, aligning with ethical standards and regulatory requirements.
2. Safety and Robustness: The community-driven approach in open source development allows for extensive testing and validation across varied environments and use cases, enhancing the reliability and safety of AI systems. InstructLab, developed under the open-source paradigm, can significantly contribute to process safety in AI model development. By utilising the Large-scale Alignment for chatBots (LAB) technique, which leverages taxonomy-guided synthetic data generation for model tuning, InstructLab reduces dependency on costly human annotations and proprietary models, thus enhancing model accessibility and reliability. This approach ensures that AI models are more adaptable and aligned with specific user requirements, bolstering their safety and robustness. Furthermore, the open-source nature of InstructLab allows for broad community participation in model development and improvement, ensuring continuous enhancements in model performance and security. The transparency in training data and processes provided by InstructLab reinforces the safety standards essential for trusted AI implementations.
3. Non-discrimination and Fairness: With the collaborative nature of open source, a diverse set of contributors can examine and improve algorithms, helping to identify and mitigate biases more effectively than in closed, homogeneous environments. This is the kind of work that Red Hat engagement in MLCommons supports, by providing a rigorous framework for assessing the fairness of AI systems. This collaboration emphasises developing AI Safety Benchmarks that not only test the safety of AI applications but also focus on minimising biases that could lead to discriminatory outcomes. The benchmark's structured testing environment helps identify potential hazards in AI responses, ensuring systems are evaluated under diverse scenarios reflective of real-world applications. As a result, this open source initiative contributes to setting industry standards for fairness, helping to mitigate risks of bias and ensure AI systems are trustworthy and equitable across different user groups.
4. Privacy and Data Governance: Trust in AI-decision making also relies heavily on the ability to trust and verify the data by which models are trained or subsequently tuned. In a public sector webinar last year on "Building Stakeholder Trust in Government, Data & AI", I spoke about the importance of data provenance, in particular the need to ensure reliability and trustworthiness in the data used for AI to prevent biases and errors. I also shared some of the work I have been contributing to in terms of enabling Federated Data Governance, effectively helping enterprises top implement universal controls for data management, crucial for maintaining security and privacy across expanding digital landscapes. If you are interested in the space of data governanvce, I have some exciting news that I will share this week as part of my presentation at the Open Source in Finance Forum 2024 in London, covering "OS-Climate: A Data-Driven Open Source Approach to Climate-Aligned Finance Investing".
5. Accountability: When AI systems are developed in an open source context, it's easier to track decisions back to their algorithmic origins, facilitating accountability. Moreover, community-driven updates allow for rapid redress and rectification of issues. This is why i am thrilled by recent announcements such as our partnership with Stability AI, making it easier for our customers to easily access and integrate cutting-edge open source LLMs.
6. Sustainability: The impact of AI on sustainability is profound, offering both challenges and solutions. On one hand, the computational demands of training and running AI models can be substantial, leading to high energy consumption. On the other hand, AI can optimise resource use and energy efficiency across various sectors, significantly reducing waste and enhancing sustainability efforts. In this space, cloud-native architectures play a crucial role in measuring and also managing AI sustainability by enabling more efficient scaling, distribution, and management of AI workloads across multiple environments. Recently with our IBM Research colleagues under the sponsorship of Tamar Eilam , and with the support of Sanjay Podder , we started to engage with the Green Software Foundation as part of a newly formed Green AI Committee with the intent to channel and synergise a number of existing cloud-native sustainability efforts and contributions towards a more comprehensive Sustainable AI framework with broader support and adoption by the larger community.

Looking Forward

The journey towards trustworthy AI is complex, and while with the above initiatives, Red Hat has the potential to lead the way in ensuring that AI technologies advance in a manner that is beneficial, ethical, and sustainable, much work need to be done still. This is because trust in AI remains a challenging topic in itself, primarily due to the absence of a universally accepted framework for evaluating or benchmarking model trust across the industry. While various organisations and entities (including some of which Red Hat is participating in) have proposed guidelines and standards, the lack of a cohesive, widely-adopted approach means that trust benchmarks can vary significantly, leading to inconsistencies in how trustworthiness is assessed. This fragmentation hampers the ability to comprehensively compare AI systems on an equal footing, complicating efforts to ensure that AI behaves reliably and ethically in diverse scenarios. As a consequence, without a standard metric for trust, organisations must rely on a patchwork of methodologies, making it difficult to achieve transparency and accountability universally.

As AI technology continues to advance and permeate more aspects of daily life, establishing a clear and agreed-upon standard for trust evaluation will be crucial for fostering widespread confidence in AI solutions. I trust and hope that Red Hat, by continuously fostering a transparent AI ecosystem through an open source approach and contribution, is helping to not just optimise processes but also building a foundation of trust that will underpin the next generation of AI innovations.