ERES Regulations and AI Systems

After my last article on AI risk management framework, a couple of friends asked me to delve a little deeper into the how part of this discussion. They asked - 'how do we do what you are trying to say, especially regarding good data attributes'? While there is no direct answer since it will depend on what you are trying to do, where you are trying to do it, when you are trying to do it, etc., I thought of penning my thoughts on a few ways (how) to achieve the good data attributes - key to any GxP compliant system.

Electronic Records and Electronic Signatures (ERES) regulations, such as 21 CFR Part 11 and EU Annex 11, set stringent requirements for ensuring the accuracy, reliability, consistency, ability to discern invalid data, confidentiality, integrity, and availability of electronic records and systems. When integrating AI systems within regulated environments, it is crucial to implement robust controls (some organizations are calling them guardrails) that meet these regulatory expectations. I have outlined a few controls and best practices and included an example to help understand how AI systems can adhere to ERES requirements through them. Do note that this is not all-inclusive, is generic and you should base your decision on the intended purpose. The human-in-loop factor should be considered wherever the risk crosses the tolerance limit.

Key requirements and their controls

Accuracy

NIST defines accuracy as the degree of conformity of a measured or calculated value to the true value, typically based on a global reference system.

Data Quality Control:

• Data Validation: Implement rigorous data validation checks, including range checks, format checks, and consistency checks, to ensure the accuracy of input data.
• Combination Review:?Employ a combination of automated and manual data validation processes to ensure the highest level of accuracy. Manual review by domain experts is crucial for validating complex data scenarios that AI might not handle well.

Model Validation:

• Performance Metrics: Regularly assess AI model performance using metrics like accuracy, precision, recall, and F1 score, and establish threshold values for acceptable performance.
• Validation Protocols: Conduct formal validation studies to ensure AI models perform as intended across different scenarios and datasets.
• Independent Validation:?Engage external validation teams to assess AI model performance. This avoids potential biases from internal teams.

Periodic Review:

• Scheduled Reviews: Conduct periodic reviews of AI system performance, comparing system outputs with known accurate results and threshold values.
• Peer Review: Have AI models reviewed by SMEs to validate their accuracy and relevance.

Algorithmic Transparency:

• Explainable AI (XAI): Implement techniques to provide transparency into AI model decisions.
• Documentation: Maintain detailed documentation of AI algorithms, including their design, training processes, and assumptions.

Reliability

NIST defines reliability as the ability of a system or component to function under stated conditions for a specified period of time.

System Reliability:

• Redundancy: Implement redundant systems and failover mechanisms for continuous operation during failures.
• Regular Maintenance: Schedule regular maintenance and updates for both hardware and software components.

Model Monitoring:

• Drift Detection: Monitor AI models for performance degradation or concept drift, using statistical methods to detect significant deviations.
• Retraining Protocols: Establish protocols for periodic retraining of AI models with new data.

Robust Testing:

• Stress Testing: Perform stress testing under extreme conditions to ensure reliability.
• User Acceptance Testing (UAT): Involve end-users in testing to ensure the system meets real-world reliability expectations.

Incident Management:

• Incident Response Plan: Develop and regularly update an incident response plan for quick mitigation of system failures.
• Root Cause Analysis: Conduct thorough analyses for any incidents to prevent recurrence.

?

Consistency with Intended Performance

Requirement Specifications:

• Clear Documentation: Maintain required documentation of system requirements, including performance specifications and intended use cases.
• Traceability Matrix: Use a traceability matrix to map system requirements to specific functionalities and tests.

Change Control:

• Controlled Environment: Implement change control procedures to manage and document changes to AI models and systems.
• Impact Assessment: Conduct impact assessments for proposed changes to evaluate their effect on performance and quality.

Continuous Improvement:

• Performance Monitoring: Continuously monitor system performance using dashboards for real-time metrics.
• Feedback Loops: Implement feedback loops to gather user input for iterative improvements.

Quality Audits:

• Regular Audits: Conduct regular audits to ensure alignment with regulatory and quality requirements, and intended performance.
• Third-Party Reviews: Engage third-party auditors for independent assessments.

?

Confidentiality

NIST defines confidentiality as preserving authorized restrictions on information access and disclosure, including means for protecting personal privacy and proprietary information.

Access Controls:

• Role-Based Access: Implement role-based access controls to restrict access to sensitive data.
• Multi-Factor Authentication: Use multi-factor authentication for enhanced security.

Data Encryption:

• Encryption at Rest and in Transit: Ensure all sensitive data is encrypted both at rest and during transmission.
• Key Management: Implement robust key management practices for secure generation, storage, and management of encryption keys.

Privacy Enhancements:

• Data Anonymization: Anonymize personal data to protect individual privacy while using data for AI training.
• Data Minimization: Collect and process only the minimum amount of data necessary.

Security Policies:

• Information Security Policy: Develop and enforce a comprehensive information security policy covering data handling, access controls, and incident response.
• Employee Training: Regularly train employees on confidentiality and data protection practices.

?

Integrity

As per NIST, the term 'integrity' means guarding against improper information modification or destruction, and includes ensuring information non-repudiation and authenticity.

Data Integrity:

• Audit Trails: Maintain comprehensive audit trails that log all data creation, modification, and deletion activities.
• Checksums and Hashing: Use checksums and hashing algorithms to verify data integrity.

System Integrity:

• Intrusion Detection: Deploy intrusion detection systems to monitor for unauthorized access or anomalies.
• Patch Management: Regularly apply security patches and updates to all system components.

Data Governance:

• Data Stewardship: Assign data stewards to oversee data integrity and compliance with governance policies.
• Data Quality Metrics: Establish and track data quality metrics for continuous improvement.

System Integrity Checks:

• Regular Scans: Perform regular integrity scans of system files and configurations.
• Tamper-Evident Logs: Ensure audit logs are tamper-evident.

?

Availability

As per NIST, availability means ensuring timely and reliable access to and use of information.

Disaster Recovery:

• Backup Strategies: Implement regular data backup procedures with secure offsite storage.
• Disaster Recovery Plan: Develop and test a disaster recovery plan for quick recovery from disruptions.

System Scalability:

• Scalable Architecture: Design AI systems to handle increased workloads without compromising performance.
• Load Balancing: Use load balancing to distribute workloads evenly.

Capacity Planning:

• Predictive Analytics: Use predictive analytics to forecast system usage and plan capacity.
• Resource Allocation: Dynamically allocate resources based on real-time demand.

Business Continuity Planning:

• BCP Testing: Regularly test the business continuity plan (BCP).
• Critical Systems Identification: Prioritize critical systems and processes for recovery.

?

Ability to Discern Invalid Data

Data Validation Rules:

• Validation Algorithms: Incorporate algorithms to automatically identify and flag invalid or suspicious data entries.
• Error Handling: Establish robust error-handling procedures.

User Training:

• Awareness Programs: Conduct training sessions on identifying and reporting invalid data.
• Feedback Mechanism: Implement mechanisms for users to report data issues.

Advanced Validation Techniques:

• AI-Based Validation with Human-in-Loop: Ensure a human-in-the-loop approach where domain experts validate the output of AI-based validations
• Cross-Validation: Validate data against multiple sources for consistency.

User Interfaces:

• Intuitive Interfaces: Design interfaces for easy identification and correction of invalid data.
• Real-Time Feedback: Provide real-time feedback on data validity.

Let us take an example to understand this better.

A pharmaceutical company implements an AI-based system to manage clinical trial data and ensure compliance with 21 CFR Part 11 and EU Annex 11. The AI system handles data from multiple sources, including electronic health records, lab results, and patient-reported outcomes.

Let us look at how this system can meet the above requirements.?

Accuracy:

• Data Validation: Automated validation rules check for data consistency and accuracy, flagging discrepancies for review
• Model Validation: AI models are validated through performance tests and reviewed by clinical data experts and external validation teams
• Periodic Reviews: Periodic Reviews to compare system outputs with accurate results
• Explainable AI: Implemented to provide transparency in AI model decisions

Reliability:

• Redundancy: Deployed on a cloud platform with built-in redundancy
• Incident Management: An incident response plan addresses system failures or data breaches
• Stress Testing: Performed under extreme conditions to ensure reliability
• Root Cause Analysis: Conducted for any incidents to prevent recurrence

Consistency with Intended Performance:

• Change Control: Procedures manage and document changes to AI models and systems
• Continuous Monitoring: Performance dashboards monitor system metrics in real-time
• Regular Quality Audits: Ensured alignment with regulatory requirements
• Third-Party Reviews: Independent assessments provided

Confidentiality:

• Access Controls: Role-based access and multi-factor authentication enforced
• Data Encryption: All data encrypted both at rest and during transmission
• Data Anonymization: Personal data anonymized for AI training
• Information Security Policy: Comprehensive policy covering data handling

Integrity:

• Audit Trails: Comprehensive trails log all data activities
• Checksums and Hashing: Verify data integrity
• Intrusion Detection: Systems monitor for unauthorized access
• Data Governance: Data stewards oversee integrity and compliance

Availability:

• Backup and Recovery: Regular backups are performed with offsite storage
• Disaster Recovery Plan: Developed and tested for quick recovery
• Scalable Architecture: Designed to handle increased workloads
• Predictive Analytics: Used to forecast system usage

Ability to Discern Invalid Data:

• Validation Algorithms: Automatically identify and flag invalid data
• User Training: Staff trained on identifying and reporting invalid data
• AI-Based Validation with Human-in-Loop: Domain experts verify AI-based validations.?

Conclusion

?Implementing AI systems in compliance with ERES regulations requires a holistic approach that addresses all aspects of data integrity, system reliability, security, and availability. By incorporating these controls and best practices, organizations can assure their AI systems not only meet regulatory requirements but also enhance the overall quality and trustworthiness of their electronic records and processes. Regular audits, continuous monitoring, and proactive risk management are essential to maintaining compliance and ensuring that AI systems operate effectively and securely.

References

1. 21 CFR Part 11
2. EU Annex 11
3. GAMP 5
4. General Principles of Software Validation (US FDA)
5. NIST SP 800-53
6. ISO/IEC 27001
7. ICH E6(R2)
8. IEEE 1012-2016
9. Explainable AI (XAI)
10. Privacy Impact Assessment (PIA) Guidance

Disclaimer: The article is the author's point of view on the subject based on his understanding and interpretation of the regulations and their application. Do note that AI has been leveraged for the article's first draft to build an initial story covering the points provided by the author. Post that, the author has reviewed, updated, and appended to ensure accuracy and completeness to the best of his ability. Please use this after reviewing it for the intended purpose. It is free for use by anyone till the author is credited for the piece of work.