Do You need AI Governance?

As Gen AI is taking the world by storm,? privacy and compliance teams are at frontline to ensure right safeguards are put in place, and at the same time not being a barrier for innovations.

Some noteworthy events.

• NIST introduced AI Risk Management Framework.
• EU Parliament proposed EU AI Act.
• HITRUST has released the latest version of CSF v11.2.0.

Organizations need to establish a Gen AI Governance Framework to build and sustain trust in AI. Framework needs to ..

• Identify and Manage inherent biases in the data being used.
• Secure models, algorithms, data from unauthorized access, data corruption, and any adversarial attacks.
• Understand and document the training methods and decisions criteria; Make them available to the human operator challenge, validation as necessary.

When your end users interact with AI systems (in addition to general data privacy protection),

• Obtain user consent if you are processing their data [Clearly mention how their data will be processed]
• Notify end user about the AI interaction [End user needs to be notified that they are interacting with AI system]
• Provide options for the end user to select the level of participating in the AI interaction.
• Consider implementing governance guardrails for AI ethics validation, data lineage, and evaluate models for bias and fairness.

Open ended nature of Gen AI widens the concerns of verbatim leaks from the training data; Curation of training data excluding private information plays a crucial role. It becomes important to ensure that you have custom models provisioned separately with specific endpoints that you alone have access to, if you have the use case of fine tuning the Foundation Model's (FM) base model.

Make sure that the FMs (provisioned as a managed service) do not make use of the input data for model training purposes; Ensure to turn on data protection controls. As a safe practice, ensure that the input data (along with prompt) you use for receiving inferences from FMs do not contain private information.

As a best practice, ensure that you have a secure and private connectivity to the FMs provisioned (through service providers). If your service provider enables you to fine tune those base models, and you want to fine tune these base models with your datasets, ensure that service provider hosts base model separately first for fine tuning and provides you a secure and dedicated endpoint to access this model.

Governance means being responsible. It translates to (not limited to)

• Avoiding toxic responses. There is subjectivity involved in determining what is toxicity, as it is people-culture dependent. Your governance framework should be able to define a policy on Toxicity.
• Being fair. Ensuring that the training dataset is not skewed with more positives or with more negatives. Such training would bias the outcomes.
• Preventing IP violations. It is a murky area. Responses would depend on the training datasets used for training FMs (Also called LLMs). It is hard to prevent 100%; however, the organization must have a grievance mechanism to hear out objections and handle those objections.
• Preventing Plagiarism and cheating. There are debates happening around the world with respect to using Gen AI. There are two sides arguing with pros and cons; the mechanism around verification presents challenges. Again, an organization must have a grievance mechanism to address this challenge, possibly with human involvement.
• Given the capabilities of Gen AI, organizations may try to power every use case with Gen AI. Organizations must have a mechanism to assess the needs of Gen AI with an eye on cost of operation.

Organizations must invest in creating an AI Governance Team comprising of the functions - Finance, Legal, Engineering, and Operations. CxOs need to push for establishing AI Governance in their organizations, because AI is powerful.?

Great power comes with great responsibility.