Selling with Data #48 - Enterprise AI needs enterprise governance
Most of the customer meetings I have had about AI eventually turn to AI governance. At first, I thought of AI governance as model data governance (i.e., how do companies use their own data and ensure that the model isn't hallucinating). Over time, my appreciation of governance has expanded to also include enterprise scale governance, model governance selection, post production model governance and organizational design governance.
Companies want to dive into AI, but realize that unlike consumer AI, enterprise AI needs a higher standard of controls, because for companies the stakes tend to be much higher.
A challenge with governance is that it means different things to different people. For the data scientist and data analyst, governance might mean frameworks, rules and control. For privacy and line of business leaders, governance might mean transparency, regulated controls, avoiding embarrassing situations and enterprise scale.
While companies work through their own AI governance, last week the European Parliament adopted a draft of the AI Act as the world’s first comprehensive regulation for AI governance (source), that serves as a first lens to score current popular foundation models and to help evaluate if governance is needed.
One of the best articles I read to unpack the significance and details of the AI Act is Stanford University's Center for Research on Foundation Models report, Do Foundation Model Providers Comply with the EU AI Act?
The article says: "Foundation model providers rarely disclose adequate information regarding the data, compute, and deployment of their models as well as the key characteristics of the models themselves. In particular, foundation model providers generally do not comply with draft requirements to describe the use of copyrighted training data, the hardware used and emissions produced in training, and how they evaluate and test models."
To go further into the findings, let's review each of the four categories and how they scored.
Data (48 out of a possible 120 - AI Governance Score: 40%)
Data is the source of the information that foundation models, especially large language models, use to train the parameters of the models. Data informs the model and its recommendations. Data includes what data was used to train the foundation model, controls on suitability, bias, and appropriate mitigation, and a catalog of copyrighted data used to train the model. Poor visibility into data lineage, data quality, and data bias presents a governance concern, especially for generative AI.
Compute (33 out of a possible 80 - AI Governance Score: 41%)
Compute is the amount of energy used to train the model. Similar to crypto currency where there were concerns about high energy cost to mine the crypto, the same energy and sustainability concerns exist for AI models. Compute includes the cost and effort to train the model and the energy consumption required. Excessive compute not only results in high operational costs, but also presents a sustainability challenge for companies.
Model (68 out of a possible 160 - AI Governance Score: 42%)
The third category is the model itself. This covers a description of the capabilities and limitations of the model and available public / industry benchmarketing on how the model performs. Without model governance, many models operate as a "black box" with little internal visibility or insufficient attribution or traceability for the result set on the inputs and data sources.
Deployment (54 out of a possible 120 - AI Governance Score: 45%)
The final category is the deployment of the model, after it has been trained and put into production usage. This includes the disclosure if the content is created from a human or a machine, information about the creators of the models, and sufficient technical documentation for compliance and governance. Deployment governance ensures that a model that operates correctly on the first day, continues to operate accurately and without bias or drift. This is especially critical if a model is informing high yielding business decisions and do not have the proper oversight and governance.
领英推荐
Report card of top Foundation Models compliance with the Draft EU AI Act
The image below considers how several popular model scores against the draft EU AI Act criteria.
Top Scoring Models: Bloom, GPT-NeoX, PaLM 2
Bottom Scoring Models: Luminous, Claude, Jurassic-2
Update - Who is responsible for AI Governance?
As soon as I clicked publish on this article, Time released an article "Exclusive: OpenAI Lobbied the E.U. to Water Down AI Regulation". I thought this was relevant to this article and worth an update. The article says that while OpenAI CEO Sam Altman has been very loud about the need for AI regulation during numerous interviews, events, and even while sitting before U.S. Congress. According to OpenAI documents used for the company's lobbying efforts in the EU, OpenAI wants regulations that heavily favor the company and have worked to weaken proposed AI regulation.?OpenAI's position is that the regulatory focus should be on the companies using language models, such as the apps that utilize OpenAI's API, not the companies training and providing the models. (source).
This creates an interested question - who is responsible for AI Governance? Is it the organizations that are creating the models or the companies that use the models?
Summary
The AI Act illustrates significant shortcomings in the governance of popular models, as defined by the criteria established by the European Union. Currently, popular foundation models generally fail to disclose adequate information regarding their models' data, compute requirements, model benchmarking, and post production deployment accuracy. There are also important questions about the legal interpretation of copyrighted data that are now being reviewed. As companies move to incorporate AI into their operations, the standards that were acceptable for consumer workloads require a higher quality and governance hurdle. Enterprise governance is gaining focus because companies have to operate at a higher threshold. It isn't just about models being trained on a customer’s data. Enterprise AI requires a full enterprise governance platform.
Share your comments if you agree or have any examples.
Good selling.
Very interesting subject - thanks for the info
Vice President, Data and AI Worldwide Technical Sales at IBM
1 年Whether required by statute or not, I believe the burden of AI governance falls squarely on the users of the model. It will most likely be those enterprises that bear the brunt of any gaffes related to inappropriate use or results from AI initiatives. An enterprise has to "own" their own actions and decisions. Of course, that ownership requires a high standard of trust from important vendors that may be providing the LLM. In order to gain that level of trust, the model providers will need to embark on a certification/validation program in conjunction with any legislative requirements. Tooling that addresses that compliance and transparency will become an invaluable addition to every AI program.
Cloud Sales Leader
1 年I believe companies also need a way to ensure their employees are using the right LLMs for the right use case or business problem. The issue with a black box like chat GPT is the users get used to one LLM and the reality is many enterprises will leverage different models for different use cases to increase the accuracy, decrease compute cost, and lower latency. Organizations need to understand the data set the model was trained on and what the model is good for. Watson Orchestrate is the perfect way to provide governance on what LLM is used for a certain use case or “skill”. IT can set guidelines on what can be done with different LLMs to ensure accuracy in the models used.