Building trust in AI with Zero-Knowledge Proofs (ZKP)

The proliferation of AI models, particularly those delivered “as a service” over APIs, will increasingly raise questions around how can business or consumer users trust the veracity of the predictions or classifications that these systems deliver.? For example, how can a user be assured that a model has executed faithfully without tampering or error???

As regulators grapple with the implications of different forms of AI, it is also probable that, in the future, regulators across different industries may also be interested in how to ensure trust in the use of artificial intelligence.? Whilst there are, of course, various mechanisms that can be used, it’s interesting to consider whether there can be private sector, decentralised solutions that give the users of AI assurance without the need for heavy regulation or supervision.

A cryptographic concept called a Zero-Knowledge Proof (ZKP) may enable this.? A ZKP is a cryptographic protocol where one party (the “prover”) can prove to another party (the “verifier”) that a statement is true without disclosing any information other than the truth of the statement. ?Invented in 1985 by Goldwater, Micali, and Rackoff, it’s a relatively old concept that has experienced a rapid acceleration in recent years; largely brought about by innovations in the fields of distributed ledger technologies and cryptocurrencies.? ?This video provides a set of explanations of the concept with progressive levels of complexity.

So how can the provider of an AI model give the consumer of that model confidence in its correct application to a particular task??

One method is, of course, for the model provider to provide the consumer with the model weights.? However, given the nature of these models, this would mean that they have effectively given away a large part of the intellectual property embedded in these models and, further, it is probable that not all consumers would have the computing capacity to run these models anyway.?

Therefore, there is a need to invent methods for model assurance that do not require the “leakage” of model weights and do not require large-scale compute capacity which, if held, would also mitigate one of the reasons for using cloud-based models in the first instance.?

One possible method, leveraging zero knowledge proofs, could be as follows:

1. The provider of the AI model creates a hash for all the weights in the model.? i.e. a commitment. They publish this hash in the public domain, such as on a decenteralised ledger, some third party website, or similar.? If a provider has different models, such as a “premium model” and a “standard model”, then each would have different weights and therefore have different hashes that would be subsequently published alongside a description of the model.
2. A user who wants to test or use a particular AI model would then engage with the model provider.?
3. They would send some test data to the model provider for them to apply the model to it.? This could be via an API or some other mechanism, such as email or file transfer.
4. The provider applies the model to the data that was sent and generates the output, as per the normal process.? This, however, is not sufficient for the user to know that the requested model was faithfully applied.?
5. Therefore, the provider would generate what is called a ZK-SNARK proof.? A ZK-SNARK take some private (non-disclosed) inputs and some public (disclosed) inputs and creates a proof that can demonstrate the relationship between the values without revealing the private input. In this example, the private input would be the weights. The public inputs would be the inputs that were provided to the model and the outputs of the model.

1. The provider then sends back the outputs and a ZK-SNARK proof that validates the correct execution of the model. i.e. that given the weights (which are not disclosed) and the hash (which was shared), the model produces this output for the provided inputs.?
2. The user can verify the ZK-SNARK on the test set and, by doing so, be assured that the model executed completely without the model provider needing to provide the model weights.

There have been a number of explorations of this approach, including the open source ZKML project which demonstrated the effectiveness of this approach to the well-known MNIST image classification problem.

A more concrete example of where this might be applied is credit scoring.? Ideally, all people in a given population should be scored based on the same objective characteristics without bias or discrimination.? A credit scoring system could with each classification also provide a zero-knowledge proof that gives assurance that the same model (as represented by the hash) has been used across the population.

This, of course, doesn’t solve the entire “trust problem” with AI (or indeed other forms of third party computation).? It goes some where to improve our ability to ensure authenticity and integrity of a model, it doesn’t necessarily address some of the more emergent challenges when we consider deceternalised inference or training of models (where data may have diverse pedigrees).? This excellent blog post at A16Z discusses, in more detail, some of the other areas where ZK could potentially be applied within the machine learning domain and links to some of the important literature in the field.?