Dan's Data Notes: Can a ML Model be stolen?

Overview

When you first read the post's title, you may think about physically stealing a robot or many others. However, in a world where many devices are always on a network, knowing the code running inside is more critical than having the device itself. The software gives life to our smart devices, and algorithms do the same for artificial intelligence systems.

Before we get too far into the future, let's analyze an example of how knowing the system's inner workings can help bypass it. Most countries have toll roads that their citizens often use for faster access to areas paying a premium. When technology evolved and toll booths started charging customers automatically, some reverse-engineered the system's rules to their advantage. Since most of the automatic toll booths flashed at the cars passing by to obtain good-quality images, people soon realized that affecting how the pictures were captured could bypass the established control.

Sure enough, over t developed multiple techniques to make over time their license plates less visible when passing the toll. Similarly, it has been proven possible to steal machine learning algorithms' model configurations and training data that power business applications.

Why steal the model in the first place?

There are at least two big reasons why someone would like to steal a predictive model:

• Sensitive Training data: The value of data has been heralded as strategic for organizations, and sensitive training data used for models is even more valuable. Obtaining a competing model's training data characteristics can allow the attacker to replicate the model.
• Value of the model and its Configurations: Similarly, stealing the model parameters or hyperparameters can prove valuable for cybercriminals or competitors trying to emulate or bypass the predictive capabilities.

Why is it possible?

Machine learning algorithms today are exposed via APIs. Companies can monetize said algorithms and their valuable training data by allowing third parties and external developers to embed these algorithms in their applications. For example, Google, Amazon, and Microsoft expose translation APIs that allow anyone to supplement their existing applications. However, the openness of these APIs and, in some instances, misconfiguration can make them vulnerable to exposing critical model details.

In this paper, the researchers manage to perform attacks that reveal enough details about the models to replicate the model's prediction quality from the API attack. The following prediction API providers were found to be vulnerable:

• BigML
• Amazon Machine Learning

The following model types are susceptible to model extraction attacks:

• Decision trees
• Logistic regressions
• Support Vector Machines
• Deep neural networks

What are the risks of this trend? Can it be mitigated?

Cybercriminals that reverse engineer critical algorithms, such as those used for payments or law enforcement, can pose a big threat to global commerce.

The researchers could reverse-engineer model characteristics and sensitive aspects of the training data.

It's important to secure prediction APIs, and some recommendations include the following:

• Verifying configuration for which the API exposes fields
• Ensure that parameters or data are not exposed if queries are incomplete

You can find additional details on the research paper below:

https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_tramer.pdf