Artificial Neural Networks & Adversarial Attacks

Neural networks consist of thousands and millions of artificial "brain cells" or computational units that behave and learn in an incredibly similar way to the human brain. Also known as Artificial Neural Networks (ANNs), neural networks generally consist of software simulations that behave as though they were millions of brain cells interconnected and working in parallel to solve problems, make decisions and recognize patterns just like a human brain can.

An artificial neural network processes information in two ways; when it is being trained it is in learning mode and when it puts what it has learned into practice it is in operating mode. For neural networks to learn, they must be told when they do something right or wrong. This feedback process is often called back-propagation and allows the network to modify its behavior so that the output is exactly as intended. In other words, it is trained with many learning examples and eventually learns how to reach the correct output every time, even when it is presented with a new range or set of inputs. Just like a human, an artificial neural network can use past experiences to reach the right conclusion.

Artificial Neural networks (ANN) or neural networks are computational algorithms. It intended to simulate the behavior of biological systems composed of “neurons”. ANNs are computational models inspired by an animal's central nervous systems. It is capable of machine learning as well as pattern recognition.

Artificial neural networks are characterized by containing adaptive weights along paths between neurons that can be tuned by a learning algorithm that learns from observed data in order to improve the model. The cost function is what’s used to learn the optimal solution to the problem being solved. This involves determining the best values for all of the tune-able model parameters, with neuron path adaptive weights being the primary target, along with algorithm tuning parameters such as the learning rate. It’s usually done through optimization techniques such as gradient descent or stochastic gradient descent.

These optimization techniques basically try to make the ANN solution be as close as possible to the optimal solution, which when successful means that the ANN is able to solve the intended problem with high performance.

An artificial neural network is modeled using layers of artificial neurons, or computational units able to receive input and apply an activation function along with a threshold to determine if messages are passed along.

In a simple model, the first layer is the input layer, followed by one hidden layer, and lastly by an output layer. Each layer can contain one or more neurons.

Models can become increasingly complex, and with increased abstraction and problem solving capabilities by increasing the number of hidden layers, the number of neurons in any given layer, and/or the number of paths between neurons. Note that an increased chance of over-fitting can also occur with increased model complexity.

Types of Neural Network

Feedforward Neural Network: The feedforward neural network is the most simple of all varieties. Information moves in one direction only and is sent from input nodes directly to output nodes. There are no loops or cycles in this network.

Recurrent Neural Network: Unlike its feedforward cousin, the recurrent neural network allows data to flow bidirectionally. This type of network is a popular choice for pattern recognition applications, such as speech recognition and handwriting solutions.

Modular Neural Network: A modular neural network is made up of independent neural networks. Each is given a set of inputs and work together to complete sub-tasks. The final output of the modular neural network is managed by an intermediary that collects data from the individual networks.

Convolutional Neural Network: Convolutional neural networks are primarily used to classify images. For example, they are able to cluster similar photos and identify specific objects within a scene, including faces, street signs and individuals. 

Neural networks are becoming more and more common and they are often implemented without much consideration of their potential security flaws. 

Neural networks are used very commonly by most of us. Asking Siri a question, Self-Driving Cars, Face recognition camera, Face identification within social media applications, Alexa etc. all are neural networks. These are just the applications that are tangible, there are plenty of intangible applications of neural networks that people use every day. Whether it is the software program you use at work or just searching for a place to go to dinner that evening, you are likely using some form of neural network

Neural networks can be rather sensitive to the inputs that you give them. Therefore, it is relatively easy to fool with the network if you know the right buttons to push. By manipulating certain nodes of images it is easy to activate neurons associated with particular features, which can make the network give spurious outputs.

Let’s say that you use Alexa to buy things on a regular basis. One day, a particularly smart hacker sits outside of your house and hacks your Wifi (say using aircrack-ng) if you have not secured your router properly or still have a default password. The hacker now has access to Alexa, which has security privileges to make transactions on your behalf, giving your verbal approval. If the hacker is smart enough, it is conceivably possible that they could fool Alexa into giving away all of your money to the hacker, just by pushing the right buttons on the neural network.

Adversarial Attacks

The typical purpose of an adversarial attack is to add a natural noise on an image so that the target model misclassifies the sample, but it is still correctly classified by the human eye. 

Non-targeted adversarial attacks are developed to fool a machine learning classifier by modifying source image. The neural network does not return a certain class as opposed to targeted attacks. The output can be a random class excluding the original one.

Targeted adversarial attacks are designed to misclassify an image as a specified target class by modifying source image. The output of this neural network is only one certain class. Impersonation can be an example for this type of attacks because an adversarial image can disguise a face as an admin user

Defenses against adversarial attacks are aimed to build such a robust classifier so that it correctly identifies adversarial images.

Essentially, attacks on neural networks involve the introduction of strategically placed noise designed to fool the network by falsely stimulating activation potentials that are important to produce certain outcomes. By ‘strategically place noise’, consider the following image developed by Google Brain to show how the same effect can fool humans. Is the picture on the left and the right are both cats? Or dogs? Or one of each?

An illustration of how the network is corrupted by the introduction of the strategic noise.

White Box & Black Box Attacks

A white box attack occurs when someone has access to the underlying network. As a result, they will know the architecture of the network. This is analogous to a white box penetration test of a company’s IT network. Once a hacker understands how your IT network is structured, it makes it much easier to sabotage. Knowing the structure of the network can help you select the most damaging attacks to perform, and also helps to unveil weaknesses relevant to the network structure.

A black box attack occurs when the attacker knows nothing about the underlying network. In the sense of neural networks, the architecture can be considered as a black box. 

Presuming that we are able to test as many samples as we like on the network, we can develop an inferred network by passing a bunch of training samples into the network and obtaining the output. We can then use these labeled training samples as our training data and train a new model to obtain the same output as the original model.

Two classifications:

1) The adversary has access to the training environment and knowledge of the training algorithm and hyperparameters. It knows the neural network architecture of the target policy network, but not its random initialization. They refer to this model as transfer-ability across policies.

2) The adversary additionally has no knowledge of the training algorithm or hyperparameters. They refer to this model as transfer-ability across algorithms.

Once we have our new network, we can develop adversarial examples for our inferred network and then use these to perform adversarial attacks on the original model.

This model does not depend on knowing the architecture of the network, although this would make it easier to perform the attack.

Clearly, this presents a potential problem for the mass adoption of self-driving cars. Nobody would want their car to ignore a stop sign and continue driving into another car, or a building, or a person. 

Evasion and Poison Attacks

Attacks that involve in ‘fooling’ a system are Evasion attacks. 

An example would be fooling a spam detector that guards email accounts so that you are able to get your spam emails into someone’s inbox. Spam detectors often use some form of machine learning model that can be used for word filtering. If an email contains too many ‘buzzwords’ that are typically associated with spam email (given your email history as the training data), then it will be classified as spam. Now if I know these words I can deliberately change them to make it less likely that the detector will consider my email as spam, and I will be able to fool the system.

Another good example is in computer security, where machine learning algorithms are often implemented in intrusion detection systems (IDSs) or intrusion prevention systems (IPSs). When a network packet reaches my computer that has the characteristic signature of a piece of malware, the algorithm kicks in and stops the packet before it can do anything malicious. However, a hacker can use misleading codes to ‘confuse’ the network so that it does not flag up a problem. 

Poisoning attacks involve compromising the learning process of an algorithm but only works on models that participate in online learning, i.e. they learn on the job and retrain themselves as new data become available to them. 

Considering the above IDS example, these are constantly updated using online learning since new viruses are always being developed. If one wishes to prevent a zero-day attack, it is necessary to give these systems the capability of online learning through an integrated online big data pool that utilizes data analytics. In a Poisoning attack, the attacker could poison the training data by injection designed samples to eventually compromise the whole learning process. This makes the IDS useless, and you are at much greater risk from potential viruses and likely will not even realize. Poisoning may thus be regarded as adversarial contamination of the training data. Similar could be thought of for the spam detector example. 

Fast Gradient Step Method. This manipulates the sharp decision boundaries used by the classifier by the introduction of strategic noise, as we have been discussing up to now.

Some methods to defend

There are a number of methods that have been developed to defend neural networks from the various types of attack vectors.

Adversarial Training

The best way of defending against adversarial attacks is through adversarial training. That is, actively generating adversarial examples, adjust their labels, and add them to the training set and then train the new network on this updated training set and it will help to make your network more robust to adversarial examples.

Smooth decision boundaries

Regularization, acts to smoothen the decision boundaries between classes and makes it less easy to manipulate network classification using strategic noise injection.

Mixup

Mixup involves mixing two training examples by some factor λ, which is between zero and one, and assigning non-integer classification values to these training samples. This acts to augment the training set and reduces the optimistic classification tendencies for networks. It essentially diffuses and smoothens the boundaries between classes and reduces the reliance of classification on a small number of neuron activation potentials.

CleverHans

The essential story is of a horse who supposedly was able to do basic arithmetic by stamping his feet a given number of times. However, it was later discovered that the horse was actually cheating and responding to the verbal and visual clues of the surrounding crowd.

Link to GitHub:

From that name, CleverHans is a Python library that has been developed to benchmark machine learning systems’ vulnerability to adversarial examples. If you are developing a neural network and want to see how robust it is, test it out with CleverHans and you will find get an idea of its level of vulnerability. 

Penetration Testing

As with any form of cybersecurity, you can always pay someone to hack you and see how much damage they do. However, you can make them sign a document which specifies limits on what the attacker is allowed to do. This gives you an idea of your level of vulnerability to an actual cyber attack. 

---------------------------------------------------------------------------------------------------------------