Neural network weaknesses: Background

Brief talk at the TrendMicro Philippines Decode Conference 2018 generated a new wave of buzz and worries over the compromising results of expert analysis conclusions made by neural networks – systems of artificial results inference based on transferred experience.

The root of the concern lies in the structure and operating principles of neural networks, which in the aggregate prove the possibility of result misrepresentation or complete network logic substitution. In this way, a malefactor can make the network “think” in his or her favor or work completely different from what was intended. For example, a neural network can be set to work as a cyber attack booster, to replace marketing content recommendations or to corrupt clinical decisions based on laboratory tests and treatment options. A compromised neural network can interfere with operations of self-moving systems, cars, and other mission-critical applications.

In addition to the well-known fact of the neural network immaturity problem (Adversarial attacks), the possibility for a deep learning neural network logic (results inference) being poisoned (Data poisoning) through gradual false data loading (substitution) came to light.

This state of affairs can make a neural network unworkable and even dangerous for the people it involves.

Deep learning in a nutshell: What makes neural networks unique and debug-complicated?

In a nutshell, a neural network is based on a very simple component – a neuron receives a set of input data and returns a value. A neuron receives a weighted sum of input data (plus a numerical bias) and supplies it into a nonlinear activation function. Then, this function returns a value, which can be used as one of the inputs for another neuron (see the figure below).

Depending on the type of assigned task, neuron input-output combinations have many possible modifications and variations – they are called Azimov’s Neural network zoo. Most neural network arrangements are complex and have unstable behavior based on the gained experience.

Deep learning is a machine learning subset: a field of artificial intelligence (AI), where software creates its own logic while exploring and comparing large data sets in an infinite number of iterations and amplifying a weighted sum of input data and biases with each iteration.

This is an automated and self-organizing process aimed at gradual result refinement based on input data. This process looks nothing like traditional algorithm coding. Neural network operational principles determine the relationship among two groups of core software parameters: “accuracy, effectiveness” and “control, transparency”. This relationship is called “black box”, and the deviation is defined by the “golden rule of mechanics”.

Evidently, a neural network can be very good at fulfilling specific tasks, but it is difficult to comprehend billions of neurons and parameters, which factor into decisions the network makes. They are defined by the combination of deep learning algorithm types and the very model of the network itself. There are two characteristics of neural networks essential for cybersecurity:

• The neural network has an abstract function identical to weighing, which is very dependent on the logical meaning of input data. You cannot qualitatively evaluate this data, and it is impossible to say that the neural network is good or bad.
• The neural network operational algorithm is unstable and opaque. All too often, even its creator cannot foresee how the network will be functioning at the end of the learning period.

Neural Network – negative synergy: From unique and bright characteristics of deep learning algorithms to cyber attacks

Warming up to the topic of neural network models and their specific features, we can identify two main weaknesses, which can cause damage to those involved in all fields of the neural network application, from fintech to education.

• Adversarial attacks. This method of attack is based on the possibility that an inexperienced neural network may make mistakes while drawing inconsistent or sometimes even insane conclusions. One recorded case is of city police in Great Britain using a neural network for detection and marking of child abuse images, but it mistook a picture of sand dunes for a naked woman’s breasts. In another case, MIT students made some alterations to a toy turtle and a neural network classified it as a rifle.
• Data poisoning attack. This special type of attack is aimed at creating problem behavior (inference) in deep learning algorithms through overdependence on data. Deep learning algorithms have no notion of morality, common sense, and discrimination that human intelligence has. They reveal only hidden deviations and data trends they learn. One spectacular example, which revealed a possibility of this type of attack in principle, occured on Twitter in 2016. Users launched an AI-powered chat-bot created by Microsoft, and exposure to racist hate speech just over 24 hours turned the chat-bot into a Nazi follower and Holocaust denier, excessively posting abusive comments.

Irrespective of attack type, a neural network can threaten cybersecurity if it gets into the hands of the wrong people because it can be trained to be a threat. A trained neural network can be used for simultaneous control of an enormous number of computers during DDOS or Slave-attacks, which emulate site user behavior and learn the reaction system of the attacked environment.

There is every chance it will be difficult, if not impossible to secure a site from such a combined attack. A well-trained neural network is capable of attacking all fronts at the same time: data reading, user emulation, real users’ password mining and controlling thousands of computers from different locations (with potential IP spoofing in the process) to perform a hardly detectable DDOS attack.

Blockchain & S.O.I.D: The most promising defenders of deep learning networks

If it is impossible to alter or start controlling a neural network, one can take a different approach: the proper structuring of input data error correction algorithms and introducing new weighing arrangement principles. Where classical mathematics and logic have no power, other equally important sciences will work: philosophy and psychology. These are the two strong determinant constituents of training a network to recognize good and bad.

The prime and defining rule of network activity is: “A neural network should be limited to performing the single operation it was intended for. If you need to expand neural network activities, integrate several networks and use possible irrational results and their inference as a tool to achieve the desired multi-functionality."

To arrange neural network operations properly, it is necessary to design a basic matrix, which contains fundamental notions, outlines of network fields and applications preventing the neural network from dysfunctioning. To operate this matrix we define four additional control principles:

• S (Severity). The severity of input definition is a capability to block initial input and input through the first layers of the neural network as a part of the basic matrix with specifying notions. By doing so, the specifying notions cannot be replaced with opposite ones based on collective opinion (input) in the process of training.
• O (Open to Deep). S-principle works in the first input layers ensuring the protection of the network against “junk”, while at the same time not limiting the neural network in its “further reasoning”.
• I (Irrational Ability). The neural network can reject a more probable conclusion if it contradicts the neural network application, though it can use the obtained conclusion as a new functional block for further operations or interaction with other neural networks.
• D (Definability). The neural network, in the course of reasoning, can overgrow the limits of the matrix in comprehending the application domain, and rebuild local copy within the bounds of 0-49.5% (of the initial sample) to achieve the set goal.

A definition matrix is the workhorse of neural network logic along with a neuron. Thus, it must be well secured. A matrix by its nature is an enormous aggregation of data, which must be stored properly. So far, there is only one high-assurance way such a volume of data can be stored without violating compliance rules. This is Blockchain.

Blockchain was created to support distributed applications and secure storage of infinite data amounts. Thanks to its distributed structure and block arrangement method, it is very difficult to compromise (replace or delete) the data in a blockchain. The diagram below shows the architectural concept of interaction.

Thanks to the S.O.I.D principles described above and the democratic principle, which is a keystone of Blockchain, these matrices are accessible for all stakeholders of the process. This allows easy network “diagnostics”, mistakes correction (through S.O.I.D. a neural network can easily rebuild itself without damaging the existing logical conclusions), and network reasoning control assurance in relation to DPO agencies or bodies. If a network is created with malicious intent, it will be confirmed and the malefactor can be dealt with.

To learn more about the practical side of S.O.I.D principles and other ways to secure neural network operations, get in touch.