Cyber Security - The Design perspective: Operational Automation through Organic Escalation

During a cyber security incident response scenario, it is conceivable that an analyst would conduct a series of tasks, some of which are common across all incidents, some of it constrained to the technology pertaining to the alert that instigated the response, and the rest involving manual evaluation and context establishment. If one were to magnify that period of observation for a granular view of the tasks conducted over that time, this article develops a cognitive automation hypothesis. The basis for this hypothesis is that one can generate a calibration scale to uniquely identify tasks that warrant automation.

This article explores the possibility of building confidence in operational workflows escalating them into automation organically. I develop a cognitive model called SAI reinforced with a machine learning framework to organically escalate tasks into the operational automation when repeated actions deliver sufficient confidence. While the cognitive model discerns tasks from the sample space of {objective, technology-dependent, subjective}, the organic escalation is achieved through a reward-penalty model.

Finding: In the absence of external business factors, if the number of parameters influencing an alert is limited to C, and the total number of alerts in a given period is limited to K, then it is sufficient if, for any task involved in that alert remediation, the task receives at least 10.C.K endorsements during the same period, in order to escalate it into automation. The framework can seamlessly accommodate false positives that are opportunistic to any operational environment while providing savings in expenditure by transforming as a measuring scale for value of alarms in an Organisation.

The Problem. While Organisations use many advanced security tools for supporting diagnostic work, analysts frequently need to customize these tools to fit the specific needs. Conventional tools, however, provide very little if any support for such customization. Invariably, a large fraction of analysts’ time are thus spent in “tuning” them to achieve the organisational goals. This motivates one to identify opportunities for automation in the actions/workflow natively and scope the tasks such that automation can derive maximal efficiency.

An objective view of the sequence of actions that transpire between when a detection is observed in, say the organisational SIEM environment, to the time that it is remediated, can often reveal a wide range of activities, in the spectrum between completely objective tasks to purely subjective evaluation tasks. This can be regarded as your automation scale, if you will.

Figure 1. Illustrating the task categorization on workflows associated on alerts from different technologies 

The Proposal. This paper explores the possibility of building confidence in operational workflows escalating them into automation organically. The term organic escalation is derived from evolution theory wherein an activity that is mechanically repetitive, when performed “enough” number of times, can supply confidence in activity itself in addition to a refinement that delivers it to muscle memory[1]. Consider a security operational center with a number of analysts triaging and analyzing alerts/detections coming through on their organisational SIEM every day. If one were to observe and record the nature of actions conducted by the analysts, one would note three things;

1.      A set of actions conducted by the analysts, irrespective of types of alerts they are triaging/analyzing on any given day

2.      A set of actions that are completely dependent on the technologies that is producing the alerts; and

3.      A set of actions that are entirely dependent on the analysts and their understanding of the alert and the underlying technology itself.

The good news is, items listed 1 and 2 are within the scope for automation and can deliver enormous benefit to the organisational efficiency and improving bandwidth within the SOC. With careful planning and background detailing item listed 3 can be broken down into components that:

i.                   can be automated

ii.                 cannot be automated

Figure 2. Illustrating the breaking down of a subjective task

This can provide a simplified workflow that would curtail the components that cannot be automated to a few tasks. Spacing out such tasks in time can significantly enhance the efficiency by positioning those actions between largely automated sections of tasks.

[1] Muscle memory is synonymous with motor learning, which is a form of procedural memory that involves consolidating a specific motor task into memory through repetition. When a movement is repeated over time, a long-term muscle memory is created for that task, eventually allowing it to be performed without conscious effort. This process decreases the need for attention and creates maximum efficiency within the motor and memory systems.

The Learning. In principle, the model we propose is mathematically congruent[2] to the multi-armed bandit problem[3] from probability theory. Based on the reinforcement learning model in machine learning theory that also adopts a reward-penalty model to evaluate the state space, we illustrate the random approach (green line in Figure 3) is only effective 9% of the time at saturation, the average greedy (seeking short-term best reward, blue line in Figure 3) approach can only be effective 18% of the time at saturation, while the - greedy approach (red line in Figure 3) which remains greedy for a fraction of the exploration while at other times, it operates according to a state dictated action can achieve up to 58% effectiveness at saturation.

Figure 3. Illustrating the reinforcement method-based solutions multi-armed bandit problem highlighting the challenges with greedy approaches.

The lesson we elicit from this strategy analysis is the following: If an analyst actions on alerts based on a selection pattern that can be treated as random, it can deliver an asymptotic efficiency of 9%. Besides, while an analyst may be able to act on those alerts where the remedial action is apparent while disregarding (greedy approach) the other long-winded alerts, the approach can deliver only 18% efficiency and if the analyst chooses sampling approach (epsilon - greedy), wherein one may act greedy for a fixed fraction of their time (this can be treated as a periodic cycle over an arbitrary finite period, if you will), it has the potential to asymptotically deliver 58% efficiency. This efficiency is independent of how the remedial action is accomplished, manually, or automated. As a result, to holistically improve the efficiency of the process, we adopt the organic escalation which uses the concept of muscle memory to improve confidence (the threshold matrix Γ), subject to its various parameters (the set P).

[2] In abstract algebra, a congruence relation is an equivalence relation on an algebraic structure (such as a group, ring, or vector space) that is compatible with the structure. Every congruence relation has a corresponding quotient structure, whose elements are the equivalence classes for the relation.

[3] In probability theory, the multi-armed bandit problem (sometimes called the K- or N-armed bandit problem) is a problem in which a gambler at a row of slot machines (sometimes known as "one-armed bandits") has to decide which machines to play, how many times to play each machine and in which order to play them. When played, each machine provides a random reward from a probability distribution specific to that machine. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls.

The Benefits. The framework attempts to alleviate the pain-points to release the stress on the workflow that naturally equates to monetary benefit. If a particular task is manually conducted n number of times per day by k analysts, each lasting t minutes, the effort lost per day can be computed as (n.k.t/480) man-days. For instance, consider that a task conservatively takes approximately three minutes to complete and there are three analysts operating the workflow which involve that task on a given week. This indicates, if the number of alerts exceeds 53 during that week, that the S-O-C would have lost an entire man-day manually operating that task! While duration of the task is a conservative estimate, analysts in the community should be aware that the number of alerts is several degrees north of that. Besides, if there is a task that could take 10 minutes on triage and analysis, the time lost on manual operation triples!

If that corresponds to h full time analysts per year, equated to G dollars, that can saved annually if the task were to receive sufficient confidence to escalate organically. Conversely, if the task were to receive more confidence penalties than corresponding tasks, it can be dropped, again saving costs, since it is likely an ill-analysed task in the workflow, requiring further scrutiny.

Besides, if an alarm ai were a false positive, it comes to reason that the action would not necessarily be repeated often since the set of tasks associated with it can change. This directly reflects in the confidence vector for that alarm which would not mature, when observed over a “significant” period of time. Naturally, at that point, the alarm and its action can be retired without impact to the remaining workflow that operate mutually independent of each other. This demonstrates that our framework can accommodate false positive alarms while maintaining focus on the actions of import, allowing its seamless integration into any operational environment.