The Role of Analysis of Competing Hypotheses in Cyber Threat Intelligence: Case Studies and Applications in Real-World Incidents

Cyber threat intelligence (CTI) has grown into a vital component of modern cybersecurity, empowering organizations to proactively identify, evaluate, and respond to the ever-evolving landscape of cyber threats. The complexity and scale of modern attacks necessitate sophisticated analytical tools to dissect, prioritize, and address the vast amount of data generated during cyber incidents. One of the most effective methods to handle this complexity is the Analysis of Competing Hypotheses (ACH), a structured approach that helps analysts rigorously evaluate multiple explanations for observed activities. Although ACH was initially developed for military and national intelligence analysis, its methodical nature makes it equally valuable in cybersecurity, where ambiguity and uncertainty are common.

In this column, we delve into how ACH enhances CTI by analyzing real-world case studies where the technique has been applied. These case studies demonstrate ACH’s role in accurately diagnosing cyber threats, refining intelligence assessments, and guiding strategic decision-making during and after cyber incidents.

ACH: Core Principles and Value in Cybersecurity

At its core, ACH is a structured approach to evaluating competing explanations for observed events by systematically weighing evidence for and against each hypothesis. This methodology is particularly useful in cybersecurity contexts, where attackers often employ techniques to confuse or mislead analysts, such as false flags, decoy operations, and obfuscation.

ACH helps analysts avoid common cognitive biases like:

• Confirmation Bias: Tendency to focus on information that supports an existing belief or favored hypothesis.
• Anchoring: Overreliance on the first piece of information encountered, which can skew subsequent analysis.
• Availability Heuristic: Tendency to base conclusions on readily available information, often at the expense of more comprehensive data analysis.

ACH’s step-by-step methodology helps CTI teams remain objective and thorough, facilitating decision-making in high-stakes environments where incomplete or misleading data is the norm. Its steps include:

1. Hypothesis Generation: Analysts start by brainstorming all plausible explanations or hypotheses for the observed event or activity.
2. Evidence Collection: Relevant data is gathered, and each piece of evidence is evaluated against each hypothesis without preconceived notions of which hypothesis might be correct.
3. Matrix Creation: A matrix is constructed with hypotheses on one axis and evidence on the other. Each piece of evidence is assessed to determine whether it supports, contradicts, or is neutral towards each hypothesis.
4. Scoring: Each hypothesis is scored based on the amount and strength of disconfirming evidence, rather than focusing only on supporting data. The hypothesis with the least amount of contradictory evidence emerges as the most viable.
5. Sensitivity Testing: Sensitivity analysis is conducted by adjusting or removing specific evidence to determine how the conclusions might change. This step ensures that the final outcome is robust, even if certain pieces of data are later proven to be incorrect or unreliable.
6. Refinement: As new evidence becomes available, analysts can revisit the ACH matrix to adjust hypotheses, ensuring that conclusions remain dynamic and adaptable to new insights.

This process prevents premature closure on a single hypothesis, promoting a broader view of potential explanations and enabling organizations to stay ahead of adversaries who employ sophisticated, multi-layered tactics.

Case Study 1: Uncovering Lazarus Group's Financial Operations

One of the most prominent cyber threat groups of the past decade is the Lazarus Group, widely believed to be a North Korean state-sponsored actor. The group’s operations are varied, ranging from cyber espionage to large-scale financial theft. Among its most infamous attacks was the 2016 attempted theft of $1 billion from the Bangladesh Bank through a sophisticated manipulation of the SWIFT international banking system.

ACH Application

When this attack was first detected, investigators faced several competing explanations for the identity and motive of the attackers. The sheer scale and ambition of the operation led some to believe that it was the work of a highly organized cybercriminal group. Others suspected a state-sponsored operation. The application of ACH enabled investigators to break down the incident methodically:

1. Hypothesis A: The attack was conducted by an organized cybercriminal group driven by financial gain.
2. Hypothesis B: The operation was state-sponsored, likely by North Korean actors, seeking to bypass international sanctions and fund governmental activities.
3. Hypothesis C: The attackers were part of a false flag operation designed to shift blame onto North Korea or another state actor.

Using ACH, investigators began systematically gathering evidence and filling in the matrix:

• Evidence: The attackers used a highly customized strain of malware designed specifically to target the SWIFT system. This kind of development requires significant resources and expertise.
• Evidence: Several digital artifacts, such as domain registration details and malware signatures, showed overlap with previous attacks attributed to Lazarus Group.
• Evidence: The attack targeted the central bank of Bangladesh, and the funds were set to be transferred to accounts in the Philippines, a country with weaker anti-money laundering controls.
• Evidence: The timing of the attack coincided with international sanctions tightening around North Korea, and previous intelligence reports suggested the regime was in desperate need of foreign currency.

Through this methodical approach, Hypothesis B—that the Lazarus Group, backed by North Korea, was responsible—emerged as the strongest conclusion. Further investigation corroborated this, as additional links between Lazarus Group and the infrastructure used in the attack were uncovered. ACH helped analysts avoid prematurely concluding that this was a cybercriminal operation, ensuring that the right resources were focused on pursuing a nation-state adversary.

Case Study 2: SolarWinds Supply Chain Attack - Unraveling a Complex Espionage Operation

The 2020 SolarWinds attack was one of the most significant supply chain compromises in history. By embedding malware into an update of SolarWinds’ Orion network management software, attackers gained access to thousands of organizations, including major U.S. government agencies and corporations. The attack went undetected for months, raising critical questions about the identity, motivations, and long-term objectives of the attackers.

ACH Application

In the immediate aftermath of the attack’s discovery, multiple hypotheses circulated regarding the actors behind it. The nature of the attack pointed towards advanced threat actors, but there were competing theories about which nation-state was responsible. Using ACH, investigators explored several hypotheses:

1. Hypothesis A: The attack was perpetrated by a Russian state-sponsored actor, possibly APT29 (Cozy Bear), known for its stealthy cyber espionage campaigns.
2. Hypothesis B: The operation was carried out by a Chinese APT group, with objectives similar to Russia’s.
3. Hypothesis C: This was a financially motivated cybercriminal group seeking access to high-value corporate and governmental targets for extortion or blackmail.
4. Hypothesis D: The attack was a false flag operation designed to obscure the true origins and pin the blame on a nation-state actor.

The ACH process helped organize the evidence against these hypotheses:

• Evidence: The malware, dubbed Sunburst, used a highly stealthy and sophisticated backdoor that allowed long-term persistence without detection. This level of stealth and operational security pointed towards state-sponsored actors.
• Evidence: Investigators discovered infrastructure and tactics associated with known Russian APT groups, particularly APT29.
• Evidence: The targets of the attack, primarily U.S. government agencies and key infrastructure entities, aligned with Russia’s historical espionage objectives, further reinforcing Hypothesis A.
• Evidence: No demands for ransom or exfiltration of corporate data were observed in the early stages of the investigation, weakening Hypothesis C, which would have involved rapid monetization attempts.
• Evidence: There was no substantial evidence of deception or intentional attribution efforts that would point to a false flag operation, making Hypothesis D less likely.

By applying ACH, investigators narrowed their focus to Hypothesis A, concluding that the SolarWinds attack was likely orchestrated by Russian state actors, specifically APT29. ACH’s systematic approach ensured that competing explanations were thoroughly evaluated before a conclusion was reached, allowing for a measured and accurate response. The eventual U.S. government attribution of the attack to Russia further validated the ACH-driven investigation process.

Case Study 3: NotPetya - A Politically Motivated Malware Disguised as Ransomware

The NotPetya malware attack of 2017 appeared at first glance to be a typical ransomware campaign, but upon closer examination, it was revealed to be one of the most destructive cyberattacks in history, crippling global businesses and government institutions. The attack was particularly devastating in Ukraine, where critical infrastructure and businesses were targeted. ACH played a pivotal role in disentangling the true nature of the attack and its motivations.

ACH Application

NotPetya’s initial appearance as a ransomware attack led many to believe that it was a financially motivated cybercrime operation. However, the nature and impact of the malware quickly raised questions, and ACH was employed to evaluate several hypotheses:

1. Hypothesis A: NotPetya was a conventional ransomware campaign designed to extract payments from victims.
2. Hypothesis B: The attack was politically motivated, aimed specifically at destabilizing Ukraine by causing widespread economic and infrastructure damage.
3. Hypothesis C: NotPetya was part of a broader geopolitical cyber warfare strategy, targeting multinational organizations to create global disruption.

The ACH matrix was filled out as follows:

• Evidence: The ransomware mechanism in NotPetya was poorly implemented. In many cases, victims were unable to recover their data even if they paid the ransom.
• Evidence: The majority of early victims were Ukrainian entities, particularly in sectors such as finance, energy, and government.
• Evidence: NotPetya used the same exploit (EternalBlue) that had been used in previous state-sponsored cyberattacks, including WannaCry, which was linked to North Korea. However, the infrastructure and execution pointed more towards Russia than North Korea.
• Evidence: The attack spread beyond Ukraine, affecting multinational corporations such as Maersk, Merck, and FedEx. The collateral damage raised the possibility that the attack had broader geopolitical implications.

ACH’s structured approach led to the conclusion that Hypothesis B—NotPetya was primarily a politically motivated operation aimed at Ukraine—was the most likely explanation. This was later confirmed by security experts who attributed the attack to Russian military intelligence (the GRU), which had been involved in previous cyber operations against Ukraine. The ACH process also ruled out the hypothesis that NotPetya was purely a ransomware attack, enabling organizations to shift their response strategies from paying ransoms to focusing on mitigating the broader geopolitical implications of the attack.

Case Study 4: Operation Cloud Hopper – Complex Cyber Espionage Targeting Managed Service Providers

Operation Cloud Hopper was a multi-year cyber espionage campaign targeting managed service providers (MSPs) globally. This operation was attributed to a Chinese APT group, APT10, which sought to exploit the trust relationships between MSPs and their clients to gain access to a wide range of sensitive corporate data. The campaign was sophisticated, involving long-term persistence, stealthy exfiltration of data, and complex obfuscation techniques.

ACH Application

Given the global scale and stealth of the campaign, several competing hypotheses emerged about the identity and motivations of the attackers. ACH was used to explore these possibilities:

1. Hypothesis A: The operation was a financially motivated campaign by a cybercriminal group seeking access to corporate data for sale on the dark web.
2. Hypothesis B: The campaign was a state-sponsored espionage operation conducted by Chinese APT groups, aimed at acquiring intellectual property and other sensitive corporate information.
3. Hypothesis C: The attack was carried out by a nation-state actor other than China, using Chinese tactics and infrastructure to create plausible deniability.

The ACH matrix was populated with the following evidence:

• Evidence: The attack methodology involved the long-term compromise of MSPs, which provide IT services to multiple clients, allowing the attackers to gain access to many organizations simultaneously.
• Evidence: Many of the tactics, techniques, and procedures (TTPs) used in Operation Cloud Hopper matched those previously observed in campaigns attributed to APT10, a Chinese state-sponsored group.
• Evidence: The data exfiltrated included sensitive intellectual property, trade secrets, and proprietary information from sectors such as aerospace, defense, and telecommunications, all of which are priorities for Chinese espionage.
• Evidence: The attackers exhibited persistence and patience, maintaining access to networks for extended periods to exfiltrate large volumes of data without detection.

Through the ACH process, Hypothesis B—that Operation Cloud Hopper was a Chinese state-sponsored espionage campaign—emerged as the most likely explanation. This conclusion was later confirmed by government and private-sector investigations, which attributed the campaign to APT10. ACH allowed investigators to systematically eliminate alternative hypotheses, ensuring that defensive measures were appropriately focused on nation-state threats rather than misdirected towards financially motivated actors.

Lessons Learned and Best Practices from ACH in Cyber Threat Intelligence

The application of ACH across these case studies highlights several critical lessons for CTI analysts:

1. Structured Methodology Mitigates Cognitive Bias: ACH forces analysts to evaluate evidence dispassionately, reducing the influence of biases like confirmation bias and premature conclusions. This ensures that investigations remain open to multiple explanations, preventing the team from becoming too focused on a single hypothesis early in the process.
2. Evidence-Based Approach Enhances Objectivity: ACH’s emphasis on disconfirming evidence rather than just supporting evidence helps analysts avoid the pitfall of only seeking information that validates their favored hypothesis. This creates a more balanced and objective assessment of the situation.
3. Scalability and Adaptability: ACH is not limited to specific types of cyber incidents or actors. It can be applied across a range of scenarios, from espionage campaigns to financially motivated attacks. Its flexibility ensures that it can be scaled to handle both small and large incidents, making it an essential tool for any CTI team.
4. Collaboration and Cross-Disciplinary Input: ACH encourages collaboration between teams and disciplines. By providing a common framework for discussion, analysts from different backgrounds (technical, geopolitical, etc.) can contribute insights to the process, enriching the overall analysis.
5. Focus on Disconfirming Evidence: One of ACH’s most powerful features is its focus on identifying disconfirming evidence for each hypothesis. This approach helps analysts quickly eliminate less likely hypotheses and focus on the most plausible explanations, leading to faster and more accurate conclusions.
6. Dynamic and Iterative Process: ACH is not a one-time process. As new evidence emerges, hypotheses can be re-evaluated, and the matrix can be updated. This dynamic approach allows organizations to remain agile in their investigations, adapting their understanding as more information becomes available.
7. Resource Prioritization and Strategic Decision-Making: ACH helps CTI teams prioritize resources by eliminating less likely hypotheses early in the investigation. This ensures that time, energy, and tools are focused on the most plausible threats, improving the efficiency and effectiveness of incident response.
8. Enhanced Attribution: One of the most challenging aspects of CTI is attribution—identifying the responsible actor behind an attack. ACH’s structured approach enables more accurate attribution by systematically evaluating evidence across multiple hypotheses, ensuring that the right actors are identified and held accountable.

In conclusion, the Analysis of Competing Hypotheses (ACH) is a powerful tool that has proven its value in real-world cybersecurity incidents. From the Lazarus Group’s financial theft to the SolarWinds espionage campaign, ACH has enabled CTI analysts to systematically evaluate evidence, eliminate less likely explanations, and identify the true nature of the threats they face. As the cyber threat landscape continues to evolve, ACH will remain an essential part of the CTI toolkit, providing a rigorous, evidence-based approach to investigating complex incidents and ensuring that organizations can effectively defend against the most sophisticated adversaries.