How can you be sure your incident response metrics are reliable?

3 main objectives : **Align Metrics with Objectives:** Start by clearly defining your incident response objectives. Whether you're aiming to reduce response times, minimize downtime, or enhance threat detection, your metrics should directly reflect these goals. **Adaptability:** Incident response is an evolving field, and your metrics should reflect that. Regularly review and update your chosen metrics to stay relevant to changing threat landscapes and business needs. **Benchmarking:** Compare your metrics against industry standards and best practices. This provides insights into how your organization stacks up against peers and helps set realistic improvement targets.

One needs to clear differntiate between operational objectives and management objectives. Operational objectives focus on operational efficiency and efficacy. Management objectives would typically look to measure outcomes such as risk reduction, posture improvement, return on security investments. Your objectives will determine the metrics and data required to measure these.

I think think the most valuable metric for Incident Response is Dwell or turn-around time. Do you have the capability to determine when compromised hosts were initially accessed? How long was between then they gained access to a host, and when actors were first detected? How long did it take between initial discovery and containment? Containment and remediation? Time is the most valuable asset to both you and your adversary. The longer they are present, the further along the adversary moves towards their end goal. The faster you and your team can find them, the less time they have to make moves. The best advice I can give to C-Suite when considering how and where to bolster their defense is to listen to the analysts about pain points.

If working for an MDR company, considering customer success is important when crafting objectives. Since MDR providers deliver services to a range of segments, it is crucial to ensure you are targeting the correct objectives to enable success at scale. The impact of these incident response strategies touch more than one team, business unit or company, rather impacting industries. This may require various objectives which map to larger business goals, ensuring success for clients at scale.

MTTC is the most critical metric. Incident response is two pronged between detection and containment. Knowing you have a problem is the first step but doing the right thing to take control over it is immensely more relevant. Impact is immeasurable without containment. I often describe incident response like a war time triage hospital. The most important job is stopping the bleeding and stabilizing the patient. Only then can you pause to observe, orient, decide, and act on your next objective.

Sometimes it helps to broad-base your metrics. You might want to measure improvement in security posture over a period of time. To determine this, you might want to measure the reduction in number of incidents while controlling for correlation rules. You might also look to measure the success of certain security projects in terms improvement in incident count or speed of remediation.

A lot of times, the data requires stratification. Which means that you might need to convert certain large amount of operational data into a humanly interpretable summary. This might require certain amount of technology interventions and automation.

Incident response transcends the technical; transparent, understandable reporting is crucial. Translating these technicalities for non-technical stakeholders is paramount! My reporting focuses on incident summary, impact analysis, our handling, and gleaned lessons. I streamline this via tools like D3 Security's IR module and RSA's Archer as solid reporting capabilities. I employ visual aids for report clarity, including graphs, charts, and diagrams. Further, my reports always highlight preventive strategies and future recommendations to bolster security. In essence, employing advanced tools, integrating threat intelligence, and maintaining clear reporting are my critical strategies in incident response.

If supporting a large volume of clients, it is valuable to leverage scalable schemas and tools to ensure data collection and analysis is not complicated and easily understood. It is one thing to collect data and another to make it usable and comprehendible. If the methods are too complicated, teams will become dependent on a team who understands the complex system. This scenario ultimately will slow down delivering meaningful outcomes thus impacting company growth and effectiveness.

The main goals of the analysis are to eradicate or minimize the root cause of the issue and to carry out a lessons-learned analysis involving a cross-functional team. This analysis aims to evaluate the efficiency and effectiveness of incident handling while identifying any existing problems. It also seeks to capture lessons learned from the experience, identify the initial root cause, pinpoint issues that arose during the execution of actions, and highlight any deficiencies in policies, operational training, or EDR/XDR tools improvement and procedures (missing) that may have contributed to the incident.

Dwell time is the amount of time that passes between when a cyberattacker gains initial access to a system and when their presence is detected. This gives attackers plenty of time to steal data or plant malware that can wreak havoc on a system. The goal of attackers is to increase their dwell time so they can do as much damage as possible before being caught. Hence, dwell time is often used as a measure of an organization’s cybersecurity effectiveness. The longer the dwell time, the more likely it is that an attacker will be able to cause serious damage. As a result, organizations should strive to minimize their dwell time by implementing security controls and monitoring their networks closely.

Cost of an incident. Cyber incidents can have a significant impact on an organization, causing financial losses, reputational damage, and operational disruption. Measuring the cost of these incidents is essential for understanding the true extent of the damage and determining where to allocate resources for prevention and mitigation. Without accurate cost data, organizations may be under- or over-investing in cybersecurity, putting themselves at unnecessary risk. Cost data also provides valuable insights into the effectiveness of existing security controls and can help to prioritize future investments. In short, measuring the cost of cyber incidents is a critical part of managing cybersecurity risks.