What are the most important cybersecurity metrics for senior management?

Cyber Risk exposure can be calculated in different ways like Cyber risk = (threat x vulnerability x probability of occurrence x impact)/controls Cyber risk = Threat x Vulnerability x Value of Information Better way to calculate can be a customized model depending upon the Cyber ecosystem and process around The Cyber risk exposure calculation depends upon 1. Threat like attack vectors 2. Vulnerability like system flaws 3. Impact – which means actual loss or harm that can be there 4. Value of asset that can be lost by the cyber-attack. 5. Cyber exposure factor 6. Cyber loss expectancy 7. Possible Rate of occurrence of cyber attack The Model may vary from organization to organization and needs a regular calibrations.

In my experience, the top challenge is except the auditor and may be the consultant who designed the risk measurement, rarely the business and technology management and leadership understand, discuss before signing off. Risk workshops at various levels are critical to risk awareness, common language and risk ownership.

For senior management, key cybersecurity metrics includes: Incident Response Time Mean Time to Detect (MTTD) and Respond (MTTR) Number of Security Incidents Data Breach Impact Cost of Security Incidents Patch Management Effectiveness Compliance with Security Policies Risk Assessment Efficiency False Positive/Negative Rates in Alerts User Training Effectiveness

Indeed, one of the biggest challenges in Cybersecurity is measuring its impact on the existing business. Risk exposure is a good way of doing it, but it will be meaningless if calculated incorrectly. Therefore, in my opinion, the best way to measure cybersecurity is to measure the quality of its implementation process, by assessing adherence to existing cybersecurity best practices, which can be found in reputable standards and frameworks such as NIST-SP-800-x, IEC62443-x-x and ISO 2700x.

In my experience, risk exposure tends to be very subjective, and a common thought trend from many cyber personnel is “it won’t happen here”. Cyber risk metrics should not create fear or panic, nor should they push someone’s pet project forward. I have found, to create the most informative metrics, one needs to add context. Nuances such as inherited controls and network micro segmentation are often not able to influence the risk calculation but are critical in understanding the overall risk. Context is an incredible way to start crucial informed conversations about cyber vs. a battle for control over funding.

Cybersecurity metrics should be chosen carefully depending upon the Business context, applicable laws, regulations. Following 10 can be a starting point for Compliance related metrics for Senior Management. 1. Third party access review compliance percentage 2. Security Awareness Percentage for employee 3. Security Awareness Percentage for Third party supporting You 4. Compliance percentage to PCIDSS or applicable regulation/law 5. Number of Cyber Security incidents reported to the regulator/authority. 6. First Party Security Rating for organisation 7. Average Third-Party Security Rating 8. Security Audit compliance Percentage 9. Cybersecurity incident reported at Third Party 10. Compliance Percentage to Organization’s Cybersecurity Policy

Another important metric for senior management is the one that tracks compliance with relevant regulations, industry standards, and contractual obligations, demonstrating the organization's commitment to legal and regulatory requirements and reducing the risk of penalties or fines.

In my experience - as mentioned - context and understanding of the business are key components. Just pointing to a 'best practice' is not the only answer - first it must be applicable to your business. The analysts must also understand the mission/business they are assessing risk on. If you don't know, then the analysts could create more risk based on 'pushing' controls that impede current mitigations. You really need to understand the business, know the context of the controls and then perform the analysis.

Compliance is important and a driver for cyber security. But compliance frameworks take time to be built. Good security does not mean compliance. And compliance does not mean good security. It's best to thrive for good security and let "compliance" follow. (while not ignoring that compliance is important)

Most important activity to maintain cyber posture is to know the assets we are trying to protect. Asset inventory and classification and criticality of the asset to business and assigning a value with agreement with the business stakeholders is key in risk reporting. Identifying ongoing vulnerability assessment for those assets, evaluating defensive control effectiveness, and calculating residual risk and reporting the same to the business.

Like any Performance Indicator, the vulnerability assessment metrices are most effective if it is mapped to business objective and provide a direction to support business goals. Identification, remediation and continuous improvement of the vulnerabilities is the key and the effectiveness of the vulnerability assessment program can be measured by: 1. Scan Coverage Percentage (all assets – external & internal ) 2. Scan Frequency 3. Quantitative Risk Score 4. Mean time to remediate. 5. Number of Zero Day Vulnerabilities discovered. 6. Number of Observations opened after remediation. 7. Risk wise average Vulnerability age 8. Exception Percentage 9. Risk Acceptance Percentage 10. Percentage of Healthy Systems with no open vulnerabilities

Senior management is usually not interested in how much you scan and what is being found. They are interested if they have to "intervene" (by escalating, prioritising, adding budget). Metrics that show progress are more valuable e.g. MTTR (so mean time to respond/fix vulnerabilities) over time. This can help showing if the money the organisation spends on cyber security is still having impact. Every "relative" (avoid absolute numbers) metric over time can help showing the ROI of security.

Vulnerability metrics using assessment methods like those mentioned are good, but they don’t provide context. When providing metrics to Senior Management you have to inform with the data, not just throw it out there. Massive negative vulnerability numbers could easily turn positive if the right context is provided. Unfortunately the blame is usually pointed very quickly at whomever is responsible for patching… but imagine if context and root cause analysis actually showed there was a flaw in scanning? This is a critical flaw in cyber metrics. The team creating the data should not be the team reporting on it. Separation of Duties should exist here too.

This insight helps manage risks, prevent breaches, allocate resources effectively, ensure compliance, and continuously improve cybersecurity, therefore management should be informed by CISO about the progress and the overall status. Management by understanding vulnerabilities and risks associated will need to provide the approval of the proactive measures proposed by CISO, such as resource allocations or increase of the total budget of security, with this aspect companies enhance their overall security posture, ensuring better incident response preparedness and compliance with cybersecurity internal or external requirements.

One important area to focus and metrics to include is about the blind spots. How much percentage of my assets are currently undergoing VA against the total and how many of them are going through with credential based scanning.

It is not important for Senior Management to know the process or procedure but It is very important for them to know about Incident Reporting: The number of reported security incidents and damage caused by them. Also it's important to know the Business Impact Analysis due to the Incident.

In addition, incident response metrics are an important indicator. Those are metrics that measure the organization's ability to detect, respond to, and resolve security incidents in a timely manner, including average incident response time, mean time to detect (MTTD), and mean time to resolve (MTTR).

Again, the details of the process are not important to the senior management. But can the team handle an attack? How did they do what in what time? Important KPI could be: 1. Rapid threat mitigation: It measures an organization's ability to swiftly detect, respond to, and recover from cybersecurity incidents, reducing potential damage. 2. Damage control: Effective incident response minimizes the impact on data, systems, and reputation, limiting financial and operational losses. 3. Continuous improvement: By analyzing incident response performance, organizations can refine security strategies and bolster future incident handling capabilities. 4. Mean time to respond: Average time required to return a system to operational condition.

This metric focuses on measuring the speed at which the company can detect and respond to a cybersecurity incident. A shorter incident response time is indicative of a more robust and efficient cybersecurity posture. It reflects the company's ability to minimise the impact of an incident and swiftly mitigate potential risks, showcasing the effectiveness of the incident response team and the overall cybersecurity strategy. Monitoring and optimizing incident response time is vital for minimizing downtime, protecting sensitive data, and maintaining the companies's resilience against evolving cyber threats.

Senior Management is primarily interested to know how resilient is their team to handle the eventualities. Incident response metrics can help to evaluate the readiness and resilience, and learn from the experiences and feedback. The figures that could assist are: -Incidents Related to Exploited Vulnerabilities: Shows the number and impact of security incidents resulting from exploited vulnerabilities, emphasising the real-world consequences of unaddressed risks. -Effectiveness of Patching in Incident Prevention: Measures the correlation between the effectiveness of the patching process and the prevention of incidents related to known vulnerabilities.

Effective Security Awareness metrics are overlooked by many organizations. Most firms measures lead metrics or operational indicators only. *Operational Metrics* - Lead metrics #of awareness sessions conducted, #of staff undergone training, #of staff attended in awareness initiatives etc, can be good measures to understand the success of awareness related activities. *Tactical* Improvement in the click rate of Phishing Simulation Tests *Strategic* - Lag Metrics Key to measure is how effectively the awareness initiatives changed the infosec culture, how the information security culture and habits improved, and decrease in the number and impact of security incidents (that are triggered due to lack of awareness)

Security awareness metrics: It is very important keeping in view of the human resource being the weakest link in Cyber security. This metrics can provide the Senior management can with valuable insights into the effectiveness of their security awareness training program, identify areas for improvement, and promote a culture of security throughout the organisation. Some of the important rate could be: - Security Awareness Training Completion Rates - Average Quiz Scores - Phishing Test Success Rates - Policy Acknowledgment Rates

Superficial metrics - 1. Number of awareness trainings completed 2. Number of phishing links clicked 3. Attendance at cyber awareness sessions Effective metrics: 1. Pulse Survey based on 7 aspects of human behavior 2. Human Risk Score - is it improving or deteriorating over a period of time? 3. Review score of the awareness trainings provided. 4. Engagement score - do people wait till the last date to complete trainings, or jump on the trainings as soon as they are released 5. Engagement at live awareness sessions (% of audience who are off their phones and pay attention) 6. How much people contribute to cyber safety initiatives? 7. Good Saves vs. Near Misses

For me it is very hard to measure security awareness in a number. Sure it can be done by a phishing simulation. But the most companies spend the required budget for a real-life-scenario. It also can be done by comparing the success of awareness campaigns every cycle. How successful has a campaign been with what effort and methodology? You also can track how fast the process and change management worked. At the end everything is a continuous improvement. Also awareness campaigns should improve.

Similar to Buddhism it should be said that you need Panna to practice Sila and to maintain Samadhi. Without wisdom one is not able to start practicing in proper and to maintain discipline in the process while creating more wisdom in the sustainable cycle. Always be in the know and informed as this is the key to achieving success in the path. Sadhu!

It is difficult to measure ROI on security investments, however today everyone is well aware about of threats and possible loss of revenue and brand reputation. As long as one can effectively implement relevant tools and ensure zero cyber security incidents, ensure 100% availability of systems- ROI is anyway recovered for no damage done. As long as such investment are done keeping in relevance and must have tools, it is like an insurance premium. Basis past incidents and any downtime faced, calculating ROI is easier.

The ROI is the best friend of financial driven management and the worst enemy of the security team. Often there is no real ROI in security. The management just should accept it. A better metric is return of security invest (ROSI), what basically measures the damage not incurred. When a damage is not incurred like the risk management has it calculated, there is a return of the risk mitigation measures.

What about RODNI - return on investment on damages not incurred? Do have incidents which are prevented and/or mitigated any value/any interest for the board?

Picture this: cybersecurity success is not just about fortifying defenses but also about proving the worth of those efforts to the bigwigs. Enter security ROI metrics – your superhero in showcasing the value generated from those security investments. These metrics go beyond jargon, translating your cybersecurity efforts into dollars, percentages, or ratios. It's the magic wand that justifies budgets, optimizes spending, and ensures your cybersecurity endeavors get the applause they deserve. So, when it comes to metrics for the top dogs, think ROI and watch your cybersecurity strategy take center stage!

Shifting security from a cost center to a value center aligned with business objectives starts with showing the business its return on investment (ROI). Business leaders want to know what security is doing with its resources and have the ability to manage risk through investment.

Cybersecurity metrics important for senior leaders? How is my team doing? Are they burning out? Do they feel like they have purpose, autonomy and mastery over their role? Are they looking for other work? Do they talk about their career progression or development? If there is a lot of ‘no’s’ in these metrics then you’re not only not doing your job properly, but you’re also putting the organisation at risk. An apathetic team is a team who is less able to respond when things go wrong because why should they care when the company doesn’t? It costs so much to train up cybersecurity people - more fool the bad leader who doesn’t track how their team is doing regularly and make efforts to improve the working conditions where they can.

Measure and demonstrate for your C-level: 1. How cybersecurity is aligned with the business objectives in your organization. This comes down to —> how cybersecurity supports your organization to become successful. 2. The organization’s security posture and cyber resilience. How you provide protection and resilience. 3. The maturity development of the continuous improvement related to cybersecurity. How you ensure current and future readiness. And build up the measurement based on your organization’s security requirements. The data will be a mixture of for example technical and organizational data collected from risks, compliance, customer requirements, industry standards, laws, regulations, technical metrics, operational excellence.

Track the meta-metrics that show how metrics are trending. Seek to become antifragile. Fragile systems are prone to break, robust systems delay breakage and resilient systems adapt after a break, but antifragile systems improve after breaking. Don't let a crisis go to waste. Improve processes after every incident. Then measure how the metrics improve. For example measure improvements to the "time to detect" a repeat incident. Measure the success of preventing a pen-tested repeat of the incident. Measure how your leading indicators improved after incident resolution. Mistakes will be made and our job is to make sure we don't repeat them. Rather we should make new and more interesting mistakes. Then improve the process after each one.

I'm seeing a growing trend for senior management to want more in-depth qualitative conversations about cyber security, moving beyond compliance scores, charts, and heatmaps. Metrics are useful for demonstrating trends and patterns but using them to portray any meaningful message is incredibly hard. "Security awareness compliance is 87%" doesn't tell management what they want and need to know... Would we survive a NotPetya scenario? How long would business operations be impacted? How much would recovery cost us? Those conversations are more difficult to orchestrate and don't scale easily, but the value exchanged more than justifies the effort.

Before establishing security metrics to present to senior stakeholders, we need to understand 3 things: 1- What are the key security priorities within the organisation? 2- What do the senior team need to know (patching cadence, results of internal scans may not be relevant for senior teams). 3- What does good look like over the next 12-18 months? Over the years, I have used a wide variety of metrics for both Exec and Senior stakeholders broadly covering; risk exposure, coverage/implementation, effectiveness, compliance, capability, incident response, efficiency and impact. All metrics, whether tech or non-tech should be meaningful, consistent, verifiable and reliable. Focus on what is important.