Cybersecurity Metrics: Problems and Solutions

Is your metrics program coordinated with decision-making?

Almost 4.2 billion records were exposed in 4,149 data breaches in 2016, according to a report from Risk Based Security. The worst-hit sectors were businesses at 51 percent of reported breaches, surpassing unknown (23.4 percent), government (11.7 percent), medical (9.2 percent) and education (4.7 percent) industries. While the number of data breaches remained about the same in 2015 and 2016, the number of records compromised skyrocketed last year, according to the report.

But wait, the Identity Theft Resource Center said there were 1,093 reported data breaches in 2016 — 40 percent more than the 780 breaches in 2015. Confusing things further, the Privacy Rights Clearing House counted 538 breaches occurring in 2016 with just over 11 million records lost.

To be sure, there are plenty of explanations, different definitions, regional exceptions and so on to account for the conflicting numbers. But how do we easily explain this to business leaders?

What’s my point? The security industry has a metrics problem — and not just with counting breached records. If you add in disparate definitions of “security incidents,” numbers of “vulnerabilities,” “threats” or even what’s included under “cybersecurity,” you will see that different organizations use different terms, accounting and approaches, making apples-to-apples comparisons very hard.

What About Enterprise Security Metrics?

But enough about tabulating industrywide security metrics. How are you doing at measuring risk in your organization?

Sadly the gap between management expectations and reality usually gets worse when serious academic rigor is applied to measuring local cybersecurity programs. Many governments are just happy to have any security metrics at all. Often, easy-to-find items like “spam emails blocked” or “viruses detected and eliminated” are the only things counted, since network and security tools easily capture these cyberalerts. But is this practice acceptable?

Digging deeper, is your security health report truly measuring risk and evaluating future investments in people, process and technology? No doubt, reporting big numbers to managers (often measured in the millions of hostile data elements removed) looks impressive on management reports, but has anyone asked tough questions about these reports lately?

Have you ever matched the metrics you’re collecting to management decision-making? Are the relevant definitions clear and consistent? Is the threat intelligence data reliable? What is the process for creating security action items and priority levels? Who is (truly) looking at the captured data in a timely manner? Where can leadership turn for answers during an incident?

Maintaining Impactful Metrics

What can be done? Here are three steps you can take to strengthen cybersecurity metrics, communicate risk levels, and recommended actions to the right people up and down the management chain.

Know your enterprise security data, collection capabilities, policies and current reports. Who is doing what regarding your organization’s metrics collection processes now? Review risk assessments and security operation capabilities that are already in place from an end-to-end perspective. Ask what reports are really being read and used, and by whom.

Talk to top executives, financial staff, external partners and your internal team about “must have,” “nice to have” and “wasteful” metrics. What compliance reports are required by auditors? How can internal and external partners help? What risk-measuring results are expected? Consider if cyberinsurance checklists and processes can help document risk-reducing steps that lower premium costs.

Build (and use) a meaningful security dashboard for executives. Make sure the detail behind the metrics are real. As you build your future metrics model, examine best practices and talk with industry peers to understand what is working in your business sector. A few years back, the National Governors Association’s Resource Center for State Cybersecurity helped to build a template that can be used for government security dashboards. These templates are a helpful start. The Center for Internet Security consensus metrics are also valuable.

Building security metrics, measuring risk and improving cyberincident communications aren’t “one and done” processes. Seek to constantly improve and refine cybersecurity metrics, while maintaining your historical data and capturing trends.

Don’t just “check the box,” recheck your cybermetrics.

This article was originally published in the April / May 2017 issue of Government Technology Magazine.

You can follow Dan Lohrmann on Twitter: @govcso

 Dan’s blogs can be found at: Lohrmann on Cybersecurity & Infrastructure