Cybersecurity: Leading with Results

3 Questions to Adopting Outcome-Driven Metrics for Cybersecurity

A PDF of this article can be found here.

Organizations face a wide variety of Cybersecurity challenges in the modern landscape.?Frequently these threats are addressed with governance plans, Standard Operating Procedure (SOP) checklists, and reports filled with weekly or monthly metrics quantifying how many issues were addressed.?At SiloSmashers, one of the ways we endeavor to achieve Peak Performance is by deploying not just status report-type monthly metrics, but metrics that are themselves tracked over time – and lead directly to specific cost-related outcomes.

When it comes to Cybersecurity, it’s not enough to track the checklist of spending, hiring, data exfiltration, documents downloads, or ISO standards compliance. This was once the industry approach, including at SiloSmashers: once the checklist is complete, so is the assessment. Those numbers are a starting point for taking action – but what do we do with the information? Are we understanding how these metrics apply to the organization and correlate to costs? …Or are we just checking that ever-present box??And then… what’s next?

Put plainly: Customers won’t care how much an organization tracked standards compliance and spent on Cybersecurity if their systems are breached.

Cybersecurity is not a one-time event or product; it’s an ongoing process.?Tracking compliance-based metrics from the latest standards outlined in NIST SP 800-53 or ISO/IEC 27001 alongside your Information Systems Security Officer (ISSO) is only the first step.

While compliance is unquestionably essential, it doesn’t necessarily translate to effective Cybersecurity. Outcome-driven metrics, on the other hand, measure the actual effectiveness of security measures by tracking and evaluating the organization's ability to prevent, detect, and respond to attacks in real-time. These metrics, part of SiloSmashers’ Peak PerformanceTM approach, involve continuous assessment and proactive responses to evolving threats, making them more effective in ensuring robust Cybersecurity. By focusing on outcomes first, organizations can better adapt to the dynamic cyber landscape, ultimately providing a higher level of security.?And it doesn’t have to be a huge effort to pivot in this direction.

The Evolving Landscape

The cyber landscape is constantly changing with new threats and the evolution of old threats, necessitating a shift in Cybersecurity approaches.?Traditional Cybersecurity controls, like those from NIST SP 800-53, are still necessary but not sufficient on their own. Moves to a continuous monitoring approach can lower risks to an organization and make an outcome-driven approach simpler to integrate. Continuous integration lends itself well to tracking the same metrics the outcome-driven approach reaches for.?In addition, continuous reviews of a platform’s security landscape allow the organization to identify (and therefore address) risks and threats much more rapidly.

Emerging Security Approaches

In general, properly designed security metrics and key performance indicators (KPIs) are essential for strong Cybersecurity operations and help communicate Cybersecurity investments as business decisions to executives.?Traditional metrics such as the number of incidents closed or attacks faced are less relevant, because they are not actionable.

Outcome-driven metrics (ODMs) provide a clearer picture of Cybersecurity success by directly linking security metrics to business outcomes.?Organizations can prioritize these outcomes by focusing on measuring, reporting, and investing in specific security outcomes. This includes driving investments based on protection levels measured by ODMs, leading to better resource allocation.?ODMs indicate protection levels and help business leaders understand and invest in different protection levels for various parts of the organization. This alignment helps explain the impact of Cybersecurity spending to executives.

Question 1: What metrics do we already capture?

Progressing to an Outcome-Driven Metrics Posture

Making the change to focus on outcomes may seem like a paradigm shift involving new data capture, new tools, and new processes. Like SiloSmashers, most organizations are already in position with the data capture.?What needs to shift is in prioritizing the data to focus on and understanding how to apply that information to the organization’s costs.

The Gartner research and consulting firm benchmarks 16 ODMs, including time to patch, endpoint protection, and Mean Time to Recover. These benchmarks are straightforward enough to track in a simple spreadsheet. No additional tools are needed. Their simplicity affords clear understanding across stakeholders and allows organizations to easily track progress over time.?Once stakeholders and executives are on the same page, the mission presents itself clearly.?Funding and prioritization become a more direct conversation, rather than an exercise in technical nuance.

In a later stage of maturity, ODMs are categorized and measured in the context of critical and high-risk assets, third parties, alerts, vulnerabilities, incidents, and policy exceptions.

Question 2: How does each metric benefit the organization?

Outcome-Driven Metrics and Organizational ROI

Key to capturing and taking action on these metrics is understanding how they apply to an organization’s overall costs.? A flip answer is “Because we need to protect ourselves and our customers.” This response is true in spirit, but doesn’t dig deeply enough. ODMs can contribute directly to an organization’s bottom line by avoiding risk and data exposure, and also by spotlighting opportunities to streamline processes to build in security at the outset, rather than as a salve after the fact.

The Monte Carlo ROI Pyramid provides a direct way to calculate ROI for a product or process:

In the early stages of adoption, this calculation can be simplified by just looking at low-hanging fruit type opportunities to integrate important steps in the Cybersecurity process:

• Users granted permissions to data stores they shouldn’t have access to? Minimize the manual steps – Automate the permissions process by role.
• Employees clicking on too many phishing emails? Implement an email-based phishing exercise and automate mandatory training using a tool such as Microsoft Purview.
• Too many devices, servers, and events to crawl through, and in multiple tools? Save troubleshooting time by integrating a security aggregation tool such as Axsonius which can help pinpoint specific scenarios and the tool/source from where they occurred.

ODMs act as value levers for Cybersecurity investment, helping CIOs balance the need to protect the business with operational requirements.? From a vendor or employee standpoint, helping to identify and then take action on these levers creates a key advisory relationship with leadership.

Question 3: What can we act on immediately?

The data metrics help organize the information, and the ROI may provide a way to prioritize and triage implementation.?Once a decision is made to pivot to an ODM-centric process, the next step is to determine what is achievable based on what is already in place.?The steps with the largest ROI may take time to implement – but identifying, achieving, and championing a small victory can be the first step in that journey of a thousand miles.

That first win will help gain confidence from leadership to go tackle those larger efforts and ultimately protect the organization even more completely.

High risk categories make excellent candidates to tackle first, as they help prioritize real-world needs, and their impact is significant to the organization.

SiloSmashers And Our Federal Customers

SiloSmashers’ work with our federal partners involves a range of duties, not the least of which is oversight of the CIO Cybersecurity practice.?As our customers’ Peak Performance Partners, SiloSmashers took the outcome-driven metrics approach seriously, and deployed multiple tools to monitor and manage the cyber risk and threat landscape effectively.?At SiloSmashers, we understand ISSO resources are always at a premium at any organization. We introduced the use of aggregation tools such as Splunk, Microsoft Purview, and the Axsonius Cybersecurity Asset Management Platform, which can automatically capture event log and audit data. These tools offer views across multiple systems, providing an undiluted “clear pane of glass” view of the entire organization. Simple querying, alerts, and dashboards allow security professionals to quickly view and analyze their outcome-driven metrics.

SiloSmashers’ Splunk Implementations

Splunk is a foundational tool with many of our federal partners. It is a comprehensive platform that correlates machine-generated data – machines such as servers, network devices, and applications. It is used at many federal agencies to track various security metrics by collecting and indexing log data from these diverse sources.

In support of federal customers, SiloSmashers created visual dashboards to monitor the security posture in real-time, detect anomalies, detect unusual login behavior - and use these ‘heads up’ metrics to respond to incidents quickly.?Presenting risk information visually via charts, graphs and even simple incident counts can quickly convey the urgency of a situation without a stakeholder needing to understand all the underlying data.

ISSOs additionally create alerts that transmit emails when specified security circumstances occur, such as a user repeatedly failing a login, or an application exfiltrating data at unusual hours of the day.?These alerts are constantly being checked against the data and are automatically transmitted regardless of whether an administrator is actively reviewing the data or its dashboards.?Alerts can be created against specific scenarios, or against thresholds met – making them perfect for tracking outcome-driven metrics.

SiloSmashers M365 Purview Alignment

In a similar space, Microsoft Purview is a unified data governance solution that helps customers manage and protect data specifically interfacing with the Microsoft 365 environment. It offers capabilities for data discovery, classification, and protection, which are essential for tracking security metrics related to data governance and compliance. Purview assists security officers to identify sensitive data, monitor user access and usage, and ensure compliance with regulatory requirements.?

Purview affords many of the same capabilities as Splunk, and many of our customers are maturing into leveraging this tool alongside Splunk, to maximize the efficiency of both systems. Alerts and dashboards in Microsoft Purview are similarly available and provide similar capabilities to manage and protect data specifically in the M365 environment.?One major differentiator for Purview is the system’s ability to test and manage user responses to phishing/social engineering attempts, and to track outcome-driven metrics around such user responses.? ?

The Future of ODMs and Cybersecurity

Transitioning to utilizing outcome-driven metrics (ODMs) is essential for modern Cybersecurity practices. By focusing on the actual effectiveness of security measures, organizations can better adapt to the dynamic cyber landscape and ensure robust protection across their entire attack area.

ODMs provide a clearer picture of Cybersecurity success by linking security metrics to business outcomes, better enabling decisions based on hard facts in lieu of anecdotal evidence.? Resources can be allocated in a more strategic manner, waste minimized, and Cyber focus can be aligned to the greatest threats. ?Where new tools or processes are needed, ODMs provide the foundation for building a business case supported by the ROI, which can streamline any new proposal.?

?While implementing ODMs may at first appear to present challenges, tools that aggregate data to a single reporting platform can help offset these difficulties; organizations will find they already capture most of the data required for the transformation.?To get started, begin by asking the three key questions.?Those 3 questions can start you down the path to Peak Performance.

About the Author

SiloSmashers’ Rob Doyle is a long-time federal consultant, having worked with agencies as diverse as the Department of Homeland Security (DHS), Defense Information Systems Agency (DISA), Department of Health and Human Services (HHS), as well as a variety of clients in the commercial space.?His work in the information and systems architecture space has provided customers with successful solutions following DevSecOps principles, from strict contract-driven systems development, to platforms providing governance for citizen-developer-driven solutions.

About the Company

SiloSmashers, founded in 1992, is a Woman-Owned Small Business (WOSB), and an industry-leading International Organization for Standardization (ISO)-certified strategy, technology, cyber security, and management consulting firm. We are anchored on the premise that breaking through programmatic silos removes barriers to promote effective program coordination and communication. SiloSmashers holds our employees to a high level of integrity, knowledge, sound management and a client-centric service approach in our pursuit of Peak Performance.