Decoding cybersecurity compliance: Strategies for auditing vague regulatory standards
As more cybersecurity regulations are introduced, the challenge of ensuring organizational compliance increases, as do the responsibilities of internal auditors. How can auditors ensure that organisations comply with laws ranging from flexible, principle-based guidelines to highly prescriptive technical requirements?
I recently watched a very insightful lecture by James Dempsey from Stanford's Cyber Policy Center titled “Cybersecurity Enforcement: If Government Regulation Is Going to Increase, How Will It Be Enforced?” (I highly recommend checking it out— the really interesting stuff begins at the 14-minute mark).
In his lecture and in a series of articles, Dempsey explores four different approaches to cybersecurity regulation and the complexities of enforcing these rules.?
This prompted me to think about how auditors can navigate the enforcement and auditing of different regulatory types, particularly those that deal with the goal-based or principle-based requirements. In this article, I’ll try to summarise some of Dempsey’s key points and offer my own thoughts on auditing these types of cybersecurity regulations.
Different types of regulatory frameworks
One of the four forms, or approaches, of regulatory system is mandatory information disclosure. For cyber security this most often takes the form of Data Breach Notification Laws and mandatory incident reporting requirements as seen in the European NIS regulation. However, I will set this aspect aside for now, as these measures apply ex post (after an event occurs). Instead, I’ll focus on the other three ex ante approaches, which are much more interesting.
Besides information disclosure, Dempsey identifies three distinct core regulatory types; means-based, performance-based and management-based. Regulatory requirements don’t always come well defined into these categories, so I have created three Venn diagrams to highlight and describe the three different approaches.
What Dempsey refers to as 'management requirements' are often called process-based or procedural requirements. The intent of these requirements is to compel an organisation to implement specific managerial processes. A common example is the requirement to perform risk analysis. By enforcing these processes, the expectation is that the organisation will independently identify and establish appropriate protection mechanisms.
Performance based requirements are very closely related to goal based. Their intent is to set goals or targets that an organisation must strive to achieve, but they do not specify how to do so. Principles-based requirements, while somewhat different, serve a similar purpose, namely, to avoid establishing prescriptive rules for how to achieve the intended outcome and thereby allowing for innovation within regulated organisations. A clear example of a requirement in this category is the GDPR requirement to ensure “appropriate security of the personal data, including protection against unauthorised or unlawful processing”.
The third distinct category is rule-based requirements, these describe in detail how to attain the intended outcome, often specifying particular means. An example is the PCI DSS requirements, which outline in great detail the bit lengths that constitute Strong Cryptography. Overly prescriptive rules can feel restrictive for organisations seeking to innovate or develop new means and methods. ?
At the end of his article-series, Dempsey’s concludes that regardless of the form, any regulatory system is only as effective as its enforcement. He particularly highlights the the challenges enforcing performance-based regulations in cybersecurity, as well as the difficulties associated with management-based regulations.
The importance of enforcement
Dempsey provides a great reference to emphasise his crucial point that a regulatory framework is only as effective as its enforcement. He cites the work of Boston University law professor Rory Van Loo and points out that outside of cybersecurity, monitoring or inspection is the most common means to enforce regulatory systems. As a specific example Dempsey refers to a truly fascinating study from 2019 by the US Federal Reserve Board that describes what happens to banks when the regulatory agency suddenly loses most of the staff responsible for supervising savings and loan associations. This study concludes, perhaps unsurprisingly, that reduced supervision impacted the “financial institutions’ willingness to take risk. The additional risk took the form of more risky loans, faster asset growth, and a greater reliance on low quality capital. This response to less supervision boosted banks’ odds of failure.”
How to enforce requirements
In his excellent lecture Dempsey also outlines six methods of enforcing regulatory standards. These include two ex-post methods; Private litigation and Case-by-case regulatory enforcement. And four ex-ante methods; Self-certification, Third-party audit/inspection, Government review of plans and Government supervision/inspection/monitoring. Dempsey then describes with great clarity the pros and cons of the different methods. Rather than delving into the specifics of each approach Ill rather just conclude that Dempsey only provides very high-level descriptions of different strategies for enforcement. ?
What I felt was missing, and wasn’t within the scope of the lecture, was a more practical, hands-on approach for conducting the supervision/inspection/monitoring (or audits if you will) of those vague high-level, principle-based, management-based, or process-based requirements. ?
This brings us to the core issue—how do we effectively audit organisations against flexible and often ambiguous regulatory standards? Just to be clear, the rest of this article is my own thoughts and ideas, and Dempsey is entirely innocent of any follies or errors here.
领英推荐
Auditing the different approaches
Now we have established that there are at least three distinct approaches to regulatory requirements each with its own set of pros and cons, as detailed in Dempsey's lecture. In the following sections, I’ll try to outline a practical approach to auditing these types of regulations.
I would however want to argue that it’s not really three different approaches, it’s more of three different points on a scale/spectrum. Even at its loftiest level, the principle-based requirements, the requirement is just an instrument for a desired outcome, and that outcome is in most cases a cyber security control, be it technical or administrative, the intent of the requirement is to (if applicable) make the organisation implement a control mechanism.
I realise that what I have written above is a bit hard to comprehend, so I asked chatGPT for a clearer version and it actually came out quite succinct: Even with principle-based requirements, the goal is to implement a control mechanism, whether technical or administrative, to achieve the desired outcome.
This is a process, and the process will, or at least should, generate security controls. And when it comes to auditing security controls there are well established processes for that, with hands on testing and sampling being one of the most efficient. Again, I would like to point to PCI DSS for examples of testing-procedures used to verify the implementation of technical controls. Auditing of security controls is a solved problem, it’s the two other areas that are hard to audit.
They’re really just points on a spectrum
As I tried to establish above the intent of process-based requirements for instance is to force the organisation to perform certain managerial processes with the intent that the organisation will by itself identify and establish appropriate goals and security principles. The most common being the requirement to perform yearly risk analyses and remediation plans.?
Once goals, principles, and performance targets has been identified, either because they are spelled out in the regulatory requirements or because the organisation has self-established those goals, they can be interpreted into very specific means-based rules tailored very precisely for the organisation or even specific departments within the organisation.
This can be visualised as a scale, with broader, more flexible principles on the left and highly detailed rules on the right. The goal of the requirements on the left is always for the organisations to develop their own specific rules.
When the regulatory requirements are vague and open-ended, such as is the case with GDPR, that puts all the onus on the organisation to fill out all the blanks on the right side of the scale.
How to audit Ambiguous and Unclear requirements
In order to audit organisational compliance with management requirements, goal-based requirements or any other type of “left side” requirements we must do two things;
The first step in auditing compliance is to understand how the organisation translates high-level, flexible principles (the "left side" of the scale) into goals, and further into specific detailed actions and rules (the "right side" of the scale). The first question of each regulation is basically “what <organisational rules> have you created to reach this goal?”. But before we continue, a word of advice. As an auditor, there is a clear pitfall to watch out for: auditing compliance with flexible, principle-based requirements requires a deep understanding of both the high-level goals and the detailed technical measures created by the organisation to fulfil them. The auditor must have the prerequisite skills to fairly assess whether a security control meets the intent of the requirement.
Create a map: Examine the process that the organisation uses and identify the steps the organisation takes to break down broad, goal-based requirements into actionable rules and technical controls. This may sound hard to do, but in practise its often quite simple, those organisations that actually have this in place have no problem explaining it. Those organisations that are unable to be specific in this area are simply non-compliant.
Validate the decision-making process:
Examine who within the organisation makes the decisions on which controls or rules to apply to meet those broad principles. If the decision making for high level "left side" flexible principles are left with developers or individual project members instead of organisational stakeholders, the organisation will struggle to comply. Look for a clear traceable path from high-level requirements to very specific enforceable rules and guidelines. Organisations that have successfully implemented this process can easily explain how they do it, the challenge lies in the execution, not the documentation. If an organisation fails to even document its work, you can be certain the task hasn’t been properly carried out.?
The second step is more straightforward, as it mostly involves traditional auditing of security controls. The key is to avoid applying flexible principles (the “left side”) directly to specific systems. However, this is a common request. You might be asked to audit a specific system, or even a vendor's product, for GDPR compliance - something that falls far to the left on the scale we discussed earlier. Just say no, it’s an organisation that complies with GDPR not a system or product.
In contrast, frameworks like PCI DSS focus more on technical controls, which are much easier to apply directly to systems or products. PCI DSS provides detailed, prescriptive requirements, placing it on the “right-side” of our scale.
I could be wrong …
Im a stronger believer in the adage that writing is thinking, which is one of the reasons I’ve written this article. I genuinely welcome any feedback or differing opinions on the points discussed. If you have a different perspective or additional insights on auditing cybersecurity regulations, please feel free to share your thoughts in the comments. Your input is greatly appreciated!