The Illusion of Security—Why Passing an Audit Doesn't Mean You're Safe

I'll start this article by saying that I'm coming to the realization of why Stephen King made the observation: "To write is human, to edit is divine." I will add that "to rely on AI to do either for you is foolish at best."

As I was gearing up to write this article, I looked over the introductory article and saw so many things I should have presented differently. But this is a journey, and I'm grateful you let me take you on it.

Disclaimer - I love audits!

Audits and auditors can be CISO's best partners. First, they can find blind spots that the CISO has in their current program. Next, with their findings, the auditors' findings can serve as an independent verification and leverage with the Executive Team and the Board in justifying investments and prioritization of initiatives. That said, people often confuse compliance with security. This article will reference the SOC 2 and ISO 27001 standards. These are the two most common standards we deal with, but they are not the only ones with flaws.

Introduction - Making Cybersecurity Concepts Accessible

Over the years, I've found that one of the best ways to explain cybersecurity concepts is through skeuomorphic examples—real-world analogies that make complex ideas more relatable. When it comes to illustrating the dangers of blind compliance, my favorite go-to analogy is the behind-the-scenes story of the RMS Titanic disaster. I use it for many complex cybersecurity topics, and I'll have to stop myself from digressing into unrelated topics.

But, before I delve into a rather technical and dry topic, indulge me as I introduce the story of the Titanic. I promise it'll all make sense soon.

The Titanic Exceeded Compliance With Maritime Regulations

On April 15, 1912, the RMS Titanic, one of the most advanced ships of its time, struck an iceberg and sank. Over 1,500 people lost their lives—not because the ship lacked technology, but because of a fatal reliance on outdated compliance standards.

It is easy to misattribute the primary cause for the loss of so many lives (other than the big hole and cracks in the hull after the boat struck the iceberg) to a simplistic excuse of an insufficient number of lifeboats. It was one primary contributing factor; there were many others. While the RMS Titanic only carried 20 lifeboats. That said, the UK Merchant Shipping Act of 1894 and 1906 regulations, which defined requirements for ships 10,000 tonnes and over, specified a mandatory requirement of 16 lifeboats. The Titanic, launched in 1912, over six years after the most recent update to the Maritime Regulations, was over 40,000 tonnes. So, the RMS Titanic exceeded its compliance requirement by having four more lifeboats than specified by the outdated regulations. Even if every lifeboat had been fully occupied (which they weren't), the ship still wouldn't have been able to accommodate all passengers and crew.

The RMS Titanic was fully compliant. And yet, compliance didn't save the passengers when disaster struck.

Why Does This Matter to Our Topic?

I will claim that today's cybersecurity audit standards share the same deficiencies as the UK Maritime Shipping Act of 1894 and 1906. They are outdated, deficient, rigid, and often used incorrectly.

Now, let's switch back to talking cybersecurity.

What is Wrong with Today's Cybersecurity Audits?

In this section, I will discuss what I see as wrong with audits and why we cannot fall into the false sense of security that comes with passing an audit. In the next section, I will offer some possible ways for us, as an ecosystem, to address these issues.

We can divide the deficiencies into two broad categories:

• Systematic Deficiencies: These relate to the structural or procedural limitations of the audits themselves
• Operational Deficiencies: These are deficiencies in how companies engage, conduct, and use audits.

Other than the nature of the deficiencies, the main difference is that the systematic deficiencies are common across all audit standards. The operational deficiencies can vary for each company and auditor involved. In this section, we'll explore both categories at a high level. Unfortunately, to keep the length of this article manageable, we won't be able to explore each of the deficiencies in depth.

I welcome a face-to-face conversation over your adult beverage of choice to do that deeper dive.

Current Systematic Deficiencies

As noted above, all audits today share these fundamental problems. These deficiencies are not necessarily intentional; they tend to be inherent features of the audit process.

• Most Cybersecurity Audit Standards are Outdated. Regulatory frameworks like ISO 27001 and SOC 2 provide a solid foundation for cybersecurity but fail to keep pace with rapidly evolving threats and technologies. Their static nature means organizations rely solely on compliance risk and operate with outdated security models.
• Cater to the Lowest Common Denominator. Most frameworks tend to have open-ended and high-level requirements, possibly to be accessible to more organizations. This one-size-fits-all approach misses unique attributes critical for different organizations.
• Audits are Backward Looking. As is the nature of most audits, they will tell you how well you followed the framework over the past year (or however long the audit period is). To paraphrase the popular investment axiom: "Past performance is not an indicator of future security."
• Audits Only Provide a Snapshot. Even though the audit period can be for a whole year, the auditors typically take samples from the audited data. These are discrete elements and may not represent the organization's security posture due to different types of sampling errors. A simple way to think about it is to compare this to stop-motion animation. While the set of discrete images may join together to create a moving picture of the security program, if not sampled correctly, that picture can be choppy and incomplete.

Current Operational Deficiencies

Unlike Systematic Deficiencies, these deficiencies tend to vary between different audit standards. Their impact can also vary significantly based on the auditor and the organization they are auditing.

• Who Watches the Watchers? Taken from the Latin phrase "Quis custodiet ipsos custodes," this deficiency concerns the varying degrees of requirements from auditors. For example, ISO 27001 auditors must pass the ISO 27001 Lead Auditor Certification Exam, while the SOC 2 auditor must be a Certified CPA. We will discuss the challenge of personal certifications in a later article. These limited requirements lead to a wide variation in auditors relative to experience and skills. You can have an auditor with less than one year of experience and an auditor with more experience than the CISO they're auditing.
• Relaxed Audit Procedures and Standards create variations in Audit Results. While I am not an auditor myself (and I do not play one on TV), I have observed, in my years of doing security audits, that different auditors take very different approaches to performing the audit and delivering the results. Two auditors from two different firms will likely result in two different results.
• The Motivation Behind the Audit Varies. Some companies merely view audits as marketing and sales tools. They allow companies to proclaim their "Commitment to Security" and satisfy basic third-party risk requirements. More mature companies with seasoned CISOs view the audit as an opportunity to identify blindspots, test assumptions, and improve the overall state of their security programs.
• Companies and CISOs Can Choose Their Auditors. No objective standard or guidance is dictating which audit company they should use. This fact, combined with the factors above, allows Audits to go from being a "Rubber Stamp" to a thorough "Detailed Examination."
• Audits Tend to Have a Limited Scope and Information Asymmetry. I like to refer to the parable of the Blind Men and an Elephant. The parable illustrates how limited perspectives can lead people to misunderstand the truth, as each person perceives only a fragment of a larger reality. The same is true of auditors and companies. The company sees the entire elephant, whereas the auditors only get to observe the part of the elephant the company chooses to expose to them.
• Companies Tend to Misuse and Overrely on Audits. Many organizations rely on these certifications as proof that their vendors are secure, including for an audit. But if our certification doesn't necessarily mean we're secure, why would we assume that our vendors' certifications are any different?

So, What Can We Do to Fix This?

When introducing myself and my leadership approach to new organizations, I tell them that I will never tell you about a challenge and leave it at that. I will always seek to offer possible solutions. Doing the former is akin to whining; doing the latter demonstrates my commitment to solving and actively working to solve these challenges. In this section, I propose some possible solutions to address the deficiencies I've indicated.

Addressing the Systematic Deficiencies

If audits alone are insufficient, what can organizations do to improve security? A better approach is to shift from compliance-driven security to security-driven compliance—where compliance is an outcome of a well-executed security program rather than the goal itself.

Here are some strategies to make security assessments more meaningful:

• Develop a Flexible Certification Framework – Instead of rigid, slow-moving compliance standards, organizations should advocate for frameworks that can be updated in months, not years, to keep pace with technological advancements, emerging threats, and evolving attack techniques. By allowing frequent updates and adaptability, security frameworks can remain relevant and provide meaningful risk reduction rather than outdated compliance.
• Build a Modular Audit Structure. Rather than going with a one-size-fits-all, build a modular structure that allows companies and auditors to create a Custom-Fit Audit, allowing the compliance test to fit the company and not vice versa.
• Adopt a Continuous Assurance Model. Rather than relying on annual or periodic audits, companies should implement real-time security monitoring and automated control testing. This model enables identifying security gaps as they emerge, reducing the reliance on a static, backward-looking assessment. A strong example is the FedRAMP Continuous Monitoring (ConMon) process, which requires ongoing security monitoring, vulnerability scanning, and control effectiveness reviews rather than a one-time audit snapshot. This approach narrows the lookback period to a month rather than a year.
• Measure What Matters. Security should be risk-driven, not compliance-driven. Organizations must establish real-world security metrics focusing on attack resistance, incident response effectiveness, and overall resilience rather than simply confirming that predefined controls exist.

Addressing the Operational Deficiencies

Dealing with operational deficiencies may prove more challenging. This challenge is because it requires removing the Incentive Structures that are at the root of some of our current weaknesses.

• Audit the Auditors. Develop a more rigorous international standard to certify auditors. This standard should accommodate the proposed changes so that auditors must recertify with every update to the audit framework.
• Make the audit process prescriptive and repeatable. The auditor's performance of the audit should be independent of their background, experience level, or skill set. The approach proposed above should address this challenge. Two different auditors from two companies should be able to arrive at the same audit result.
• Randomly Assign Auditors. Create a database of Auditors with the relevant modules they are certified to audit. When a company requests an audit, the certifying authority randomly assigns an auditor. This auditor can perform two consecutive audits for the company and then rotate out. This limitation will prevent the auditor from developing blind spots for the company they are auditing.
• Enhance Transparency in Audits – Many audits rely on carefully curated documentation rather than real security operations. Encouraging auditors to dig deeper, assess real security incidents, and engage with security teams directly can result in more meaningful evaluations that reflect actual security postures.
• Create an Audit Exchange Database. Rather than rely on proprietary subscription services, companies should be able to register their audit results. Like public financial reports, these audit results must be available for anyone to see up and down the supply chain. To prevent abuse or misuse, companies can require approval to release their results to a specific company.

Final Thoughts

It's time to move beyond checkbox security and build real, adaptive security programs. Compliance alone won't protect an organization—but a well-integrated security strategy will. The challenge for every security leader is to look beyond the audit and focus on actual risk reduction.

Security isn't about passing an exam but staying resilient in an ever-changing threat landscape.

The Hard Truth: Compliance ≠ Security

I'll repeat what I said at the start. Regulatory frameworks and audits aren't bad—but they're not enough in their current form.

The real question isn't "Did we pass the audit?" It's "How do we ensure we're secure?" and "How can we prove it to others?"

That's the conversation every security leader should be driving.

If you made it to this final section, thank you for your persistence. I would also love a comment telling me what you thought of this rather long and dense article.