It's Not about Money. It's about Quality

For those who have read my other articles, this may seem like “just another rant”. I promise you, I’m not an angry kid. I just get very worked up about the things that are a passion for me. In this case, I’m passionate about companies wasting money for “so called” penetration tests that provide no value. As a matter of fact, in the example that I’m outlining here it actually will cost the company MORE money, than just the cost of the penetration test and they will (arguably) be no more secure than before the test.

The bottom line up front (BLUF) to this rant: I will literally give away FREE penetration testing consultation and/or service execution to any company that feels they’ve gotten a bad deal. Contact me and we will work out the details.

Let’s begin. 

This particular company’s story begins like many. There is an ask for an evaluation of security through penetration testing activities from a reputable third-party. I can value this stance from senior leadership. It goes a long way when dealing with Cyber Insurance underwriters, external auditors, or (hopefully never) the public when there is a security event. The CISO at this particular organization did what most CISO’s do: they went out to the street to find a vendor who could provide the test, asked for Statements of Work, and then picked a vendor. There is nothing out of the ordinary about any of this, and frankly, it’s how you would expect an organization to search for a reputable vendor.

As a matter of fact, we have such a saturation in the market space of vendors who have chosen to contribute their talent to the professional services arena, it has become nearly impossible for organizations to decide what is a good deal on cybersecurity services and what isn’t. Penetration testing has gotten to be no different. There are so many brands out there (large and small) that simply utilize regurgitated language “borrowed” from one consulting company, only to be reused to promote their own “flavor” of penetration testing standard. There is an excellent website out there called “The Penetration Testing Execution Standard”, however it hasn’t been updated since 2012 and its “Reporting” section specifically states “Contribution Needed”. This, undoubtedly, drives leaders to select companies based on peer review, word of mouth, or just which of their vendor contacts they are closer with.

As for this CISO, we can assume that they made a selection based on the later. In this case, the vendor chosen (which will remain nameless for this article) was a very large, multi-billion dollar, company that is easily within the Fortune 100 list. This is an organization that has a consulting arm that claims specialties in many facets or cybersecurity consulting, IT, and other high profile projects. Their services command a very large fee and as such the resultant deliverable is expected to be of the highest caliber. For the sake of this hypothetical conversation we’ll say that this CISO paid “a couple hundred thousand” for this penetration test. Read that again - “a couple hundred thousand”. The expectations for this test should be growing exponentially now.

Fast forward toward the end of the engagement. Due to the nature of this particular hypothetical company, the potential risks involved with this test warrant the need of an outside council review of the engagement and the results to attempt to shield the company for any potential legal liability. As such, there is an entire legal team assigned to interface with the vendor, receive the results (ahead of the client) and begin the work of sanitizing them into a risk appropriate language. Several weeks later, the company is presented with a comprehensive report that has been stratified and prioritized by the outside council.

After a much required review with all of the senior leaders of the organization, it is decided to set up a Tiger Team that can move forward with remediation efforts. Now imagine, upon your review of the prioritized legal findings you discover that one of the top 10 issues discovered was this:

• Title: Outdated and Vulnerable HTTP Web Server Versions: High-Risk Vulnerabilities
• Description: This system is using Apache HTTP version 0.8.15, an unsupported version of HTTP web server. Apache HTTP <1.3 has been unsupported by Apache Software Foundation as of February 3, 2010.
• Rating: High (9/10)
• CVSS: 10 (however, no CVE’s where given)

At this point, there will be three types of people who read this article, read the above, and immediately think to themselves:

1. Oh My GOSH! They have EOL systems that were discovered as part of the Pentest! This is the greatest pentest in the world! That company should burn as they are still using EOL stuff! Patch/upgrade/Cybersecurity For The Win!
2. Oh…wow…I thought you said this was a pentest?
3. I have no idea what the big deal is, please educate me.

If you are Group 1, this article isn’t for you. If you are Group 2, this article isn’t for you but hopefully you see that we have got to make a change together. 

If you are in Group 3, this article is for you, and I would love to help you see why this is a big deal. In our totally hypothetical scenario above, this company has paid a crazy amount of money to a supposed professional services company in exchange for what it anticipated would be a “penetration test”. Then they paid additional money for the legal team to review it, sanitize it, and get it ready for consumption. Finally they will pay a crazy amount of money to reach into their organization, form a Tiger Team to research, investigate, and remediate this VULNERABILITY.

1. This VULNERABILITY could have been discovered with an established Threat and Vulnerability Management Program. Just to demonstrate how simple it would have been to discover this: you could have hired a college Intern, given him a free version of OpenVAS (available in Kali Linux) and told the intern to scan the environment cheaper than the entire engagement. The report would have produced the same exact result, and you would have grown talent organically.
2. This organization didn’t have externally facing web applications running Apache. All their web servers were (theoretically) running Windows IIS. So this is a false positive.
3. There is no publicly available exploit code (based on my research) for Apache version 0.8.15. This notice is just to tell you that it’s no longer supported.

Now there will always be Cybersecurity “purist” out there that tell you that “it doesn’t matter” and because its an EOL system you must forsake all business practices and upgrade. Again - this article isn’t arguing that point.

This article is highlighting, yet again, the problem we have in this industry that we allow organizations like this vendor to exist, and that we have companies (like this hypothetical company) that unknowingly purchase crap services.

The very small soap box that I will get on is summed up like this:

Penetration testing is supposed to be defined as “testing a system the same way a hacker would”. A (good) hacker would never run Nessus/Nexpose/OpenVAS against an external network range. Furthermore, by including your vulnerability scan results into a pentest report, you are clearly demonstrating that you felt the need to include SOMETHING to attempt to smoke screen your client that you provided some value. Either because you are afraid that “nothing” makes you look incompetent or that your service doesn’t provide value. If you are a penetration tester, and you feel that you have done a client justice by running a vulnerability scanner against their environment, not validated any of the results for false positives or demonstrated exploitation, than you are the problem and I kindly ask that you talk to me or your peers and lets try and raise your game a little bit (I will not tell you to “try harder”).

If you are a company and you believe you have been a victim of less than comprehensive penetration test, I will review your results (with the appropriate NDA’s in place) and provide you with a FREE consultation of the review.

If you are a company and you want a FREE penetration test done, arguably, the right way (in my eyes), contact me and we will work out a Statement of Work and I’ll be more than happy to execute.