Improving The Efficiency Of Vulnerability Remediation Strategy

The research paper presented at the Workshop on the Economics of Information Security in Boston two months ago piqued my interest as it describes a machine-learning (ML) based prediction model that claims to improve the efficiency of a vulnerability remediation strategy multi-fold by predicting vulnerabilities which are likely to be actually exploited.

It does come across as a better alternative to the commonly observed feature-based strategies, such as the one driven by published exploits or CVSS score, which are largely influenced by some regulations that require patching CVSS 9+ within 15 days, CVSS 7+ within 30 days and PCI-DSS that requires merchants to remediate CVSS4+.

The ML-based prediction model is evaluated along the following four dimensions and claims to achieve comparable level of coverage (with published exploits or CVSS scoring strategy) with just one-quarter of the effort.

• Coverage – e.g. if 100 vulnerabilities are being exploited and yet only 15 are remediated, the coverage is 15%
• Efficiency – e.g. if we remediate 100 vulnerabilities, but only 15 are ever exploited, efficiency is 15%
• Accuracy – measuring correctness of predictions, mathematically, the sum of true positives and negatives divided by the sum of samples
• Level of Effort – actual number of vulnerabilities to be patched to satisfy the strategy (performance defined by coverage and efficiency)

Using aggregated data from a variety of sources (more than 100,000 corporate networks) along with MITRE’s CVEs from 2008 to 2019 and real-world information about prevalence and exploitability of each vulnerability, including evidence from SANS ISC and others, the researchers make a distinction between published exploits (about 13% of all vulnerabilities) vs exploits in the wild (about 5%), while observing that only about half of all exploited vulnerabilities have associated published code. Their model uplifts the prediction accuracy, which in turn improves the efficiency with their claim that an organization seeking broad coverage of vulnerabilities that are exploited in the wild (e.g. 70%) would need to remediate only about a quarter of vulnerabilities, usually identified with other approaches.

Regardless, exploits in the wild as the preferred outcome measure seems right, if we assume that very few vulnerabilities are actually a focus for attackers in the real world. Their model helps identify and prioritize vulnerabilities which are likely to be actually exploited, and as pointed out, given that most organizations will always have more exposed vulnerabilities than resources to fix them, exploring this model further seems worth pursuing.

Of course, the inferences drawn seem limited by a number of factors - the exploits observed by their signature-based IDS/IPS (and hence no zero-day vulnerabilities in scope), their model possibly trained to ignore vulnerabilities “not yet” exploited and their measure of risk relates to only threat (likelihood that a vulnerability will be exploited), ignoring the security controls, value of assets or the actual impact that a successful exploit may cause. 

While there is merit in fine tuning this model in the context of our individual organisations, my view is that we should also continue to focus on preventing the following seven types of coding mistakes found consistently over the last two decades, which I highlighted last year, emphasizing on quality as the backbone driving cybersecurity strategies.

1. Failing to validate (sanitize / encode / escape / parameterize) inputs
2. Inadequate or improper security configuration
3. Lack of improper authentication
4. Broken authorization (entitlements and roles)
5. Lack of or insecure encryption
6. Insufficient logging and monitoring
7. Improper error handling

CC-BY Viren Mantri, 2019, licensed under a Creative Commons Attribution 4.0 International License.

Disclaimer: All views expressed here are entirely mine.