EPSS and gaps in the CISA KEV: Real-world examples

Originally published June 7, 2023

My concerns about CISA’s KEV catalog are growing.

I have previously mentioned my strong doubts the Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerability (KEV) catalog was “authoritative,” as it claims to be. And I was suspicious about the opacity of their methodology.

A major gap, in my mind, was the fact that the Exploit Prediction Scoring System (EPSS) version 3 incorporates exploitation data for 12,243 unique CVEs but the KEV catalog contained only 933 entries as of mid-May 2023.

And now I have more evidence.

Frankly, it appears to me that CISA’s methodology consists - at least in part - of semi-randomly googling vendor security advisories and examining them for reports of exploitation in the wild. And sometimes this happens with a delay of almost two decades.

CISA did not comment on a draft of this article that I sent them last week. Although I know most government officials have tight restrictions on what they can say on-the-record, without any sort of statement from them I can only use the evidence below to craft my analysis. And it’s not a good look for CISA.

This investigation was all triggered by a LinkedIn post purporting to show how using the EPSS might fail to reflect the risk from actively exploited vulnerabilities. And in this article, I’ll break down the logic used here and go deep on all of the vulnerabilities listed.

Bottom line: while not a silver bullet, I remain convinced the EPSS (enhanced with the Deploy Securely Risk Assessment Model [DSRAM] for non-CVEs) remains the single most effective freely-available data source for known vulnerability prioritization. While this approach can miss the mark sometimes, it is the only comprehensive, logical, data-driven, and free system available.

CVE-2004-1464

Suspicious timing

What first got my attention was that CISA just added a nearly 20 year old vulnerability to the KEV. The press release states that this move was “based on evidence of active exploitation.” But as we discussed previously, the term “active exploitation” is very misleading as it. According to CISA’s own documentation, “active exploitation and exploited are synonymous,” meaning that a CVE exploited 19 years ago and one exploited yesterday will be both be on the KEV catalog.

But did CISA just get evidence that this was exploited in 2004? Or do they have recent evidence it was exploited yesterday? Hard to tell from their statement…

…until you check the vendor bulletin.

If the changelog is to be trusted, it looks like Cisco reported this vulnerability being exploited in the wild in August of 2004!

So CISA waited 19 years after a public announcement, and then “re-announced” the “active exploitation” of the vulnerability.

Why?

This does not make any sense to me, at all. And unless there is an explanation I am missing here, this example strongly undermines the suggestion that the mere fact of a CVE being added to the CISA KEV should change your security posture at all.

EPSS evaluation

If you wanted to take an approach that mitigates substantially more risk (all other things being equal) than a CVSS 7+ strategy but still only remediate ~14% of total CVE findings, you could use a threshold EPSS rating of 0.0176 (which is 1/5 of the threshold suggested in the most recent EPSS paper).

Assuming one multiplied the EPSS by 100, as did the author of this post, the result would be 1.76.

Thus, this issue would be above the threshold and you should remediate it.

My hypothesis for why this EPSS rating is relatively low is that there are probably very few, if any, instances of the vulnerable software in production anywhere in the world. It’s hard for attackers to exploit code that doesn’t exist, and I highly doubt there are many internet-facing instances of this software running.

So even if the EPSS rating were below the threshold, it would be something of a moot point.

CVE-2016-6415

A seven year delay

While I wasn’t able to locate any archived page history for the 2004 vulnerability (the bulletin seems to have moved to a newer URL) I was able to find one for this issue.

If you review the vendor advisory, first published on September 16, 2016, you will see the following text:

“Cisco Product Security Incident Response Team (PSIRT) is aware of exploitation of the vulnerability for some Cisco customers who are running the affected platforms.”

So we know for a fact that Cisco publicly stated this vulnerability was exploited in the wild seven years ago, but without any sort of explanation, CISA added this to the KEV catalog in May 2023.

Again, why?

I cannot think of any explanation except for the fact that CISA has no standardized process for incorporating new issues into the KEV catalog.

Building some sort of web scraping tool that looks through all of the Cisco advisories and identified mentions of active exploitation using GPT-4 or similar AI model would be quite easy to do, and would yield far more comprehensive results than appear to be present in the KEV. Better yet, CISA could just ask the vendor for a structured database of all know exploited issues, for which the company already has data.

It doesn’t appear to have done either of these things.

EPSS evaluation

Despite being an older CVE, it still seems like a popular one!

And this is clearly a “remediate” issue based on the EPSS of 0.973.

CVE-2023-21492

This issue is very interesting to me. The fact it is recently published, known to be exploited but has a low EPSS score (below the threshold I describe above) is intriguing.

So I did some digging.

CVE-2023-21492 requires local access to exploit. So right off the bat, this would seem to require either vulnerability chaining (if attacking remotely) or physical access to the device.

Achieving either of these things requires substantial know-how or resources, suggesting a highly capable attacker. After reviewing some press reporting, I noted that “the Samsung security vulnerability was discovered by Clement Lecigne of the Google Threat Analysis Group (TAG), indicating likely abuse in connection with a spyware campaign.”

If you do some research on Clement’s work, it looks like he collaborates with groups like Amnesty International. And he writes about spyware vendors and the exploits they sell.

So I think it’s a fair assumption to say that CVE-2023-21492 is specifically targeted by well-resourced threat actors with non-financial objectives (human rights activists don’t generally have a lot of money but often have powerful enemies). These groups - often advanced persistent threat (APT) actors - are extremely dangerous.

With that said, they have very discrete goals. And the vast majority of enterprises don’t need to worry about APTs targeting human rights activists for political, rather than financial, reasons.

That’s why threat modeling is so important. If you think malicious actors will target individual mobile devices, moving laterally to chain vulnerabilities together, then absolutely you should focus on these types of issues.

But that is a small subset of organizations.

I’ll also note that since this CVE impacts a mobile device, and none of the EPSS data partners (to my knowledge) collect data from mobile devices, there is highly unlikely to be any exploitation attempts factored into the score. Combined with the fact that this requires local access, it’s score is relatively low and below the threshold.

A potential miss for EPSS here (and the Common Vulnerability Scoring System, as well, considering this is scored at 4.4/10, which is “medium” severity), but based on the above detail, not a huge one.

CVE-2023-28204

This issue does not have an EPSS score as of May 30th, but does have a vector string available. Although who found it is not 100% clear, it was discovered alongside similar vulnerabilities attributed to our friend Clement. Likely exploited as a zero-day vulnerability, its presence on mobile (iOS and iPadOS) suggests a similar threat picture as the previous issue.

In any case, its exploitability according to the latest DSRAM data is 0.0177 (network attack vector, no privileges required, user interaction required).

Thus, this would be a “remediate” issue using our threshold.

CVE-2023-32373

As of May 30, 2023 there is no vector string available. Probably another discovery from Clement.

Since the only thing we know about its exploitability is that this has been publicly declared exploited in the wild, we cannot do any quantitative analysis.

So this is a potential miss for EPSS.

A potential option for organizations would be to assign an arbitrary exploitability score (e.g. 0.90) for any known exploited issues (whether mentioned in the KEV catalog or a vendor advisory) that don’t have an EPSS rating.

CVE-2023-32409

No vector string available as of May 30, 2023. Confirmed discovery by Clement.

Identical analysis as per the previous vulnerability.

CVE-2023-2868

This final issue is quite interesting. CVE-2023-2868 has been exploited as early as October 2022, per Barracuda Networks’ advisory. But it was only reported in May of 2023. Here is abbreviated timeline from Barracuda:

• May 18, 2023, Barracuda learned of anomalous traffic originating from Barracuda Email Security Gateway (ESG) appliances.
• May 18, 2023, Barracuda engaged Mandiant to investigate.
• May 19, 2023, Barracuda identified CVE-2023-2868 in ESG appliances.
• May 20, 2023, Barracuda applied a patch for CVE-2023-2868 to all ESG appliances worldwide.

Based on the above, what happened was that the vendor:

• identified anomalous activity;
• found the vulnerability (which had been exploited as a zero-day); and
• pushed a patch to fix the issue globally (before announcing it and before any other attackers could exploit it)

If my reading of the advisory is correct, it seems like the only way to remain vulnerable to this issue would be to somehow block updates to your ESG appliance (a very odd thing to do).

Its EPSS rating is 0.00645. While in the 76th percentile, this would be a non-remediation issue based on our criteria (assuming you could make such a decision, which it appears you cannot).

Although this looks like a possible EPSS miss, the low score makes complete sense. This is a known exploited issue that also has very low exploitability.

That’s because there have probably been zero vulnerable instances of this product in the wild since May 20th. So there is nothing left to exploit (and no observations by EPSS data partners).

If fixing this had required customer action (applying a patch to an instance they manage themselves), you would likely have seen substantial exploitation activity (because some customers would have failed to do this) and thus a high EPSS rating.

June 8 update: immediately before the release of this post, Barracuda changed positions on this issue. It appears that there does not exist any known software-based method of closing this vulnerability and Barracuda is advising customers to physically replace their ESG devices. Additionally, the EPSS score as of June 8th was 0.01638. This is below the threshold described above and looks like a miss for EPSS due to ongoing active exploitation. With that said, it’s not clearly so. The reason is that there is some informed speculation that an APT is responsible for this exploitation and has a very narrow set of targets. And it’s not clear whether the recommendation to physically replace the appliances is a) based on an abundance of caution due to the fact that Mandiant found the original threat actor present patch was issued but the patch did close the initial attack vector or b) the patch was wholly ineffective and threat actors continue to exploit CVE-2023-2868. The former would not necessarily mean EPSS is off target, but the latter would.

Conclusion

So, based on all of this analysis, what should you do?

• Develop threat models that help identify likely attacker identities and techniques, tactics, and procedures.
• Conduct penetration testing and static analysis to identify previously unknown vulnerabilities.
• Use the EPSS (enhanced by the DSRAM) as your primary source of known vulnerability threat intelligence.
• Supplement it by following vendor advisories (for products you use) closely for information about exploitation in the wild. This may even give you evidence of exploitation before the relevant CVE or other known vulnerability is publicly available. And, as I have suggested in the past, mandate this type of reporting in your contracts.
• Do not rely on the CISA KEV as a way to learn about such exploitations in a timely manner, but rather use it as a secondary source of information.

While having a consolidated list of known CVE exploitations is a good idea, CISA’s execution and communication leaves much to be desired here. I have encountered many security professionals that take the KEV catalog’s self-description of being the “authoritative source” of vulnerabilities undergoing “active exploitation” at face value.

They have unfortunately been badly mislead.