Making Sense of the MITRE ENGENUITY ATT&CK Evaluations

This is the first year that the evaluations tested a vendor’s ability to offer protection from the attacks. The focus of previous years has been detection. Can you detect the type of attack this APT uses? It’s my opinion that it’s better to block the attack than to detect it and alert on it. It's like placing video cameras all over your house, but not locking the door.

There were 10 tests.

Notable Market Leaders

Crowdstrike missed 3

Sentinel One missed 1

Blackberry/Cylance missed 4 (Machine crashed four times)

FortiEDR missed 0

The big question is: How did the first gen AV vendors block all of these advanced attacks? I have a theory. This is my own opinion and it does not represent my employer.

First Generation AV Vendors are very binary. If the hash is in the database, don’t let it through. They also have the ability to block all scripting. Given that the 10 attacks MITRE used are very script focused, they likely blocked them all. If this is accurate, how many enterprises actually turn off all scripting in their environment? How realistic is this configuration?

How is FortiEDR different regarding scripting attacks?

Unlike other vendors, we don’t block all scripts or just detect the script being run. We review the script’s behavior and we block it post-infection right before it’s allowed to write-to-disk or get a network socket. Example: If the script tries to steal credentials, we will recognize the bad behavior and block it.

Why am I excited & why am I writing my first LinkedIn article?

I’ve been screaming from the mountaintop for three years how this solution’s approach is the most effective. Being validated by MITRE is a great feeling.