Visiting McNamara's Fallacy and Folly

Talking about a pivot - I was about one thing on data/evidence fallacies with things security/resilience, and in looking for an image, came across the McNamara Fallacy.

I think that is a better topic.

Robert McNamara was the United States Secretary of Defense from 1961-1968, which covered the start and a large part of the Vietnam War. McNamara insisted that that enemy body counts were a precise and objective measure of success, but this did not count on factors like the impacts of the common Vietnamese people, as USAF Brigadier General Edward Lansdale pointed out. The McNamara Fallacy is summarized as measure what is easily measured and disregard what cannot be easily measured or given a quantitative value.

McNamara has the dubious distinction of having a second logical flaw named for him - the McNamara Folly. Treating each soldier as abstractly more or less equal to another, McNamara thought that with the right training and superior equipment, simply increasing the number of soldiers would be a positive. Thus he decided to lower admissions standards to increase the number the soldiers. The results were that these new inductees died at 3x the rate of those soldiers that met the earlier standards.

Consequences to security

Increasingly, I've seen an emphasis on measuring anything about security and using only that chosen to make decisions. Thus backing into the McNamara Fallacy.

Now, many of what is measured may have value for a given domain and for well-established technologies and designs for systems which have previously been attacked by sophisticated threat actors (though, note my prior article If __, you might not be Secure By Design Part 2 | LinkedIn). But technology, new design concepts, complexity, applying to new domains - these all represent areas where the old measures may not apply - the measures may be all examples, to stretch an analogy, of putting new wine in old wineskins.

And what of the case where a nation state adversary holds in reserve attack capabilities for the future when the element of surprise will aid them? There is no historic evidence about such attacks.

The effect of attacks on new technology, new design, and old approaches in new domains are not quantitatively measurable, at least not reliably. Yet the bias is to assume they are - it feeds the measuring what can be easily measured even if it may not be applicable in a new context.

(and this is not even getting into the streetlight effect fallacy - the idea of only looking where it is easy to look).

I believe McNamara's folly is roughly seen in security when simply putting countermeasures into place without thinking of the quality of the countermeasures and the quality of their integration. Sami Saydjari in his book Engineering Trustworthy Systems talks to this with his observation that security without assurance is veneer security. (and this is not to even address the need to consider inherently secure design). Though again note the need to avoid holes instead of plugging holes with quality plugs (countermeasures). See Time to Stop Avoiding Avoid | LinkedIn.

Thoughts, reactions?

Unless otherwise stated, all views expressed are mine and don’t necessarily reflect those of my employer or MITRE sponsors.