Thought for Today

Anecdotes are not Facts nor are They Case Studies

An anecdote is usually a short narrative of an interesting, amusing, or biographical incident. Anecdotes are sometimes the starting point of a proper investigation, but it is all too often the ending point and every point of a pseudo-investigation.?

Man prefers to believe what he prefers to be true.

– Sir Francis Bacon

There is a broader statement about those?believing?in anecdotal evidence, either found by themselves or heard from others.?Anecdotal evidence is often used in politics, journalism, blogs, and other contexts to make or imply generalizations based on very limited and cherry-picked examples, rather than reliable statistical valid studies.

You can do little with persons who are disposed to accept these curious ... superstitions. You have no fulcrum you can rest upon to lift an error out of such minds as these, often highly endowed with knowledge and talent, sometimes with genius, but commonly richer in the imaginative than the observing and reasoning faculties.

– Oliver Wendell Holmes MD, in remarks to graduating class of Bellevue Hospital Medical College in 1871.

When we hear?I know someone that uses this method and she's happy with the outcomes, or I know a person that got vaccinated and nearly died has several core fallacies wrapped into one.

The first is the self-selection problem?of statistics. In our software development domain is the Standish problem. Send out a survey, tally the results from those that were returned. Don't publish how many surveys went out and how many came back. Or most critically, what the population of potential respondents was for the surveys and if the numbers of responses represent a proper statistical sample to assure high enough confidence on the returned surveys to draw any credible conclusions.?

In our current Covid anti-vaccine world the internet is ripe with anecdotes -several here on Linked In and many more on Facebook, Twitter, and Instagram. All unsubstantiated claims of unsubstantiated evidence. Remember

An anecdote is a?sample?of one?from an unknown population

The next is the?Anecdotal?sample.?I know a guy that...?in support of the suggestion that by knowing someone that supports your?conjecture, your conjecture is somehow supported.

These are both members of the?cherry picking?process. The result is lots of exchanges of questions to the original conjecture that have no basis in evidence for the conjecture.

When you encounter such a conjecture, apply Sagan's?BS detection kit

• Seek independent confirmation of alleged facts. Start with double-blind, peer-reviewed journal articles or government research sites.
• Encourage an open debate about the issue based on this peer-reviewed available evidence.
• Come up with a variety of competing hypotheses explaining a given outcome. Considering many different explanations will lower the risk of?confirmation bias.
• Quantify whenever possible, allowing for easier comparisons between hypotheses' relative?explanatory power.
• Every step in an argument must be logically sound; a single weak link can doom the entire chain.
• When the evidence is inconclusive, use?Occam's Razor?to discriminate between hypotheses.
• Pay attention to?falsifiability. Science does not concern itself with?unfalsifiable?propositions.

When there is pushback from hard questions, you'll know those making the claims have no evidence and are essentially BS'ing their constituents.