The Confirmation Trap

Data analysis can help us sort through complexity and make decisions, but even with the best analytic tools, we’re still vulnerable to human mistakes. For instance, we’re likely to pay more attention to findings that align with our beliefs and to ignore other facts and patterns in the data. This is called the confirmation trap. You can avoid it by trying to embrace information that counters your (or your boss’s) beliefs by doing the following:

• Specify in advance the data and analytical approaches on which you’ll base your decision. This will reduce the temptation to cherry-pick findings that agree with your prejudices.
• Actively look for findings that disprove your beliefs. Ask yourself, “If my expectations are wrong, what pattern would I likely see in the data?” Enlist a skeptic to help you.
• Treat your findings like predictions, and test them. If you uncover a correlation from which you think your organization can profit, use an experiment to validate that correlation.