What is “data hubris”?

In an insightful article by Kate Murphy (no relation) in the Wall Street Journal (May 24, 2021, R1), “Don’t Give Up on Focus Groups,” we learn how reliance on big data can lead us astray.  Drawing conclusions from numbers alone is cleverly described as “data hubris.” And while the article deals with marketing research, the point applies just as well to compliance work.

Recent years have seen a fervent pitch toward using big data to determine where to focus on compliance programs and whether the programs are actually working.  Guidance from the US Department of Justice, for example, makes it clear that if programs are to be credited, they need to engage in data analytics.  

But as the article makes clear, over reliance on data analytics can lead to bone-headed mistakes.  What always needs to be added is input directly from people. In one story, for example, Lego Group made a deep error by looking at larger trends and dumbing down its products.  But when it gathered direct input from its target audience it gained the insight that led it to enormous success.  

My perspective is that in the compliance and ethics field we need both.  In the antitrust field, for example, I was among the first, if not the very first, to push for the use of screening in compliance programs.  I picked up on the important insights of Romy Abrantes-Metz, who championed screening in enforcement activities, and she, Patrick Bajari, and I produced the first article calling for use of this data analytical tool for antitrust compliance programs, Abrantes-Metz, Bajari & Murphy, “Antitrust Screening: Making Compliance Programs Robust,”  https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1648948 . 

But we need to recognize that data analytics is, and will never be more than, one tool.  There is no substitute for talking with your people.  The Wall Street Journal article discusses the value of focus groups.  I agree that focus groups will remain an important tool.  So is the old idea of “management by walking around.”  So is that highly technical, analytical tool known as “lunch.” When doing training, just having the trainer hang around after the presentation and chat with people can offer insights not available from studies.

Without question, surveys have an important place.  Data analytics also offer an essential tool.  But don’t rely solely on numbers.  Don’t fall victim to data hubris.  Look at the numbers, but then go listen to your people.