A/B Testing is not yet a data-driven culture - An expected standard of social behavior is

Traditional Companies Tend to Copy Technology Rather Than Cultures

In 2001, Google innovated the A/B test and developed a testing infrastructure that tested nearly everything imaginable. To decide on the best shade of blue, they conducted A/B tests against 200 different shades to find out which one was most popular among their users.

?As is now well known, Google was very successful with this approach. So successful, in fact, that Google has become a significant model of what today is understood under digitization or a data-driven company.

No wonder, then, that many traditional companies want to copy Google's success.

Traditional companies tend to copy what is easily observable and changeable. In the case of Google, that means A/B testing and a large data platform with a few data scientists thrown into the mix.

However, data, testing infrastructures, and technologies are just the visible part of digitization. They are the easy-to-copy part. The tip of the iceberg. But what really makes a data-driven company?

My hypothesis: What Google really did was develop a social culture of decision-making driven by data and technology.?

The social culture of decision-making is the actual revolution of digitization for traditionally oriented companies, not data or technology.

?Simply adopting the easily copyable part of digitization can lead to disastrous decisions in companies, even though they are data-driven.

Example: The most famous A/B Test in the world that robbed Coca-Cola customers of the American Dream

The most famous A/B Test in corporate history is the "Pepsi Challenge." During the mid-1980s, Pepsi introduced the Pepsi Challenge in many markets. In a double-blind tasting, customers preferred Pepsi over Coca-Cola. This put significant pressure on Coca-Cola and led to market share losses.

Coca-Cola conducted its own studies which confirmed the results: customers preferred Pepsi in the A/B Test because it was sweeter than Coca-Cola.

So, Coca-Cola made a data-driven decision to develop a product that was sweeter than Pepsi. They called it "New Coke." And internal A/B Tests indeed showed that customers preferred New Coke over Pepsi.

Robert Goizueta then made a swift decision with his management team that "The best has been made even better" and launched New Coke on April 23, 1985, accompanied by a major advertising campaign and commercials:

Watch the commercial here

However, against the clear expectations of top management, the product launch turned into a disaster. Customers nationwide protested vehemently against Coca Cola's top management: "They have taken away our freedom of choice," the customers protested. The protests grew so quickly that Goizueta soon realized he had made a huge mistake. New Coke turned into an existential crisis for the company. On July 10, 1985, Goizueta apologized to customers and brought back the old formula. New Coke had failed, even though the decision was made based on data.

What happened?

Top management failed to understand WHY people really bought Coca-Cola. They didn’t buy it for the taste.

Further A/B Tests showed that people preferred the sweeter drink at a single sip, but not when drinking a whole glass or bottle. The data here wasn't as clear-cut. More importantly, people didn't drink Coca-Cola for its taste but for what it represented: the American Way of Life. Coca-Cola is the drinkable American Dream. By changing the formula, Coca-Cola's management robbed their customers of the American Dream.

The Mercedes as a Swiss Watch - or WHY do Chinese (not) buy (electric) Mercedes?

Another data-driven mistake occurred recently with Mercedes. They had developed a new electric vehicle, the flagship EQS, which surpassed all previous technical achievements in terms of range and many other technical values. The EQS was technically a masterpiece and won all A/B Tests. But it didn't sell well in China.

It has since been learned that Chinese view electric vehicles more as a consumer good, which they frequently replace and which shouldn't be too costly. A Mercedes, however, is seen as something mechanical, like a Swiss watch—a status symbol into which one invests a lot and which can rightfully cost a lot. The lack of success with the EQS is increasingly becoming a threat to Mercedes. In the automotive industry, mistakes due to long development times are not so easily corrected like with Coca-Cola.?

Overconfidence and cognitive Bias lead to wrong decisions or the overestimation of one's own ideas and perspectives

How does it happen that people make such poor decisions based on data? Looking at examples like New Coke or EQS, it becomes clear that the data simply did not answer the relevant questions. The Pepsi Test indicates that people prefer a sip of Pepsi over a sip of Coca-Cola, but not a glass or a bottle. And the Pepsi Test certainly doesn't answer WHY people drink Coca-Cola or why they buy it. These were the questions the management should have asked the data.

Top management drew fatal conclusions from the data: If New Coke wins the Pepsi Test, it will be bought by customers. What a prime example of distorted recognition (cognitive bias). In combination with the decision to quickly bring New Coke to market as an "easy decision," it was a fatal mix of overconfidence.

Unfortunately, on an individual level, there is very little we can do to counteract this. Training can help us become aware of the typical mistakes in decision-making. But it can't protect us on an individual level. Even the most intelligent people fall victim to these distortions of our brains.

The Human superpower: Valuing the ideas of others - fact-based argumentation culture

To really understand the data-driven success of Google, it's worth exploring its origins. Google originated from an academic, scientific environment. Its founders, Larry Page and Sergey Brin, were actually doctoral students at Stanford University. And it was this scientific culture that has made Google so successful.

Science is a social behavioral norm that was developed in the 17th century by England's Royal Society. This behavioral norm was so successful because it laid the groundwork for the technical revolution of the last centuries.

As noted by the New York Times in a review of the book "The Knowledge Machine" by Michael Strevens:

Read the review here

Thus, modern science began, accruing its enormous power through what Strevens calls “the iron rule of explanation,” requiring scientists to settle arguments by empirical testing, imposing on them a common language “regardless of their intellectual predilections, cultural biases or narrow ambitions.” Individual scientists can believe whatever they want to believe, and their individual modes of reasoning can be creative and even wild, but in order to communicate with one another, in scientific journals, they have to abide by this rule. The motto of England’s Royal Society, founded in 1660, is “Nullius in verba”: “Take nobody’s word for it.”

Different perspectives from people are the source of progress. But it is the behavioral rule "Nullius in Verba" that forces people to use a common standard language for arguing their points and arriving at a collective understanding. And this is independent of intellect, status, or religion.

In summary:

1. We are exceptionally bad at judging ourselves

2. We are exceptionally good at evaluating arguments of others.

3. The social norm of exchanging arguments based on data makes tech companies successful in digitalization.

4. Technology is just the tip of the iceberg

Digitalization as a cultural revolution of corporate culture - a digital norm in the boardroom

In traditional companies, science often has a negative connotation of being academic and theoretical. As soon as these words come up, an idea in a typical corporate day-to-day is dead.

If traditional companies want to become data-driven and successful, there is no way around adopting the underlying culture instead of just data and technology.

Relying on traditional behavior based on data carries the risk of cognitive bias and overconfidence. Without a social norm, this can lead to poor decisions that can endanger companies existentially.

Stayed tuned for further articles to demonstrate this behavioral norm in practice.