The art of experimentation: A path to continuous product growth

“Take nobody's word for it”

In the realm of product growth, where innovation meets strategy and potential meets performance, experimentation emerges as the catalyst that transforms possibilities into realities. Like a skilled alchemist blending rare elements to create gold, experimentation is the magical process that turns ideas into tangible success stories. But wait, how does experimentation actually work??

'Nullius in verba' is Latin for “Take nobody's word for it” – it is the motto of one of the world's oldest scientific societies, the Royal Society. It neatly expresses the idea that the credibility of information derives from evidence, observational or experimental.

So, the root of experimentation starts from “Take nobody's word for it” and reaches a conclusion through clear evidence from an unbiased test.

Why experimentation is important

Imagine you're asked to draw a circle. The more tries you get, the better your circle will be. Likewise, incremental growth comes through experimentation. For instance, to boost the conversion rates of a website, we might experiment with elements like Call-to-Action (CTA), pricing, and promotions on the website.

Key principles for experimentation

Let’s look at the main principles of experimentation:

One blend (Experiment) at one time

Experimenting with one thing at a time is crucial for clarity and precision in our endeavors. Just as you can't juggle one ball in multiple iterations simultaneously, you can not use the same variation that is already being used in other unconcluded experiments.?

For example, if you're testing pricing on your website, stick to one change per test because both sides of the equation need to be equal first. Mixing in other changes, like altering the color of a CTA, muddies the waters and makes it hard to see what's really affecting the results.

Avoid human biases over facts

If I keep it simple, in the magic of experimentation, human biases are like fog that clouds our judgment.?

Here is how:??

Two years ago, I ran my first small experiment: Changing the color code of a CTA. Initially, my human bias led me to favor Variation A (it had my favorite color), but when confronted with the evidence, it became clear that the control variant outperformed. So, it is very crucial to understand the fact that anything you feel is going to work in a certain matter, does not even fall under the core definition of an experiment.

Looking at the rightly calculated metric

Choosing the right metric is essential when experimenting. Just as you wouldn't use a hammer to tighten a screw, you need to select the metric that best reflects what you're trying to experiment with.?

Then, the most important part is the definition of your selected metric. Take conversion rate again as an example. This is nothing but a simple piece of math that divides sales by traffic but the problem occurs when people see different traffic numbers (which can be relatable if you have seen ‘New Users’ and ‘Total Users’ on GA4) and come up with different calculations.?

This is why not just the right metric but the rightly calculated metric for your experiment is super important.

Documentation: The backbone of Experimentation

I would define documentation as the backbone of an effective experimentation process. Without it, you’re like a ship without a compass - moving in multiple directions and eventually reaching the same point where you started, without even realizing it.?

Many businesses were seen wasting time and resources by repeating experiments because they didn't document their findings. It's crucial to maintain a comprehensive document for both concluded and ongoing experiments, including their findings, accessible to all teams.?

This practice saves time, prevents duplication of efforts, and fosters alignment across the organization. Therefore, investing in documentation is essential.

Innovation is planted through Experimentation

In general, businesses often see experimentation as a way to improve their online presence and boost performance metrics such as conversion rate (apologies for being addicted to this metric and using it over again). However, they miss out on a bigger opportunity: using experimentation to test, learn, and even fail as a path to strategic innovation.

When Jeff Bezos talks about experimentation, learning, and failure, he isn't just talking about small tests like changing button colors. He's referring to bold experiments like the Fire Phone that could reshape an entire business.

Experimentation has become synonymous with testing for metric improvements, but it should really be about taking a critical approach to business decisions and driving innovation.

On a final note

Experimentation is the guiding light in the journey of product growth and innovation.?

Rooted in the wisdom of “Take nobody's word for it,” each experiment becomes a step towards discovery and transformation. From focusing on one experiment at a time to choosing the right metric, every decision is a deliberate move toward clarity and precision. But, don’t forget to document that clarity and precision.

Moreover, innovation isn't just a destination; it's the journey itself. As we embrace experimentation, we open doors to redefine success and drive lasting impact in the ever-changing landscape of business.

About the Author

Shahzaib Ahmed is a Senior Growth Hacker at PureSquare.

He specializes in data analysis, growth hacking, and conversion optimization. With his tech-savvy skills and a keen eye for user experience (UX), Shahzaib drives innovative strategies for impactful growth. His passion for innovation and relentless pursuit of excellence make him a valuable asset in achieving and exceeding growth objectives.

You can also connect with Shahzaib on LinkedIn.

Moreover, if you’re looking to secure your online experience, click here to learn more about PureVPN.