How To Build a Successful, High-Functioning Testing and Optimization Program

Much as every organization claims it is data driven, most are either in the process of building an optimization practice, or improving the one they have. There’s an art and science to testing—and while the science is inherent in the testing measurement, the art is in the many different ways you can start (or restart) your program. This effort has many names: Conversion Rate Optimization (CRO), Optimization, or sometimes simply AB Testing—all are used interchangeably. There’s little harm in the choice; just be sure everyone is aligned on what to call this so you are communicating together.??

What’s In A Name?

To begin, consider the word practice to name your optimization effort. The word optimization practice sets expectations, as the process of testing is inherently about taking shots and learning, practicing, and getting better at whatever you are doing. The optimization practice (or program, your choice) builds competency over time, and the team involved learns from experience what works well for your website or company versus what delivers slower improvements.

And to improve you must put in the hard work, which when done well starts with planning and education.

Start Here

A plan for your testing program should first consider who will need to be involved. Though you might be solely responsible for the testing practice, you will need a team of individuals, and their input, to succeed. If you have an internal leader who will champion the program, start with them and always keep them involved in your efforts—they are your primary audience for updates on progress, as they will help you get greater visibility within the organization. Their involvement alone will demonstrate the importance of what you are doing.

After an internal leader, consider who could be your technical partners. If you have the support available, involve developers (front end, back end, or full stack) as early as possible. You will need them to understand the implementation of your testing tool (a topic due a whole separate post), and you will want them to be aware of any changes that you are making to the site once the testing practice is running to avoid potential surprises.

Next, think about who else could be great partners of the program. The goal of a new testing team is to ensure everyone feels involved, that collaboration is welcomed, and that you want their input. Think about the diversity of opinions as well. Consider inviting members from a variety of groups that are not just a part of your own team. Who you include depends on your organization. A few examples are the Sales team and the Marketing Product team, but also consider groups that might not be top of mind, like the Finance and Call Center teams (each will have unique perspectives and their insight will be invaluable!)?

A breadth of teammates will give the practice greater visibility, provide a more diverse set of testing ideas, and keep your focus balanced, as there are usually parallel internal goals for each team (like quarterly goals to drive traffic but maintain a high rate of engagement, or deliver leads but ensure they are high quality, but low cost). Plan for a bigger team (6-8 members, for example) and expect not everyone will have capacity to join every conversation.

Great Expectations

The crawl > walk > run paradigm has no better application than website testing, so once you have a list of invitees to the testing team, start setting expectations. Education for the testing team and clarifications for the business at large will ensure anyone interested will know how the testing practice will succeed. A key concept then is defining that success—it is painful to admit, but no matter what industry you are in, most tests do not deliver a clear, statistically significant winner. This is worth restating: do not expect to have a clear winner for every test, because you are not doing this just to have winning tests, you are doing this to learn. Have fun, remember it is a process (a practice) and appreciate the surprises. A test that does not deliver a statistically significant result does not mean it fails. Instead consider what you can learn from it. Nelson Mandela is attributed a quote that is a good mantra here: “I never lose. I either win or learn.” Ensure the team and the business has heard this, and you will have a much more successful practice.

In my next post I’ll cover the details of getting your practice moving.

In the meantime, here are those other cover art characters. Credit: Midjourney bot, with the prompt: A mad scientist smiling wide eyed in front of a steampunk machine full of dials and levers