7 Steps to Foster a Culture of Experimentation from Scratch

Experimentation programs?at your organization can be exciting, especially when you know the positive impact that effective experimentation can have on the digital experience for your users. Unfortunately, the many competing priorities in organizations mean it’s not uncommon for others to feel nervous about optimization and experimentation, which leads to resistance and a lack of widespread support. To stay ahead of the competition and achieve sustainable growth, you need experimentation—which makes overcoming challenges in developing your testing program essential.

An impactful experimentation program?that achieves long-term success relies on a culture of experimentation within your organization. In this article, we’ll outline some steps to help you combat resistance, develop a culture of experimentation, and build company-wide support for a testing program that will serve you long into the future.

STEP 1: GET THE TEAMS TOGETHER

The first step in kickstarting any successful experimentation program is to get teams together to align on goals, timelines, and expectations. This process may involve bringing together stakeholders from product teams, marketing teams, leadership, and others. It’s critical to ensure everyone is on the same page and understands the importance of experimentation in achieving your business objectives. True and earnest experimentation relies on participation and team buy-in—the process won’t succeed otherwise.

During these team gatherings, you’ll want to accomplish the following:

• Answer key foundational questions:?Come to a consensus on the fundamental baselines for your testing program. Some key questions to gather feedback may include:What are your primary KPIs??What are you, as an organization, most interested in improving? If you have multiple answers, what’s the order of priority?
• How are you going to measure these KPIs??Make sure there’s a single source of truth for all goals defined.
• Do you have the right technology in place??If you already have a testing tool, does it have the necessary functionality? If not, how will you decide which testing tool to use? If you’re currently using Google Optimize, it’s time to look for a replacement. Check out our guide?“The End of Google Optimize—What Next?”?to learn about your best options.

1. Define clear roles and responsibilities:?Decide beforehand who is responsible for each aspect of the testing process so that things run smoothly once an experiment is ready to progress. Ensure responsibilities like test ideation, discovery, evaluation, prioritization, development, Q/A, final approval, and post-test reporting/analysis are appropriately assigned with checkpoints.
2. Assign tasks:?Depending on how your teams are structured and the level of involvement across each team, you may dedicate each task to specific teams or allow for some crossover to suit skill sets. Once tasks are clearly assigned, each team can develop its own standard operating procedures for carrying out its assigned roles. For example, the development team may want to define a standard Q/A process with multiple checkpoints to ensure no test launches before meeting specific criteria. Encourage team members to make suggestions and collaborate to foster cohesion and creativity beyond the initial experimentation.
3. Facilitate early ideation:?Check to see if there are any teams that already have ideas or initiatives for website optimization in the works. Product teams may have new features they’re developing, the website team may be planning a redesign, or the marketing team may be interested in personalized landing pages to complement upcoming campaigns. All these ideas are great candidates for early test initiatives and are examples of great ways to tie existing priorities to the culture of experimentation. It also helps further ingrain the idea that experimentation exists to help all teams meet their goals, rather than as a separate goal.

By getting everyone involved from the start, you can build a sense of ownership and accountability surrounding experimentation, as well as address any qualms teams have about testing before you start.

STEP 2: LAY THE FOUNDATION

Now that you’ve garnered some excitement and established baselines for your program, you’re ready to move on to the next step. Before you launch your first experiment, lay the foundation that makes a culture of experimentation possible. This process includes developing the necessary documentation so you’re ready to hit the ground running once the organization is fully prepared. At BlastX Consulting, we recommend developing the following essentials to start:

1. Predetermined testing criteria?and simple sets of basic requirements that all experiment recommendations must meet before being approved help focus efforts on achievable goals. These criteria will differ for everyone, but consider factors such as traffic volume, baseline conversion volume, and which KPIs you want to improve across your site. We also recommend requiring that all testing recommendations are clearly and easily measurable with a defined hypothesis and clear plans laying out the next steps once the test concludes.
2. A good prioritization framework?considers each testing idea and evaluates it based on three factors: Potential, Importance, and Effort. You’ll want to develop your own specific criteria for what defines each of these metrics; but in general, potential will gauge how significant an impact a test could have on its primary KPI, importance will measure how closely a test aligns with your overarching business goals, and effort will evaluate how easy or difficult a test will be to develop and implement.
3. A testing roadmap?will keep track of all your testing ideas and plans that have passed the predetermined criteria and prioritization stage and will act as a calendar for your testing plans. You can organize tests in this document by planned launch dates, areas of your site, and?primary KPIs measurements. You can also use this document to keep track of tests after they conclude by adding data points like KPI lift, confidence intervals, business impact, and any other relevant data points.
4. Create a template document?to outline the details of each test you run. This document should serve as a single source of truth to anyone viewing it. Don’t skimp on the included information. Make the reason for the test (including any discovery analytics data), the hypothesis, the planned KPIs to be measured, mockups of each variable included in the experiment, and any other details clear and available upfront. Testing briefs help to standardize the experimentation process and act as great documents to help get all teams on the same page before a new experiment launches. For bonus points, you can also refer to these documents after a test concludes and update them with the results and next steps to turn these briefs into post-test reports.
5. A testing repository document?is a high-level catalog or folder where you can easily access all your testing briefs, reports, and more information about every test. In this repository, it’s best practice to include only a few details about each test up front that can provide an “at-a-glance” summary, such as the test name, area of the effect, and whether it was a winner/loser/inconclusive. Then provide links to your testing briefs and reports for each one so users can deep dive into a specific instance without sifting through every testing document.

This chain of documentation will help ensure that your experimentation efforts are organized, efficient, and focused on achieving your goals. Having a clear plan and process in place helps reduce the risk of experimentation becoming a haphazard wandering with no clear direction.

STEP 3: HOST LIVE IDEATION SESSIONS

It’s important to help teams feel involved and empowered in experimentation. Host live ideation sessions that give everyone a chance to share their ideas and contribute to the experimentation process. Limit each ideation session to just one or two areas to avoid losing focus and adding confusion. You can kick-start these sessions by sharing relevant data about the current status and even provide a few example test ideas to get the group’s creative juices flowing. Then, invite all participants to share ideas and document them all.

By involving everyone in the ideation process, you can tap into collective creativity and expertise within your organization.

At this stage, welcome all ideas to encourage participation—you can run each idea through your pre-determined criteria and prioritization framework afterward. By involving everyone in the ideation process, you can tap into collective creativity and expertise within your organization, which leads to better ideas and more successful experiments.

STEP 4: START SIMPLE, BUT WITH BIG IMPACT

When launching your first experiment, you’ll likely have the most success with a focus that’s simple to build and launch but also has the potential for significant, measurable impact—bonus points if it’s an idea that meets that criterion and is also something many stakeholders are excited about. This stage is where pre-developed documentation comes in handy. You can utilize your prioritization framework to help identify which test(s) will be the “easiest” from a development perspective while also promising business impact potential. By selecting a first test that takes advantage of low-hanging fruit, you can fuel excitement and?motivation to continue experimenting?with less risk.

STEP 5: PLAN OUT NEXT STEPS AHEAD OF TIME

To maintain momentum and avoid losing steam after your first experiment, plan out your next steps ahead of time. Have a log of your next tests ready and planned so that your actions are clear following your first test. You could follow slightly different possible courses of action depending on whether the test was a winner or loser, but ensure a plan is in place regardless of the testing outcome.

Every test, whether a “winner” or “loser” in terms of the main KPI, still provides insight and an opportunity for business impact.

Equally important, make it clear that no matter the outcome of the test, there’s potential for learning and iteration. Every test, whether a “winner” or “loser” in terms of the main KPI, still provides insight and an opportunity for business impact. Consider the results from every perspective to extract value irrespective of the outcome. All experiment results should feed back into the roadmap and help to identify future optimization opportunities. It’s a cycle of continuous learning, iterating, and adapting.

By having a clear plan for next steps well in advance, you can ensure that experimentation remains a priority, avoid getting discouraged by “losing” tests, and maintain momentum after the first experiment.

STEP 6: SOCIALIZE THE RESULTS

Keep experimentation top of mind for everyone at your organization by regularly sharing the results of your testing efforts. Providing?consistent transparency?around how optimization is driving forward organizational objectives is the easiest way to ingrain experimentation into your company culture as a permanent fixture.

You can achieve this by implementing a monthly or quarterly testing newsletter, sharing test results as part of any company-wide update that already exists, speaking about your efforts in company-wide meetings, or even just forming a new Slack channel where all post-test result reports are freely shared. This type of transparency can also help increase excitement around experimentation and drive more participation.

STEP 7: MAINTAIN A REGULAR CADENCE

Finally, to ensure that experimentation remains a part of your company culture, maintain a regular cadence of discovery, ideation, experimentation, and iterative learning. Nurturing and tending to an experimentation culture means continually testing, learning, hosting ideation sessions, and sharing details of wins, losses and impacts even after the initial excitement of launching your first experiment has faded. By maintaining a regular cadence and high visibility within your organization, you can build a culture of experimentation that becomes ingrained in your organization’s DNA.

To ensure that experimentation remains a part of your company culture, maintain a regular cadence of discovery, ideation, experimentation, and iterative learning.

The goal is achievable, but not an easy process. Whether you’re stuck on the first step or hit a roadblock further along, everyone could use some outside help. When you can’t see the forest for the trees, use?BlastX Consulting?to help you set your sights on a new path forward.

View the original

ABOUT THE AUTHOR

Brittany Cole

As Senior Conversion Optimization Strategist at BlastX Consulting, Brittany helps businesses enhance customer experiences through strategic A/B testing, personalization, user experience design and conversion-focused copywriting. Brittany began her career as a digital marketer and data analyst, then moved into the CRO (Conversion Rate Optimization) field in early 2018. She has enjoyed flexing her data muscles alongside the creative and psychological aspects of the field. Brittany has earned a BA in Journalism with a Minor in Business from The University of Missouri-Columbia, an MS in Data & Analytics from Southern New Hampshire University and an MS in Management and Leadership from Western Governors University. She’s also a certified expert of Optimization Methods in Business Analytics through MITx.