Fast feedback loops - how to set up quick experiments for your business

Experimentation isn’t just about testing ideas - it’s about learning fast, adapting quickly, and building a cycle of continuous improvement. Every experiment, whether it succeeds or fails, provides valuable insights that move you closer to your goals.

Here’s how to embrace experimentation as a learning experience and set up quick experiments to gain actionable insights.

Why Experiments Are Powerful Learning Tools

1. Low-Risk Learning Quick experiments allow you to test ideas without committing large amounts of time, money, or resources. They create a safe space for trial and error.
2. Data Over Assumptions Rather than relying on guesswork or intuition, experiments provide real-world feedback, helping you make informed decisions.
3. Fast Feedback Loops The quicker you test, the faster you learn. Small, focused experiments shorten the time between action and insight.
4. Adaptability By consistently experimenting, you create a culture of flexibility and resilience, essential for navigating uncertainty and change.

The Anatomy of a Quick Experiment

To maximise learning and minimise waste, your experiment needs a clear structure:

1. Define the Hypothesis Start with a testable statement: "If we do X, then we expect Y result." Example: "If we reduce the signup form to three fields, we expect a 15% increase in conversion rates."
2. Set Clear Metrics Identify what success looks like. What specific data points will determine if your experiment worked? Example: Increase in website clicks, improved engagement rates, or reduced churn.
3. Choose a Small Scope Focus on one variable at a time. Avoid trying to test multiple changes at once, as this can muddy the results. Example: Test a new email subject line rather than overhauling the entire email campaign.
4. Run the Experiment Quickly Keep the timeline short—ideally a few days to a few weeks. Rapid iterations help maintain momentum and deliver insights faster.
5. Analyse Results Review the data with an open mind. What worked? What didn’t? What unexpected patterns emerged?
6. Iterate or Scale If the experiment worked, consider scaling it. If it didn’t, analyse why and iterate with a new approach.

Examples of Quick Experiments

Here are some specific quick experiments we implemented to optimise Play Moments? across key areas:

1. UX/UI Experiment: Onboarding Flow. Tested a simple 3-step onboarding process vs. a more detailed tutorial to see which led to higher completion rates. Result: The simple process retained more users, as they preferred a faster setup. The simple 3-step process retained 24% more users during onboarding than the detailed tutorial. Users who experienced the simple process were 37% more likely to return on Day 1. Onboarding time was 70% shorter, catering to the preference of busy parents seeking a quick setup.
2. Distribution Experiment: Freebies for Sign-Up. Offered a downloadable activity sheet vs. a free trial of the app to see which incentive converted better. Result: The free trial generated more conversions and engagement with the app. The free trial incentive resulted in nearly double the sign-up conversion rate compared to the downloadable activity sheet. Users who opted for the free trial were 3x more likely to engage with the app within the first week.
3. Pre-Launch Campaign Experiment: Countdown Campaign. Ran a 25-day Christmas Activity Advent pre-launch countdown with daily activities of the app’s features to build anticipation. Result: We've built a waiting list of 10,000 subscribers.

Use this Experiment Picker?tool?from the Board of Innovation for more ideas.

Tips for Effective Experiments

1. Start Small: Begin with simple tests that are easy to implement.
2. Stay Objective: Be open to the results, even if they contradict your assumptions.
3. Focus on Learning: Treat every outcome - positive or negative - as valuable information.
4. Involve Your Team: Encourage collaboration and diverse perspectives in brainstorming and analysis.

Where to Record Experiments

1. Dedicated Tools and Software

• Notion: Create a custom database or workspace to record and organise experiments, including hypotheses, results, and insights.
• Airtable: Use its spreadsheet-database hybrid format to track experiments in an organised and searchable way.
• Google Sheets or Excel: A simple and accessible option to track experiments with customisable columns.
• Trello or Asana: Create cards or tasks for each experiment and categorize them by status (e.g., "To Do," "In Progress," "Completed").
• Specialised Experimentation Tools: Platforms like Optimizely or VWO can help record and analyze experiments for digital products.

2. Centralised Knowledge Hub

• Use a shared folder or internal wiki (e.g., Confluence) to consolidate experiment records for easy access across the team.
• Tag experiments by business function (marketing, sales, product) to allow team members to find relevant insights.

3. Experimentation Journals

• For small teams, keep a physical or digital journal to document informal experiments and insights.
• This can be useful for quick brainstorming sessions and ad-hoc experiments.

How to Record Experiments

1. Use a Consistent Framework

Document experiments in a clear and consistent format. Here’s a simple template you can use:

• Title: Give the experiment a descriptive name.
• Objective: What are you trying to learn or achieve? Example: “Determine which onboarding process increases retention rates.”
• Hypothesis: State your assumption in an “if-then” format. Example: “If we simplify the onboarding process to three steps, retention will increase by 20%.”
• Experiment Design: Outline the methodology. Example: A/B test onboarding flows for 500 new users.
• Metrics: Define the key performance indicators (KPIs) you’ll measure. Example: Retention rate, onboarding completion rate, time spent on onboarding.
• Results: Record the data and outcomes. Example: Simple onboarding resulted in a 30% higher completion rate than the detailed tutorial.
• Insights: Summarise what you learned. Example: Users prefer a faster setup process, so we’ll implement the 3-step flow permanently.
• Next Steps: Outline follow-up actions or adjustments. Example: Test adding a personalization step to the onboarding flow.

Why Experimentation Fosters Growth

When experimentation becomes a habit, learning accelerates. Teams adopt a mindset of curiosity and adaptability, fostering innovation and resilience. This iterative approach to problem-solving creates a culture where action drives insights, and insights fuel better decisions.

Final Thought Quick experiments are the secret to learning fast. The key is to start small, measure everything, and iterate relentlessly.

What’s one small experiment you can set up today?