What to Test & How to Measure?—?Value Stream Discovery Loop

Great tools exist to track the current state of startup’s and enterprise innovation team’s business model assumptions.

Ash Maurya’s Lean Canvas and Osterwalder’s Business Model Canvas come to mind. Tools/frameworks also exist to define and track experiments, such as Dave McClure’s Pirate Metrics, LSM’s Experiment board and our own Moves the Needle Experiment Map. Similar tools have been around for ages, by the way; used in lean manufacturing processes, Agile, kanban, scrum, etc.

We appreciate the analog nature of these tools, because teams gather around the boards and discuss, brainstorm and share. The digital versions will come?—?well, they’re already here. I should say the digital versions may eventually become useful, but in the lean innovation world, we’re still pretty early and analog works better.

The pressing question remains, however, where do the assumptions come from that populate these canvases? How do we articulate what our beliefs are about how the business is going to function, once it matures from idea to the real world? Further, how do we know where to start, what we should focus on, and what we should be measuring?

Announcing… the Value Stream Discovery Loop!

When I first discovered the Value Stream Discovery Loop (download-able at the bottom of the original post here), it completely changed how I viewed developing new products.

Like Lean Startup, the concept of a “value stream” also comes from the lean manufacturing world. Lean practitioners use value stream mapping tools to help organizations define all the business activities undertaken that directly create value for the customer.

In this vein, the Value Stream Discovery (VSD) loop helps you articulate what business activities you ASSUME you must undertake in order to create value, while facing extreme uncertainty.. Because as we all understand by now, in any startup or innovation endeavor, most of what we think we know is pure conjecture. We’re operating within the unknown.

The primary challenges have been these:

1. We tend to think in terms of the things a business ought to do, rather than how the customers “want” to interact with a product or company. I put want in quotes, because this isn’t necessarily something a customer can predict, so must be measured in their actual behavior, not from survey responses or even interview results.
2. There a gazillion activities a company might do (as described as expected tasks in hundreds of job descriptions), but what are the handful that actually capture the awareness of a person at the right moment in time, move them through a marketing funnel or sales pipeline, provide value such that they’ll want to repeat the experience, and delight them such they refer them to others?
3. Where should a startup or innovation team focus attention? What is the next bottleneck that needs to be removed to push growth?

The Solution

The trick in today’s customer experience driven world, is that rather than starting with the myriad business activities you might undertake, you focus on desired customer behaviors. You hypothesize your Value Stream from the customer’s (or user’s) perspective. By documenting what you must measure tied to customer behavior, the tool is then useful in creating a dashboard for your validated learning. in other words, the customer behavior determines the specific metrics you must track to measure progress, tied to the particular phase of the innovation endeavor you are in.

So working with hundreds of startups and innovation teams, I decided on 7 phases customers go through in their relationship with a product or company. Read from right to left:

The fundamental idea is this:

• Your customer moves from awareness of the value you are creating to being passionate about the company.
• The business does “stuff” to move customers along the progression.
• For each thing the business does, the customer behaves in a specific way.
• You measure the behavior.
• You hypothesize what the business must do; then run experiments to validate.
• By the way, I purposefully made this go from Passionate to Aware (left to right), because 1) it’s where the focus of the entrepreneur/intrapreneur should be early on and 2) there are infinite ways to go from aware to passionate and limited ways to get a market segment passionate.

For different products and different business models, there may be more or fewer states or perhaps they’re in a different order. But fundamentally this works for all businesses, including those with existing products in the market. For example, you might “zoom in” on the funnel portion and determine many more steps a buyer must go through in a large business-to-business deal. Those steps might correlate to a classic b2b sales funnel.

The customer experiences your business from right to left. But you hypothesize activities and customer behavior from left to right. (Imagine your ideal customer and imagine what behavior they exhibit that indicates passionate and what did you do to create that emotion?) Finally, you optimize from inside out (MVP, Growth & Conversion, Funnel, last is Acquisition.) Yes, you must get tranches of users (aware) to test and optimize everything else, but you’re not trying to scale or optimize aware until everything else is proven.

Let’s Pretend We’re Airbnb…

Airbnb is a great example for several reasons:

1. They had a specific customer type in mind.
2. They looked at their universe from their customer’s perspective.
3. They have had to continually iterate on multiple parts of their business model to find the next stage of growth.
4. THEY WORKED BACKWARDS. Brian Chesky, Airbnb CEO said, “do things that don’t scale. We start with the perfect experience and then work backward. That’s how we’re going to continue to be successful.”

As a marketplace, Airbnb would have needed to complete a Value Stream Discovery for both travelers and hosts, but we’ll look at just the traveler side in this (non-Airbnb approved) example.

Aware

Customer: Becomes aware of your existence.
Business: Reaches out through channel at moment of need.
Metric: Result of call to action.

Airbnb was not trying to serve “everyone.” They specifically wanted customers who were like them: professional travelers who wanted to connect with people who lived where they travel, who wanted a “locals” experience. These people are not searching for experiences on Booking.com or Priceline.

When it came time to scale “Aware”, they targeted a site with a monstrous amount of traffic and a top destination for people looking for vacation rentals. With an ingenious hack, Airbnb rentals appeared in the right place at the right time, for people looking for a travel experience beyond a hotel: a Craigslist ad (see Airbnb: The Growth Story You Didn’t Know).

For new customers, the desired behavior was for them to click on the Craigslist ad and land on the Airbnb site. They would measure unique visitors to an Airbnb listing via Craigslist.

Intrigued

Customer: Believes you speak to them with a powerful message.
Business: Promises utility value (we will solve problem x) + aspirational impact.
Metric: Next step in pipeline or funnel.

Airbnb iterated through several different messages, trying to find that right hook that promised an inexpensive room to rent, but also hoped to fulfill the aspiration of their chosen market: a great “locals” experience.

Actual Airbnb messaging iterations include: “Stay with a local when traveling” (launch?—?see archived webpage here), “Find a place to stay” (2009) and “Welcome Home” (2015).

Their home page messaging was all aspirational: we hope you have an amazing experience connecting with the local experience. The desired behavior was perhaps to learn more about Airbnb by clicking on “About Us” or perhaps a FAQ. They would measure the web browsing path of visitors from Craigslist, eg % of customers hitting FAQ who landed on Airbnb listing via Craigslist ad.

Trusting

Customer: Believes this company is capable of fulfilling the promise.
Business: Demonstrates proof of product and evidence of capability.
Metric: Next step in pipeline or funnel.

Airbnb has had to evolve and create new activities to build trust while onboarding customers beyond their early adopter market. They began with stats: “Rent nightly from real people in 16,981 cities in 190 countries” (see archived webpage here).

They also used shingles from major media outlets like CNN, NY Times and TechCrunch to lend legitimacy. The desired behavior of those who are trusting might be to browse listings in a particular area. Measurement is similarly tracking browsing path by source (Craigslist).

Convinced

Customer: Crosses rational or irrational gap to buy (or sign-up).
Business: Demonstrates value vs price, plus something extra to get over the hump.
Metric: Usually, money.

What did Airbnb have to put on its website to induce new customers to actually pick a room and complete booking? Like other aspects of their business, it evolved over time as they attracted new market segments. They added social proof via reviews and a “Trust & Safety Center: Your guide to a positive Airbnb experience.”

Desired behavior was to book a room. Measurement was payment from a valid credit card.

Hopeful

Customer: Wants to try and get value.
Business: Gets customer to attempt value as quickly as possible.
Metric: Initial use of specific, discrete product functionality.

Hopeful is tricky. The longer the gap between buying and trying, the less likely the customer ever gets to “satisfied” or “passionate.” Think about the toy package that declares, “batteries included!” The child can’t “activate” with the product without batteries.

After booking an Airbnb rental, the traveler likely feels a bit of trepidation. Did I book the right room? Will the host be responsive? This is a key moment for Airbnb, since this is an area where big brand Hotels (Airbnb competition) have already established trust that the room will be available, sufficient and hotel personnel there to iron out any issues.

Airbnb goes to great lengths to connect traveler and host as quickly as possible via text messaging or email. The behavior they need to see is that the host and traveler successfully connect to manage key exchange and hopefully establish a rapport. Airbnb should track that this is happening and the amount of time between booking and successful communications.

Satisfied

Customers are satisfied when the product solves the problem it promised to. You can’t over promise, since if you can’t fulfill it, you have no business. You can’t underpromise, or no one will care.

Airbnb promises a clean, safe rental to travelers, that’s less expensive than a hotel room. It must fulfill that promise or the traveler will never come back. So the behavior to track is that they do in fact come back. They book again. The challenge Airbnb has it that people don’t book every day.

For SaaS applications, for example, satisfied might be measured by tracking how many times customers use specific product functionality every day. “Customers check their marketing activity dashboard daily, add new campaign once every week.” Airbnb must track # of rentals per month or year. They might also think of different ways to engage customers in between bookings to measure engagement.

Passionate

Customer: Has achieved or is on journey to achieve aspirational impact.
Business: “Whole product” or brand that accompanies customer on journey.
Metric: Some combination of measuring value achieved, sharing, referrals, testimonials, “showing-off” the brand.

Satisfied customers don’t share with their friends that you did what you promised to do, Passionate customers do. How do you get customers to passionate? Tip: It’s not (just) your product. A product may have such a mind-blowing user experience that it drives passion, but more often than not, you must do something else in your business model that drives passionate behavior.

Airbnb’s provide specific mechanisms to measure passion: Only passionate customers consistently participate in the community, leave reviews and share their experience with friends. So this is what they must measure, % of customers that exhibit that behavior; how often they contribute.

But how do customer’s become passionate? In Airbnb’s case it’s likely that customers have realized the aspirational component of Airbnb’s reason for being, their “why”: travelers have an exceptional travel experience they don’t get with hotels. They have a “local’s” experience. They feel like they’re “home.”

The Value Stream Discovery Loop

The value stream discovery loop meets you where you are. It is the tool designed to help you identify the assumptions and metrics that matter at each step of the value stream. You can download it at the bottom of the original post here.

If you offer a new product, you experiment around creating satisfaction. If you already have satisfied customers, you might have one team focusing on product execution building off the MVP, a team optimizing conversion, while still another, discovering what is necessary to build passion.

By measuring customer behavior all the way from “Aware” to “Passionate”, you can discover where to focus to get the biggest return. The Value Stream Discovery loop is a great way to not only get all those assumptions out of your head, but also to create a dashboard of metrics you must track to measure your progress.

Happy discovering!