Making Right Product Decisions

I occasionally consult start-up founders early on in their journey. Many quote they want their first product hire to have 'good business intuition', only few quote 'data driven' as a key skill they're looking for. I like this paradox as I believe business intuition comes from being curious enough to dive deeply into an ecosystem, understand its driving forces and the data behind what moves it, and as a result - have the intuition to make the right decisions. The more data points one has - the stronger intuition they build.

In this article I will share the frameworks that help me make product decisions, and can help with questions such as: How to choose the right initiative to invest in? How to choose the right start-up to join? How to choose an investment opportunity? And much more. I will also cover the main pitfalls that can lead to negative outcomes and how to watch out for these and mitigate them.?

How to use data to make better decisions?

Tip #1 - Understand metric drivers before brainstorming solutions -

Lets say you defined a goal you'd like to reach or a metric you're looking to grow. A common practice that would follow is hosting brainstorming sessions with the team on initiatives that can help reach this goal. A key step that can help the brainstorming session be effective is to first understand what are the contributing factors to this metric movements. This is the set of questions I usually go through:

• What was the main contributor to the growth of the metric when comparing it 12 months ago and now? is it top of funnel growth or expansion of value from existing users? Here I'm looking to understand where I should invest short and mid-term (short - double down on what's working, mid-term - address gaps in what's not working as new solutions will take time to materialize).
• What channel did most of the user growth in the metric come from. Here I'm looking to see if I can invest more in performing channels, kill non-performing ones and open new ones.
• how effective discovery of the product is - where in the product traffic comes from and what's the conversion rate of the different points. Here I'm looking to see if we can grow improve discovery by adding new points or optimizing existing ones.
• How often users use the product and what unique attributes do those who retain have vs those who don't retain. Here I'm looking to understand if I have better market-fit with certain use cases/geos/segments I can double down on and where I don't do well and need to understand better why and if/how to address.

Remember that some behavior is attributed to the stage in the life cycle a product is at (early adoption, growth, matured product) and to natural human behavior driven by use cases vs actual gaps in the product.

Your brainstorm now would focus on the problems for which solutions are guaranteed to create impact - for example - 'how to accelerate first time user acquisition focusing on X/Y/Z'.

Tip #2 - Understand the size of your Total Addressable Market and how it breaks down -

One of the most common predictors for success is tackling a problem with either a 1) Large enough audience or 2) Large enough monetization (ideally both). If neither exists - expect a lot of focus on profitability and uphill battle. For example, lets say you work on providing financing to individuals who sell high-end art. You're building a streamlined process for loan origination via automated evaluation and the idea sounds like a winner. One Q to dig into is - how many individuals buy high-end art w/financing and what % of high end art transactions have financing. If you discover this number is small - know you have a small addressable TAM and you'll need to go for high monetization (and the next question would be - what's your customers' Willingness To Pay?). You might see that there is a large set of transactions with no financing and believe you can educate individuals on how financing can be helpful here for leverage - know that changing user behavior is more difficult than addressing an existing one and ask yourself - even if I did change that for a portion of the population - is the TAM large enough? At the end - the idea and product may be phenomenal but if you don't have a large enough market to pursue, you'll find it hard to grow.

Tip #3 - Unit economics are key -

Lets say you're debating taking a role in a start-up; the hiring manager pitches you on how they're automating the creation of architecture plans for consumers doing house redesign - a very manual process requiring weeks of work from an architect with lot of room for error, where errors often require resubmission of documents for municipality approvals - a lengthy process. It makes sense, and you might have gone through this and it resonates. The company heavily invests in a great tech team and has a compelling vision around process automation, and you're ready to join. A key question to ask here is - how do the unit economics of a transaction break down? Asking this may unveil for example that it costs $3K in marketing to bring a lead that converts and another $2K in Sales to close that lead. The architecture doc origination costs $1K for manual labor and the company believes automation can cut it by 50%. Revenue today is $6.3K - aka $300 profit for an architect, indicating a possible $800 profit per transaction with automation for the company. Most customers do 1 redesign on average. Knowing this - is reducing origination costs the biggest opportunity here? The Cost to Acquire a Customer (CAC = marketing + sales) is very steep ($5K), and the Life Time Value of the customer is low as they don't repeat, hence you might be better off spending your efforts in reducing acquisition costs through partnerships with contractors for example vs traditional marketing, and increasing Life time value by serving an audience like contractors or real estate investors who bring repeat business. It's true that you can grow profits but over what horizon given the cost you'll invest in R&D? and what would be the valuation of the company as a result? This CAC/LTV dynamic is the very reason subscription services are so profitable (high LTV), and perhaps a pivot in strategy might be a better choice here to grow profits much faster at a lower cost of investment. If you're choosing a start-up to join or invest in - looking at the unit economics to understand where efforts would have the highest ROI can help you make the right decision.

Tip #4 - Understand the competitive landscape and create benchmarks for expected outcomes -

Expectation and context setting are important for target setting and opportunity sizing. For most problems - there are paths other walked in that can indicate what success and average case look like, helping you set the right goals. For example - lets say you're selling coffee beans and you're seeing that without bringing new customers consistently into your funnel, your revenue will be declining. This may be worrisome and raise questions like - 'Why our retention is so low and how do we revert it? Do customers not like our coffee or do they switch to buy from competitors? What can we do to grow new customers?' To answer those and understand where there is a real problem vs where the behavior is expected and similar among competitors you may look at other companies and learn from 1) public data (investor relation reports, article and interviews with company leads) and 2) private conversations with peers. This may uncover that coffee bean sellers across the board see declining revenues if they don't bring new customers, and their ratio of 'existing vs new business' is indicating lower retention than yours. If you learn you're doing better than other players including those with more experience and resources than you - this is a good signal to say - perhaps this is the dynamic of the business you're in, and something that is hard to move. It might also be that in your research you'll come across different models like a subscription one that may show better outcomes than yours - that's also a good insight to consider for your business. The knowledge either way helps 'normalize' what the expected dynamics should be (is what I'm seeing good or bad), what targets are realistic to set and what other models and lessons should be considered.

Tip #5 - Data democratization -

Make sure data is easily available to your team to make the right decisions in a timely manner. Data can be available self serve and supplemented via BI/DS teams, whatever path is taken - ensure that if a product manager or a leader on your team has a Q - they can get an answer to that Q within days/weeks, not months, quarters or years. If you want to make the right decisions - ensure your decision makers have data at their finger tips AND use this data smartly before building conviction in a product direction. I can't write this paragraph without thanking and highlighting the amazing DS partners I get to work with now and collaborated with in past experiences - you know who you are, and you make all the difference in helping your product partners make the right decisions.??

When do wrong decisions take place and how can leaders avoid them?

#1 - Team Incentives -

Each team has an outcome it optimizes for. For example, in most companies the Sales function optimizes for revenue. An additional metric that often comes to play is the conversion of a lead to a successful deal. Makes sense right? This simple metric can be the source of a huge revenue loss when not set-up correctly. How come? Your sales team would work towards having a higher lead conversion to optimize this metric. That means - they would often look at Marketing/Sales dev./Product to bring them more qualified/higher intent leads. The leads generation function on the other hand would be usually incentivized to bring the highest amount of qualified leads possible which would naturally drive it to reduce friction in the funnel. This may lead to a heated debate between 2 functions, where one might push for more friction to get the 'tire kickers out' and bring only the highest intent leads, while the other would optimize for higher amount of leads in the funnel. The reality is - your buyers do come with a certain intent, and the more friction you have - the more you'll drive away 'less determined leads'. However - it is well tested and known that a large portion of 'less determined leads' are still looking to close a deal, and the more friction you add over competitors, the less likely you are to get these leads' business. There are ways to screen, score and rank leads based on intent I won't get into, but this example showcases how metrics that make a lot of sense can also make a business waste time and lose a lot of money. So how do we avoid this? 1) Set very high level true north metrics (aka revenue), but be very cautious keeping the sub-level metrics you define (vs team defines) to minimum that's aligned?between your teams. 2) Spend time in playing 'how can this metric drive the wrong business outcomes', brainstorm with others and write down possible scenarios with mitigations, changing incentives as needed. 3) Once you recognized there's a problem in your ecosystem - whether it's in a sub-tone in a meeting/any tension raised - act fast, your team is going to optimize for what you set them up too, if you find a flaw - don't wait to change the set-up.

#2 - Desire for Consensus -

In organizations, one's success is heavily dependent on how they're perceived and work with others. This can have a dangerous side, for example - if you find yourself in a meeting heavily debating where a certain link should live vs testing it - you're likely not optimizing for outcomes but rather for finding a consensus among competing priorities. Here, having a culture that drives decision through experimentation is key, especially as the amount of users, characteristics and use cases in your eco-system grows. Leadership tone and examples matter - the more data driven culture you create and decisions you yourself make, and the more the team sees your decisions support an open testing culture, the more likely you are to have compounding growth impacts.

#3 - Strong leadership conviction -

Sometimes, there's strong leadership conviction to pursue a certain product or direction. Leadership has to show high passion and determination to rally teams around this direction. The danger can be if that conviction ends up having flaws, and resources, budget and time is spent before the company realizes this and changes course. Conviction coming from leadership makes objections harder to be raised - individuals would naturally be less comfortable disagreeing. This may be caused when information was not known to leadership when the decision was made. Estimates around potential outcomes haven't been put together or vetted before pursuing a direction. Signals from seeing similar businesses fail with competitors have been discharged. Unit economics weren't analyzed. At the end, the data behind the business case wasn't sufficiently established and as such - the initiative was never set for success. How can companies and leaders avoid it? As a leader - don't let passion and belief in your personal conviction coupled with the desire to go in a certain path fast override the research - spend the time to look at the data before deciding on a strategic direction, listen to what your teams say and if they have concerns - ask them to come back within a set time to present the data behind their concerns with alternative suggestions, don't brush it off. Research everything you can to build a conviction in the data behind 'what would need to happen/fall into place' for the product to succeed. Once you built that data based conviction - you maximize your products' chances and can have the right information to bring the team along. I'm an intuitive leader, I have strong senses around what may work or not and experience that might make me believe I make right decisions more often than not - I always remind myself that the best 'intuition' decisions I made, were ones I first backed with data.

Hope this tips were helpful to you!

If you've missed my last article - please check out 'How to Move Fast' !

#productmanagement #leadership #growth #productgrowth #datascience

Opinions are my own only!