It pays off to Experiment and Iterate.

"We spent months building this feature/ product, but we didn't see any adoption. now we are spending time maintaining it rather than building new things "

When we are working on a new product or exploring new market, we are taking risks, many a times working with a mix of data and gut feel.. and sometimes even purely on gut feel and minimal data. At beast we can be mostly right about those calls, learning from our experiences to get better. One pitfall I have seen a lot , is building products big bang where we didn't know enough. So, we end up wasting a lot of time and create maintenance overheads in many new products/startups by building good solutions for problems that customers didn't need as much. Cost overhead doesn't end there , instead of building new things, we then maintain features/solutions that very few customers use and spend bandwidth to roll them back over time..

I have done these mistakes multiple times. The one I remember the most is when we built "chat based Payments to a Person/Merchant" in FreeCharge. This was 2015 October. Why did we do it? Well, this was a big strategic move.. we believed time was right for us to promote Person to person/merchant payments and since customers were already so used to chat and how chat was the default experience in china (WeChat) even for payments and merchant transaction) , it made sense for us to build a chat based solution. We spent the next 6 months and half of our Engineering and product teams to build a shiny new payment experience , picked up top slot in IPL (Indian Premier League.. one of the most watched Sports event in India). and went all out..

What happened ?? almost nothing.. adoption was minimal.

Should we not have build the solution ? should we not have build it on chat etc?? if we hadn't we would have never known that it would/wouldn't work.. We learned a lot from this experience, but could this learning have cost less. We treated it like a product we knew our customers wanted, not like an experiment with which we will learn from and if results are good will lead to a good product. Had we treated it like and experiment, this learning could have costed us a lot less.

Lets see had we treated it like an experiment, what would have changed:

1. We would have launched a smaller set of features first and tested waters first. For example would would have launch person to person etc in first phase ( left away majority of features like request money, deep merchant experiences, deeper fraud detection capabilities etc,) run experiments on user traction, learn from it, tune it and then launch the rest if we know its working or stop and reinvent if we know it is not.
2. For, we could have chosen a lighter SLA earlier, launched a simpler solution first ( REST and Android/IOS notification based solution.) rather than full fledged longer term solution (XMPP based protocol). Choice of longer term solution costed us to support 10Mn active XMPP connections a day( 1 for every active app we had) on day 1, delayed our time to launch and continued to cost us 3,00,000 INR per month in extra infra that we continued to pay for 2 years ahead ( it takes time to undo as adoption of new app releases would takes months on a large consumer base).
3. We would have built it more decoupled, built more safeguards and On/Off switches and switched off features when we realised they were not working . Undo would have been a lot easier.
4. We would have outsourced fraud detection for V1 rather than building it ourselves .It would have taken us longer to reach the traffic at which in-house solution was essential.

All this would have helped us launch faster and with lesser cost, helping us learn more and invest the rest of bandwidth in more experiments in this / other areas.

We paid the cost not just in terms of the investment but also in motivation to invest more in this direction. For months post this, as leadership we were demotivated to invest deeper into this area and there was a natural bias to invest in other opportunities more.... (larger the investment in a bet.. most painful is the failure :)

Essentially my message here is that it pays off to iterate, run experiments when you don't fully understand what your customer wants or would get excited about, and most of the time you don't . All the product managers who think that they really know that this is what their customers want, either you have a lot of data(metrics), customer feedbacks or experiments showing that its true.. or all you need to do is look back at the last 10 times when u felt like that and what really happened to the those features when they launched......

One can argue, it will cost a lot more as a sum of all iterations rather than building it right the first time. But that assumes that you know whats the right thing to do and it will be successful.so if you average out all the times it worked out and times it didn't, I bet you at an average it's a lot cheaper to experiment and iterate, and go build big bang features.

So when do I know I need to experiment When I hear or say things like these:

• "Oh thats an interesting idea.. but don't have much data to understand user adoption but we believe it's a good bet. ".
• "Lets change this and see what happens".
• "Oh competition launched a new feature today we need to have it launched next week. we don't know how well the feature will go but we can't be left behind".

So how do we build a feature as an Experiment? These are some of the guidelines I use:

Choose early adopters wisely:

• Remember you are building for the first few/few thousand consumers. Even if it's widely successful, it will take some time for adoption to kick in. So you will have time to fix the gaps. You can take some well-informed short cuts. It's ok to say we can build this part after 6 months, not needed now. Helps you move faster and learn faster.
• You need a small set of customer to validate your assumptions. Choose them well. 80-20 rule applies well here. 80/90% of your features are for 20/10% of consumers. Skip those consumers out of your limited beta and move faster.
• Expect a fast follower next version, remember your next version is a few weeks/ a month away. So its not going to a lot of customers till then, invest iteratively in your scaling problems. If scaling is an issue intentionally reduce a sample size. The goal of initial launch is to see if you have built the right product. So what matters is conversion/ targeting efficiency, customer delight, high level pricing/cost learnings. not if you can scale the solution to most of your consumers.

Beg borrow steal:

• You don't always have to build yourself, see if buying helps you move faster in the short term and learn even if you think its best to build long term. Brainstorm on how hard migration is really going to be (if you long term solution is different), and build your solution keeping long term solution in mind. IMHO, most problems in migrations are because we didn't keep migration in mind while building the original.
• Don't be afraid of taking technical shortcuts that can help you move faster today, but remember to take the isolated/localised shortcuts/compromises that can be fixed individually without creating a network effect of cleanups. So you still need to get your ER diagram / API Syntax/ db schema right. But you can skip more api's, define a simpler background process that doesn't scale as well. build a library rather than a service, skip less frequently used user flows altogether etc..
• Use the standard hacks to slow to time/cost:

Treat it like an experiment:

• Don't be afraid to call it a beta, and be proactive to get customer feedback. It sets customers expectations well and makes even some of them more proactive to offer their appreciation and feedback on how product can be better.
• Build the On/Off switches . You may need to bring the feature down in some situations. maybe there is a user experience gap that needs you to pull back and fix.
• Be truthful to yourself, go deep and hard on experiment data to find if it's really working, fine tune, fix and remeasure. If your data is inconclusive don't be afraid to experiment more. Move ahead only once you have seen traction and met other baseline parameters.
• Learn from your experiments: What really happened , what could we do better. what can we do to run experiment faster. etc etc.. (IMHO how to reduce the cost of experimentation is a capability that companies need to invest heavily into and successful ones always have made large investments in this area. But thats a topic for another article ).

You need to think more not less:

Let me end the article by saying: motto is to do less, move fast and learn more, not think less. In fact a lot of times you will need to think a lot harder and more out of box.