What a Data Scientist has learned about Product Development

Originally posted on medium

It has been interesting to watch the development of two rather esoteric disciplines mature over the past decade.

For those who are coming into analytics within the last 3 years I’m super envious. If you had told me 10 years ago that what I liked to do for fun (machine learning) was going to mature into a seismic shift in computing and software I would have laughed at you and asked: “Then how is it that I can barely scrape buy as a ‘statistician’?”.

But much has improved. The last AI winter completely thawed and with the last five years the current vanguard of AI specialist were acquired by the likes of Google (Hinton & Deepmind ’11) and Facebook (LeCun ‘12) along with a steady flow of AI focused startups.

I have a friend, Bronze Swallow (yes his real name and yes we will start a Prog-Rock band called Cellar Door if the bottom falls out of sofware), who made the transition from gaming where he says:

“UX is a do or die proposition. There’s no such thing as a successful game with an ‘ok’ user experience.”

to software:

“In tech UX is at best an afterthought.”

because he felt like the creative constraints in software would be a great second phase of his career.

I think his transition is indicative of the pressures in tech to really nail the user experience. We are not far off, if already at the point, where you will not see a successful company that only has an “ok” user experience.

A skeptic’s paradise

The most rewarding thing you can do as a data scientist is to figure out exactly how to interpret data and integrate that insight into a living solution that either prompts better action (human in the loop) or optimizes a system (machine to machine). Maybe the second most rewarding thing is to surface an insight that at first blush sounds insane but that after it works everyone starts to come up with “just so” stories that explain away the insanity. That’s when you most feel like you’ve created something special.

Change the wording around subtly and I’ll be product owners and managers everywhere would say similarly about their own work.

But you can’t get to these counterintuitive wins be drinking the coolaid. You spend the first part of almost every project or initiative agnostic. And no matter how nice you try and come off everyone around you gets at least the sense that when they are talking, without clear evidence, that you’re probably muttering silently “opinions are like …”.

A few years ago I made the transition from one-man-and-a-briefcase consultant to real jobs. My primary motivation was to learn software product development from a data science perspective. At the time I had a half-baked theory that while on their face these two disciplines were not even remotely related that eventually they would become almost indistinguishable. Four years on and I’m sticking with my theory.

So here’s what I’ve learned from great product development minds like Vicki Thomas, John Cutler, and through passive contact with the great folks at Pluralsight.

1. You must begin by identifying a well formed problem statement
2. Gather evidence: user data, domain expert’s ideas, user feedback, etc.
3. Decide how to decide if we’ve “moved the needle”
4. Set up an experiment that can actually fail and be committed to following the evidence
5. Bask in the “just so” stories when the experiment finally converges on something successful
6. Repeat

Conclusion

If you pretty closely follow a framework that works the above steps then I’m guessing, regardless of your job title you’re either a really solid Data Scientist or a really solid Product Manager.

But if you’re indifferent to or unaware of most of the steps above you’re probably functioning primarily as an Project Manager, BI report writer, or alpha-obsessed Kaggler. These are all fine pursuits they just rarely “move the needle”.