The Struggle of Introducing Machine Learning into Your Company

Mac Steele is the Director of Product at Domino Data Lab, a company building a data science system of record that speeds up insight generation and ensures results are integrated into business processes faster, increasing the ROI of an enterprise’s data science investment.

Sam: Mac, it’s great to connect! For people who are unfamiliar with you, can you quickly introduce yourself?

Mac: Of course. I’m the Director of Product at Domino. I started as our first product manager, and was seeing how much confusion existed in the market around what data science platforms can help companies to accomplish. Today, I work to figure out how to best explain what we’re building and to shape the industry in terms of the pain points we’re seeing from customers. I get to work with the largest data science organizations in the world and am in a unique position to see exactly what they are doing with data science.

Sam: Great. Ok, so it’s hard to escape mentions of machine learning and artificial intelligence if you read any business publication or Tech Twitter. At large companies, Chief Information Officers face increasing pressure to develop “AI strategies” for their organization. Is this pressure real, and what does it mean for these companies?

Mac: That’s right - there are mentions of AI everywhere you look. A recent Bloomberg chart showed the number of mentions of artificial intelligence in company earnings calls had more than quadrupled in under a year. The issue is that people don’t completely understand it, and they conflate different ideas. There is a lot of FOMO at big companies surrounding data science and machine learning. ‘Everyone is doing it so we do it too.’ The old model was to hire a ton of data scientists, put them on an island, and expect greatness to flow forth.

There is a lot of FOMO at big companies surrounding data science and machine learning. ‘Everyone is doing it so we do it too.’ The old model was to hire a ton of data scientists, put them on an island, and expect greatness to flow forth.

Unfortunately, that didn’t work so well, and accountability is coming. Companies can’t afford to have these expensive teams sitting in ivory towers, not constantly delivering business value. CFO’s are starting to demand that leads of data science team demonstrate how they’re impacting bottom. If they can’t, their teams can’t grow any more. There’s a shift away from data scientists being heroes working on projects that don’t scale, to them working on projects that are both scalable and deliver real business value.

Sam: Yikes. Ok, what foundation does a company need in place to really take advantage of machine learning systems?

Mac: The companies that do this well have made investments in good data infrastructure. They have good business intelligence systems in place that answer the “what and when” of their business data. That’s critical before handling the “how and why” questions that machine learning and data science can help to answer.

These companies also have product management types within their organization who are tasked with integrating the findings of data science teams into existing processes and workflows. They explore the true nature of problems organizations face, figure out how existing systems work today, and make sure the outputs are seen by the right people at the right time. At this moment, we think data science has more to learn from product management than engineering.

Otherwise, you can have the case of some nightmare scenarios playing out. There are examples of companies that have built 10x better models, but don’t think about how these things would work within the context of the business -- like a lead scoring model that could have reduced churn by 10%, but was never built into CRM where sales and marketing people were actually working.

Sam: Why do companies fail to integrate work being done by the data scientists into the other workflows of other teams?

Mac: As data science teams grow beyond 5, it becomes hard to keep track of ad-hoc processes for collaborating without a central platform. Teams get inwardly focused and often obsessed with tooling at the expense of delivering results.

Sophisticated teams think about making the practice of data science better. They want to know how many projects are in flight, to understand how they’re going, who’s working on what, the range of seniority and skill across projects, and to have visibility into a project at risk of going off the rails.

Whether it’s machine learning or data science, results of these projects drive everything from marketing to sales to HR. If a business can’t keep track of what their data science team is building and what sort of progress is being made it can become really scary.

Sam: What challenges do companies face as they build a competency in machine learning?

Mac: It all starts with hiring top data science and machine learning talent - something that is incredibly difficult given this competitive market.

Companies then mandate certain tools which effectively shuts off huge portions of that labor pool. It’s hard to convince someone to learn a new IDE if they spent years perfecting their knowledge of RStudio.

Once you have attracted someone, it’s a challenge to onboard them, and give them permissions and access to all the tools they need. You try to have standard software environments for data scientists to work on, so they can select what they’re working on with the standardized tools and start working quickly. You don’t want to waste their time or to dishearten them.

In addition to onboarding, not knowing what other people have done can be a major time suck. If you can’t find where the models were made, or reproduce them, you’ll have to recreate the wheel. People hate that forensic black hole of figuring out what’s been done, and they should never lack the context to improve on pre-existing models.

Companies themselves also face the Key person risk. If a data scientist or ML engineer walks out the door, and they were sloppy about committing and documenting their progress (not just code, but also why they picked certain data to begin with), critical IP will be escaping your company that you won’t be able to get back. You need to know what they did, why they did it, and it’s almost impossible for organizations to do that.

Sam: Mac, thank you so much for your time. It was a pleasure to speak with you. Thanks for all the insights.

Mac: Thanks, Sam!

Join 30,000+ people who read the weekly ??Machine Learnings?? newsletter to understand how AI will impact the way they work.