MLOps Best Practices

Every data scientist can relate to this quote. “…developing and deploying ML systems is relatively fast and cheap, but maintaining them over time is difficult and expensive.” – D. Sculley et al .

Perhaps you have encountered it in your search to solve a problem in one of the many moving parts of your machine learning system: data, model, or code.?

While it’s relatively easy to develop a model to achieve business objectives (item classification or predicting a continuous variable) and deploy it to production, operating that model in production comes with a myriad of issues.

Model performance may degrade in production for reasons such as data drift. You might need to change the preprocessing technique. This means new models need to be shipped into production constantly to address performance decline, or improve model fairness.

Hacking together a solution usually means incurring technical debt, which grows as your system ages and/or grows in complexity. Worse, you could lose time, waste compute resources and cause production issues. This calls for "MLOps".

Some practices you should definitely consider implementing are:

• Naming conventions
• Code quality checks
• Experiments— and track your experiments?
• Data validation?
• Model validation across segments
• Resource utilization: remember that your experiments cost money?
• Monitor predictive service performance
• Think carefully about your choice of ML platforms
• Open communication lines are important
• Score your ML system periodically

Try them out, and you’ll definitely see some improvement in your work on ML systems. What other factors/practices do you consider important ? I would love to hear your thoughts.