Building a Machine Learning Pipeline – Deployment

Welcome Back! Hope you enjoyed the previous two articles on building a machine learning pipeline (Part 1, Part 2 for readers who missed the previous ones) . In this article we conclude our theoretical journey of building a machine learning pipeline. The next article will focus on building a proof of concept. Without further adieu let’s begin!

Now that we have the data and the appropriate model it is time to actually make the model accessible and available for use.

To recap the deployment phase comprises of two aspects – 

1. Deploy to live environment 
2. Observe and refine the model 

There are several ways to deploy your model. These will be covered shortly but before that consider the following: - 

Type of predictions

1. Batch processing – In this scenario the model can be engaged as a background process that is triggered when perhaps when data is available in a data warehouse or uploaded to file storage like Azure Storage Account or AWS S3. E.g. Reporting or forecasting engines generally do not need real-time prediction/classification. 
2. Real-time / on-demand – We expect our model to provide real-time predictions based on the input. For example, machine learning based recommendation systems, facial recognition, natural language processing, etc. 

Ability to Observe and refine the model

1. Being able to retrieve and analyze performance metrics is key to making sure your model is serving the needs of the users and delivering value. 
2. In the event the model is not performant, there needs to be a fairly simple way to test, update/replace the model with minimal downtime and programmatic changes. A/B testing principles can be applied by routing input data to generate comparative metrics between a potential replacement or next-gen model. 

Based on the use-case we are dealing with one or more of the deployment solutions may apply: -

1. Direct Injection – Go with this option if you have a custom application or one that can be modified easily to incorporate new modules (perhaps a mobile application). Injecting the trained model directly into the application as a component will allow you to ensure processing happens as close to both the user interaction as well as the input parameters. Minimizing data movement and reducing latency has benefits albeit at the cost of interdependence, and coupling of systems as this approach will probably require one to recode the model or provide a wrapper so as to embed it as part of the application. 
2. Convert and Import - Conversion of  the model into a portable format like Predictive Model Markup Language (PMML) or Portable Format Analytics (PFA) may make sense if the final resting place is a data mining or analytics engine that can import models coded using PMML or PFA. This is my least favorite option. 
3. Deploy as a Service – With more applications moving to the cloud deploying your model as a Service through Google’s Cloud ML engine or AWS SageMaker provides you with a pay-per-use scalable stack.  My next blog series will actually be a Proof of Concept walk-through using AWS SageMaker demonstrating the entire machine learning pipeline. 

So that is it for now. Over the last few weeks we have covered the three phases of building a machine learning pipeline. Every phase is equally important but it all starts with data! As mentioned above next up is a practical implementation of all the concepts we’ve read.