How do you scale up and deploy LSTM models in production environments?

I think it is worth defining what LSTM stand for our readers. LSTM models are long short-term memory networks mostly used in deep learning. A key feature of an LSTM model is feedback connections. The model can process an entire sequence of data.

I think it’s important to make a distinction between model optimization for performance and efficiency. Most of the techniques mentioned above (hyper-parameter tuning, architectural optimization, backpropagation algorithm tuning) primarily applies to performance optimization. On the other hand, model optimization for efficiency has a different objective since it typically refers to how fast a model makes an inference, its size in memory, its adaptability on different platforms. As such model efficiency is directly related to its complexity (ie., the number of parameters, its memory size etc). To make the models more efficient we typically try to reduce complexity, for example, by using simpler architectures, or quantizing its parameters.

It’s important that the first thing you do is separate your test dataset immediately from the rest of the data. If not, it’s really easy to “contaminate” your training and validation data with test data. This is called a data leakage. You will then get high scores on your metrics which will be false positives because your model has a “cheat sheet”.

It's important to add here that the test data should be representative for the problem we're trying to solve. It should be large and diverse enough to build confidence in the trained model.

I agree and would also mention that you need to create api calls so that a user can run the model and interact with it (give new training data or get predictions).

Many companies run their communities under many great platforms such as Microsoft and come with pre built bots with the foundations to help you structure your employee and production data easily and quickly. Having the bots on demand giving everyone the same data I real time could keep your employees on the same knowledge gaps and focused on the same major issues. I see the potentials and experimented with a few ideas. The framework and possibilities are endless.

We also need to consider deployment of models on edge devices as this is a very common use case. For example, we may need to deploy a face recognition model as part of a smartphone app, or activity monitoring model to a wearable device. These types of deployments through are generally platform dependent.

It would be great to incorporate in these discussions model optimization and deployment for edge devices. The above discussions are generally addressing the model optimization/deployments on cloud platforms however there's whole other use cases for ML on edge. I tried to incorporate model optimization/deployment considerations for edge, however it'd be nice to include them in overall discussion.