MLOps becomes mainstream - Five main trends at Big Data Tech Warsaw 2021 - Part 1/5.

Each day this week I'll share one of the five Big Data trends that will be covered in detail by numerous presentations at the forthcoming edition of Big Data Tech Warsaw (February 25-26th, 2021).

Trend 1.

MLOps becomes mainstream

The difficulty of building machine learning systems, especially scalable ones, was described by Google in a research paper in 2015 ("Hidden Technical Debt in Machine Learning Systems"). At that time many companies already had large-scale ML systems in production, but not many of them had actually dedicated ML platforms or tools built to support the end-to-end life-cycle of their ML models and the daily work of their ML teams.

The MLOps trend became more visible a few years later when a number of well-known companies started sharing information about their in-house ML stack. This particularly includes Facebook (FBLearner Platform), Twitter, Netflix, Airbnb (Zipline) and Uber (Michelangelo).

MLOps-related presentations at Big Data Tech Warsaw

A year ago we had a number of great MLOps-related presentations at Big Data Tech Warsaw 2020 given by speakers from companies such as Spotify, Disney+, Synerise. These companies were a part of the Data Science & ML track last year.

This year, however, we see enormous interest in this topic and we decided to curate a separate track called MLOps. Here are examples of presentations that belong to this track:

• We will hear about the MLOps journey at H&M in the public cloud presented by Keven(Qi) Wang. He will cover their entire MLOps stack that has been adopted by multiple product teams managing 100s of models across the entire H&M value chain. It enables data scientists to develop models in a highly interactive environment, enable engineers to manage large scale model training and model serving pipeline with full traceability. 
• Sotrender is a company that analyses a large volume of social media data. Maciej Pieńkosz will talk about their ML use-cases and GCP components they use (e.g. AI Platform Notebooks, AI Platform Training, Cloud Run, Gitlab CI/CD), covering the full lifecycle of the ML model, from experimentation, through training and deployment, to model monitoring.

• Everyone who works in IT (and especially with Big Data projects powered by open-source technologies) knows an Australian company called Atlassian. Jiamei Du will talk about how they use A/B experiments to build better products. Part of her talk will presentation will describe their MLOps tools and infrastructure to make their A/B experiments as efficient as possible.
• My colleagues at GetInData build portable and reusable ML platforms in various environments (cloud, hybrid, on-premise) using a mix of open-source and cloud-based technologies. They will share their experience and best practices that come from a number of production deployments.
• We will also hear how NoMagic robots improve iteratively and continuously thanks to the software 2.0 improvement cycle supported by an in-house data engine. Watch this short video below to cheat to see what type of robots they learn using ML/AI.

There are only a few highlighted examples, but you will definitely learn about MLOps at Big Data Tech Warsaw 2021 (February 25-26th).

What's next?

On Tuesday, I will share the second trend. Please stay tuned!

In the meantime, I encourage you to check our agenda and register before January 15th to take advantage of New Year Promotion (link).

As you might expect, this year the conference will be organized in form of an online interactive. Please check my recent blog post that explains how COVID-19 changes Big Data Tech Warsaw 2021 but makes it greater at the same time.

If you like this post, please give us a like, or share it or leave a comment. Thanks!