Modelops 2022: the state of practice

How AI-focused corporate leaders are implementing best practices to ensure business success while governing and scaling their AI initiatives

Introduction to ModelOps

ModelOps help organizations to implement AI solutions, organize a path to operationalize AI at scale as well as integrate DataOps, ITOps and DevOps proficiency. As core enterprise capabilities, ModelOps includes the processes, the operations, the tools and the technologies that companies can use to deploy to monitor and even govern their machine learning models.

In ModelOps, predictive analytics and machine learning workflows are made operational, put to work and used to impact an organization, so that teams waste less time on tasks that should be done elsewhere and instead focus on the things that really matter to them. Modelops is undoubtedly an enterprise capability that allows everyone across the organization to be involved in the collaboration of AI and machine learning solutions with the aim to work together to maximize the potential of AI.

Your AI transformation is doomed without ModelOps — Forrester

As defined by Gartner, ModelOps is a set of practices that seek to automate a common set of operations that arise in data science projects, which include model training pipelines, version control, data management, experiment monitoring, testing and distribution. It aims to make all predictive analytics, machine learning, and AI models operational.

To put it simply, ModelOps involves taking predictive analytics and machine learning workflows and putting them to work, making them work actually, and using them to make an impact on day-to-day operations, rather than just providing insights. This is really about the difference between having static insight versus being a part of an organization that moves around and affects the way things happen.

You might also like to read Predictive maintenance in industry 4.0: applications and advantages

ModelOps: an Enterprise-Level Capability

Artificial Intelligence is maturing and becoming an enterprise-wide function, support capabilities that are consistent and reliable are needed to help it succeed. Data science and advanced analytics are topics we have been discussing for a very long time. Traditionally data science has been considered mostly in terms of creating and deploying individual models, however, if we take one step back and look at the whole picture we might be surprised to understand that models development is only a very small piece of the puzzle.

• The buildup of undeployed models can eventually have a negative impact on the company’s growth;
• models require complex retraining;
• each new line of business requires a new set of data.

Now with the scale at which we’re doing data science, there is a huge need around operationalizing monitoring and governing these models, which is how ModelOps fits in.

Gartner stated that through 2023, at least 50% of IT leaders will struggle to move their AI predictive projects from proof of concept to production maturity.

You might also like to read AI Act: A Risk-Based Policy Approach for Excellence and Trust in AI

To innovate in business, enterprises need ModelOps at the core of their AI strategy, because it helps converge different AI artefacts, platforms, and solutions, as well as ensuring scalability and governance. Essentially, ModelOps, like DevOps, is undoubtedly another equally powerful emerging capability, which can give the organization of the so-called Enterprise AI a competitive advantage too.

A 24/7 operation environment is therefore necessary, as most data scientists use open-source modelling tools like Jupyter notebook or R studio. Most data scientists furthermore aren’t aware of or don’t have access to environments where latency can be observed. Embedding AI in enterprise systems is only possible if you are able to put in place a unified strategy where each area of the business can use the tools that are most appropriate for their needs while ensuring that the outputs flow into the business in an efficient, reliable, and in compliance with all regulations.

“Models are profoundly accountable to the business, more so than traditional software. They have to go under regulatory scrutiny and compliance. A properly operating model can dramatically change the topline performance of a particular business unit. So, integration between the business units and compliance departments is critical.” — Forbes

ModelOps allows people to scale data science but at the same time track it, monitor it, govern it and so on. To achieve this, ModelOps must be under the responsibility of the CIO, this is necessary for the development of a common view of any AI operationalization process across departments and BUs, including associated processes and tools, having a model data science is not only valuable but essential for their future competitive standing, or even for survival.

You might also like to read Model Operations for Secure and Reliable AI

State of ModelOps 2022

The need for ModelOps has become apparent in 2022 for businesses seeking to operationalize AI since the leaders in artificial intelligence have reached the ‘awareness’ stage of their ModelOps journeys. Awareness of the value of ModelOps, and how it can help organizations overcome the challenges of implementing artificial intelligence solutions, which we know are not easy.

Many organizations still fail to scale AI and are aggravated by the fact that we live in a constantly evolving and changing world, where therefore data is constantly changing, and there is a need to continuously release new solutions and new machine learning models, which in turn, they must be continuously updated and revised, and be available and usable in real-time. In this highly complex scenario, ModelOps becomes a necessity if organizations are to tackle these kinds of challenges and put machine learning into production and overcome all these time, skill and resource-consuming data science activities.

Recent surveys of AI-focused executives were conducted by Corinium, in partnership with ModelOp, to gain an understanding of where teams are and where the market is going. The State of ModelOps report offers an in-depth look at big challenges, trends, and strategies that are emerging.

This represents the second annual survey about the state of model operationalization. It gives an independent view of the practices and capabilities required to operationalize AI efficiently. The research sheds light on the current status of enterprise ModelOps functions and the practices that lead to excellence when governing and scaling enterprise AI initiatives.

You might also like to read Unlocking the Value of AI in Business Applications with ModelOps

The methodology used in the research

The survey was conducted in February and March 2022 and involved 100 AI-focused leaders from US-based (70%) and European (30%) companies.

Respondents were selected from enterprises with annual revenues of at least $1.8 billion USD in the financial services (35), insurance (35), manufacturing (15) and food retail (15) sectors. These included Allianz, BNP Paribas, CNH Industrial, Deutsche Bank, JPMorgan Chase and Walmart.

Their job roles range from C-level to Directors, VPs or Heads of Department, and include VPs of Data Science, Global Heads of Risk, Heads of AI and other AI-focused executives.

The survey included 18 questions about the organizations’ AI maturity and ModelOps capabilities, as well as obstacles to model operationalization faced by companies and where they’re investing to overcome those challenges.

Then, the findings have been combined with commentary from seven industry experts to put these unique insights into the state of enterprise model operationalization into context.

Following are a few of the key highlights:

• 86% of respondents say company executives are demanding answers about the return on their AI investments — but only 38% say they can provide them.
• Only 5% of respondents say they have full visibility into the models that are in production across the enterprise.
• 100% of respondents now have dedicated budgets for ModelOps — up from 51% in 2021.
• 80% say a lack of staff with the right skill sets is creating challenges, and that number grew 10% from the prior year.

How to get and read the entire report

ModelOps is seen as a key tool for facilitating coordination and collaboration across different teams.

AI-enabled companies have realized that they must standardize and automate their ModelOps processes, and they have embraced the tools that allow them to do so efficiently. As a result, they can see their AI investments more clearly, reduce their exposure, and get a better ROI — and reduce their costs of operationalising AI. Organizations that are still on the maturity curve now have a better understanding of their path to success.

The State of ModelOps report offers compelling insights into emerging challenges, trends, and strategies. How 100 AI-focused executives from large enterprises are developing practices to ensure excellence as they govern and scale AI initiatives.

The full report is available to download, you can get your free copy here https://bit.ly/3EVlYxi

Conclusion

In the “2022 State of ModelOps Report”, there is a significant increase in awareness surrounding the need for better enterprise ModelOps practices, which are defined as the ability to govern and manage AI-based decision models, along with traditional statistical models. It includes everything that happens from the moment a model goes into production until its eventual retirement (and beyond).

You might like to read also How ModelOps Helps You Execute Your AI Strategy

References

• 2022 State of ModelOps Report
• Modelop’s website
• ModelOps for Dummies
• Gartner “Innovation Insights for ModelOps” Report
• Scale and Govern AI Initiatives with ModelOps