MLOPs monitoring Solution

Really glad to showcase my work/POC for Model Monitoring using Data Bricks and Microsoft Azure ML.

Problem Statement/s

? We want to enable Model Monitoring for our UCs without hampering our Architecture Blueprint

? We have tried third part libraries like AI Evidently and Alibi Detect but it was too code intrinsic and manual work. Also it didn’t support huge data sets.

? We also have special requirement of comparing two training datasets for drift in dev environments

?MLOPs monitoring framework was required which can send emails/alerts upon drift detection and visually appealing.

Proposed Solution

? We have developed the whole solution using DataBricks and AZML service of Microsoft. This will be a generalized solution for all UCs whereby we can detect the drift between scoring data & training data, graphical insights, application insights & send emails to the stakeholders for their actions.

? I have also developed a generalized template to compare two training datasets as a special requirement from our stakeholders. It will be a similar solution as mentioned above.

?I have also given the inference data back to Data Bricks if required for any visualization in a simple tabular format.

User Impact

?There wasn't any impact to existing users/roles pertaining to any solution/s.

?This implementation is an extension to existing blue print which uses DataBricks, MLFlow and AKS as protagonist solution components.

?We need to setup the AZML service coupled with Data Bricks workspace, whereby Data Drift Monitor will be a visual delight from Microsoft

Final Product:-

Summary:- This is ready to use, template driven and low code solution for monitoring ML Models. Please feel free to criticize, ask Qs or suggestions for betterment.