Roles and Requirement of MLOPs (Part-2)

1.?Subject matter experts

a. Role: Provide business questions, goals, or KPIs around which ML models should be framed, Continually evaluate and ensure that model performance aligns with or resolves the initial need.

b.?Requirement:?Easy way to understand deployed model performance in business terms, Mechanism or feedback loop for lagging model results that don’t align with business expectations.

2.?Data scientists

a.?Role:?Build models that address the business question or needs brought by subject matter experts, Deliver operationalizable models so that they can be properly used in the production environment and with production data, ?Assess model quality (of both original and tests) in tandem with subject matter experts to ensure they answer initial business questions or needs.

b.?Requirement:?Automated model packaging and delivery for quick and easy (yet safe) deployment to production,??Ability to develop tests to determine the quality of deployed models and to make continual improvements,??Visibility into the performance of all deployed models (including side-by-side for tests) from one central location, Ability to investigate data pipelines of each model to make quick assessments and adjustments regardless of who originally built the model.

3.?Data engineers

a.?Role:?Optimize the retrieval and use of data to power ML models.

b.?Requirements:?Visibility into performance of all deployed models, Ability to see the full details of individual data pipelines to address underlying data plumbing issues.

4.?Software engineers

a.?Role:?Integrate ML models in the company’s applications and systems, Ensure that ML models work seamlessly with other non-machine-learning-based applications.

b.?Requirement:?Versioning and automatic tests, The ability to work in parallel on the same application.

5.?DevOps

a.?Roles: Conduct and build operational systems and test for security, performance, availability, Continuous Integration/Continuous Delivery (CI/CD) pipeline management.

b.?Requirement:?Seamless integration of MLOps into the larger DevOps strategy of the enterprise, Seamless deployment pipeline.

6.?Model risk managers/ auditors

a.?Roles:?Minimize overall risk to the company as a result of ML models in production, Ensure compliance with internal and external requirements before pushing ML models to production.

b.?Requirements:?Robust, likely automated, reporting tools on all models (currently or ever in production), including data lineage.

7.?Machine learning architects

a.?Roles:?Ensure a scalable and flexible environment for ML model pipelines, from design to development and monitoring, Introduce new technologies when appropriate that improve ML model performance in production.

b.?Requirement: High-level overview of models and their resources consumed, Ability to drill down into data pipelines to assess and adjust infrastructure needs.

Reference: Introducing MLOps How to Scale Machine Learning in the Enterprise: Mark Treveil and the Dataiku Team