Data Science Project on Explainable AI

One of the best decisions I’ve made this semester was to enroll myself in the most demanding professional course called Digital Consulting Project under the instructor Professor Gaurav Shekhar. This course helps students to understand new concepts like TOSCA framework, 4S, Design Thinking, The McKinsey Way of consulting, ITIL principles etc., and the group assignments involving real-life scenarios teaches different problem-solving mechanisms. Though the entire course being virtual due to pandemic, it was engaging and was a very productive experience. 

The other specialty this course provides is to work on a semester-long capstone project with an Organization. I got an opportunity to collaborate with Predactica? - An AI powered personal data scientist for business teams , an innovative startup which provides cutting edge AI/ML services to their customers and being a part this company helped me understand ins and outs of a Machine Learning data science platform. With no prior industry experience in Machine Learning, I could not only learn latest and out-of-the box technologies, but also made use of it through this project.

In this field of technology and how rapidly it changes, it is important to understand the trends early and tools available to develop and deploy new technology. I got a chance to research and develop a unique framework called Explainable AI(XAI) and its related concepts. We, as a team of 7 were responsible in creating a better explainability of a black box model where a user can input the model and expect an explanation of the model behavior, by leveraging open source tools like SHAP, LIME, Partial Dependency Plots etc., Explainable models help their users make better use of outputs of Machine Learning models and make an impact in effective decision making.

Initially, it was challenging for us to get familiar with such new concepts. However, weekly calls with the project architect, Mr. Ranjit Panda, who helped us have a better understanding of the project and its requirements and undoubtedly, the complete team effort, starting from proper project planning, dividing tasks/responsibilities, knowledge/skill sharing to implementing the project, was a rewarding and valuable experience. We also implemented an API for displaying our deliverables during the presentation.

Following are the tools and technologies used while executing the project:

SHAP - SHapley Additive exPlanations for explaining global and local interpretations and   implement visualizations such as Force plot, Summary plot etc.,

LIME - Local Interpretable Model-agnostic Explanations for interpreting local instances

PDP – Partial Dependency Plots for explaining Feature Importance

FLASK – for creating API

MySQL – for storing and retrieving the explainers and trained ML models

Overall, the course and the project were a fabulous learning with technical and professional takeaways. I feel grateful to be a part of Team Predactica, our project sponsors Ratikant Pratap Singh, Sridhar Ratakonda for their guidance and Prof. Gaurav Shekhar who’s always there to help his students, encourage and motivate them to do better. Lastly, Kudos to my extraordinary team members Aditi Dutt, Varun Banda, Om Sharan, Yash Keshan, Vatsal Nayak and Hyung Kim for not only their dedication but also from whom I learnt a lot.