Hyperparameter Tuning using MLOps

The percentage of AI models created but never put into production in large enterprises has been estimated to be as much as 90% or more. With massive investments in data science teams, platforms, and infrastructure, the number of AI projects is dramatically increasing — along with the number of missed opportunities.

I made this project by Integrating Machine Learning with DevOps. Technologies used : Git , Github , Jenkins , Docker , Keras , Tensorflow , Python

Task description

1. Create container image that’s has Python3 and Keras or numpy installed using dockerfile 

2. When we launch this image, it should automatically starts train the model in the container.

3. Create a job chain of job1, job2, job3, job4 and job5 using build pipeline plugin in Jenkins 

4. Job1 : Pull the Github repo automatically when some developers push repo to Github.

5. Job2 : By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter, run docker container to deploy code and start training.

6. Job3 : Train your model and predict accuracy or metrics.

7. Job4 : if metrics accuracy is less than 95% , then tweak the machine learning model architecture.

8. Job5: Retrain the model or notify that the best model is being created

Creating Docker Image

I used Dockerfile to create a Docker image with Python , Keras and TensorFlow installed. Also, as I used Mnist dataset for training so I downloaded that too in the image.

Here, I used "CMD python3 mlops.py" to run the python code and start training the model as soon as user runs the Container.

Creating Job Chain Using Jenkins

• Job 1

Job 1 was to Pull the Github repo automatically when some developers push repo to Github. As i wanted complete automation , I used post-commit hook of git (Pushes the code the github and runs Jenkins Task of Pulling the code as soon as the user commits the code)

#!/bin/bash


git push  #pushes the code the github

curl --user "admin:root" https://192.168.99.101:8080/job/Git%20Clone/build?token=job1  #Runs Jenkins task using Build Triggers

In the Jenkins Job, I copied the pulled repo from Github to /root/mlhp/. We will mount this volume to Docker container.

• Job 2

This job included running docker container to deploy code and start training. This job runs automatically if Job1 is stable.

As we can see, the code searches for container named "mlops_container" . If the container is stopped previously, we remove it and run another container using the same name.

sudo docker run -t --name mlops_container -v /root/mlhp/:/model/ mlops:v1

As soon as we run this Job, the docker container starts training the model.

• Job 3

Job 3 was Training the model and predicting accuracy. I used Random module of python for Hyperparameter tuning to improve the accuracy of the model.

The above code adds random number of Convolution layers containing random filters and random kernel size. The Dense layer also contains random activation functions. At the time of compilation I used random optimizers (RMSprop, Adam, SGD, Nadam, Adamax). I used accuracy as metrics to show the accuracy at the time of epochs. the model gives different accuracy every time we run it as it uses random layers.

• Job 4 & Job 5

These Jobs included tweaking the machine learning model architecture if metrics accuracy is less than 95% and Sending an Email to the user if metrics accuracy is greater than 95%.

I used smtplib module for sending the mail and os module to re-train the model.

As soon as the curl command accesses this site, Job2 runs again and Retrains the model, If the New accuracy is more than 95% then the program Sends a mail to the user telling the present Accuracy and the Layers used to achieve that accuracy.

However, If the New accuracy is again less than 95%, then the programs runs another Docker container and re-trains the model using different activation function and layers. This process continues until we achieve the desired accuracy.

Click Here for Github Repo