INTEGRATION OF MACHINE LEARNING WITH DEVOPS

More than 50% of Machine Learning models are not implemented due to the tedious task of manually training the model again and again to get the better accuracy. Ever wondered if this could be automated how much we could save our time and energy!!!

This article is about integrating Machine Learning Models with Devops tools -Docker, Jenkins and Git that will automatically train the model and find best accuracy just after developer will update the code.

PROBLEM STATEMENT

1. Create container image that’s has Python3 and Keras or numpy installed using dockerfile 

2. Create a job chain of job1, job2, job3, job4 and job5 using build pipeline plugin in Jenkins 

3. Job1 : Pull the Github repo automatically when some developers push repo to Github.

4. Job2 : By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter install image container to deploy code and start training( eg. If code uses CNN, then Jenkins should start the container that has already installed all the softwares required for the cnn processing).

5. Job3 : Train your model and predict accuracy or metrics.

6. Job4 : if metrics accuracy is less than 80% , then tweak the machine learning model architecture.

7. Job5: Retrain the model or notify that the best model is being created

8. Create One extra job job6 for monitor : If container where app is running. fails due to any reason then this job should automatically start the container again from where the last trained model left

After explaining each job I have attached their respective outputs.

1) CREATING IMAGE

We create a dockerfile where we use centos image as base and install keras, tensorflow and numpy libraries which will be used in python code.

We create this image using command : docker build -t mytensor:v1 .

2)DOWNLOADING CODE FROM GITHUB AND SAVING IN BASE SYSTEM

After the developer uploads or updates the pre-existing code , jenkins automatically download the code from github. We use pollSCM in this job that will keep checking the github code every minute and download the code as soon as there will be any changes in code.

This downloaded code will be saved inside /var/lib/jenkins/workspace/accuracy folder in my base system(redhat linux 8).

3) LAUNCHING CONTAINER

Next Job is to launch the container , train the model and find accuracy. It will check whether the code is already running , if not, it will launch a new container.

Given below is my python code . I use i as hyper parameter that keep increasing number of convolutional and maxpooling layers whenever we re-train the model to increase accuracy. At the end we are saving accuracy in "accu.txt" file.

Output:

4) CHECKING ACCURACY

In this job we are checking accuracy . If accuracy is greater than 80 ,then its great, else we re-train the code to get desired accuracy.

5) SENDING NOTIFICATION

This job will send email to developer in case the code is not working or if the previous jobs failed due to any reason.

For this we have to first install email plugin in jenkins and configure it.

After configuring we will add in our job under post-build actions.

6) MONITORING

In our final job we will be monitoring whether our model is working fine on client's side. If due to any reason it fails , this job will automatically launch a new container.

THE ENTIRE BUILD PIPELINE SHOWING TIMELINE

With this we come to the end of the article.

THANK YOU