Integrating Groovy with Kubernetes and Jenkins (DevOps Task 6)

Hola! so you guys might remember my DevOps Task 3 , if you haven't read it, then do give it a read, because this Task does sort of connect to Task 3.

Task 3 link: https://www.dhirubhai.net/pulse/integrating-jenkins-kubernetesdevops-task-3-naitik-shah

So, the problem statement for this task is:

Perform third task with the help of Jenkins coding file ( called as jenkinsfile approach ) and perform the with following phases:

1. Create container image that’s has Jenkins installed using dockerfile Or You can use the Jenkins Server on RHEL 8/7.
2. When we launch this image, it should automatically starts Jenkins service in the container.
3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins.
4. Job2 ( Seed Job ) : Pull the GitHub repo automatically when some developers push repo to GitHub.
5. Further on jobs should be pipeline using written code using Groovy language by the developer.

6. Job1 :  

  1. By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )

  2. Expose your pod so that testing team could perform the testing on the pod

  3. Make the data to remain persistent using PVC ( If server collects some data like logs, other user information )

7. Job3 : Test your app if it is working or not.

8. Job4 : if app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer

Let's discuss what is Groovy?

Groovy is a powerful, optionally typed and dynamic language, with static-typing and static compilation capabilities, for the Java platform aimed at improving developer productivity thanks to a concise, familiar and easy to learn syntax. It integrates smoothly with any Java program, and immediately delivers to your application powerful features, including scripting capabilities, Domain-Specific Language authoring, runtime and compile-time meta-programming and functional programming.

I will be integrating Jenkins with Groovy.

Step 1: Create a Docker image which has Jenkins installed using Dockerfile.

I will be using CentOS latest version, and we need to install sudo and wget, because they will be needed further in this task. Then we will be installing Jenkins and a suitable Java version, because Jenkins is based on Java. Since systemctl doesn't work in Centos, so we would require to use sudo service Jenkins start command. But service command isn't available in Centos latest image. So, we have installed /sbin/service We have also installed git because we'll need it later. The third last line is to give powers to Jenkins to perform operations inside the container. The, we have started the Jenkins by sudo service Jenkins start but we also need to start /bin/bash otherwise the container will close as soon as Jenkins starts because there will be no tasks left. Hence, we also start the bash in same command.

The we will exposing over port 8080, because Jenkins runs on port 8080.

Build an image from this Dockerfile using docker build -t NAME:TAG /location of Dockerfile/

Run a container from your newly built image & expose it to any available port using PAT.

Ex- docker run -it -p 2301:8080 --name jenpro jentest:v5

Before moving on building and setting up tasks on Jenkins through Groovy code we need to configure and setup some YML files for our deployment of server over Kubernetes, we will be needing them later.

So, I will be having two types of images, let me explain how?

So if we make a docker image to run html code, then what if a need arises for php image, so there will be two options HTML and php both, we can surely put both code in a one single dockerfile, but I will give an option this time, so I will be creating two separate images:

The HTML Dockerfile:

FROM centos:latest
RUN yum install httpd -y
COPY *.html /var/www/html/
CMD /usr/sbin/httpd -DFOREGROUND && /dev/bash

The PHP Dockerfile:

FROM centos:latest
RUN yum install httpd -y
RUN yum install php -y
COPY *.php /var/www/html/
CMD /usr/sbin/httpd -DFOREGROUND && /dev/bash

Step 2: The step 2 consists of creating PV(Persistent Volume) and PVC which we will be used later by our server pod.

Again it will be different for HTML and PHP:

For HTML:

apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: server-pv-vol
    labels:
      type: local
  spec:
    storageClassName: manual
    capacity:
      storage: 5Gi
    accessModes:
      - ReadWriteOnce
    hostPath:
      path: "/mnt/sdr/data/website"

Run the following command for creating PV:

kubectl create -f server-pv-vol.yml

Code for Persistent Volume Claim:

apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: server-pv-vol-claim
  spec:
    storageClassName: manual
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi

Command to claim storage:

kubectl create -f server-pv-vol-claim.yml

For PHP:

Persistent Volume:

apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: phpserver-pv-vol
    labels:
      type: local
  spec:
    storageClassName: manual
    capacity:
      storage: 5Gi
    accessModes:
      - ReadWriteOnce
    hostPath:
      path: "/mnt/sdr/data/website"

Command for creating PV:

 kubectl create -f phpserver-pv-vol.yml

Persistent Volume Claim:

apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: phpserver-pv-vol-claim
  spec:
    storageClassName: manual
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi

Command to claim storage:

kubectl create -f phpserver-pv-vol-claim.yml

Step 3: In this step we will be creating a deployment for the server:

For HTML:

apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: server-deploy
    labels:
      app: webserver
  spec:
    replicas: 5
    selector:
      matchLabels:
        app: webserver
    template:
      metadata:
        name: server-deploy
        labels:
          app: webserver
      spec:
        volumes:
          - name: server-pv-vol
            persistentVolumeClaim:
              claimName: server-pv-vol-claim

        containers:
          - name: html-deploy
            image: server:v1
            imagePullPolicy: IfNotPresent
            volumeMounts:
              - mountPath: "/var/log/httpd"
                name: server-pv-vol

For PHP:

apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: phpserver-deploy
          labels:
            app: webserver
        spec:
          replicas: 5
          selector:
            matchLabels:
              app: webserver
          template:
            metadata:
              name: phpserver-deploy
              labels:
                app: webserver
            spec:
              volumes:
                - name: phpserver-pv-vol
                  persistentVolumeClaim:
                    claimName: phpserver-pv-vol-claim

              containers:
                - name: phpserver-deploy
                  image: phpserver:v1
                  imagePullPolicy: IfNotPresent 
                  volumeMounts:
                    - mountPath: "/var/log/httpd"
                      name: phpserver-pv-vol

Step 4: In this step I will be creating a service to expose the pod to the outside world using type = NordPort.

apiVersion: v1
kind: Service
metadata:
  name: expose
spec:
  type: NodePort

  selector: 
    app: webserver

  ports:
    - port: 80
      targetPort: 80
      nodePort: 2301

Now the time has come, to move Groovy.

Step 5: Task 1, Production:

This will download the code from the mentioned GitHub repo, the extension will be checked to identify whether the code is in HTML or PHP. Then, it will pushed to the suitable image to docker hub so that it can be used later.

job("production") {
        steps {
        scm {
              github("sparshpnd23/face-recog-by-transfer-learning", "master")
            }
        triggers {
              scm("* * * * *")
            }
        shell("sudo cp -rvf * /sparsh")
        if(shell("ls /sparsh/ | grep html")) {
              dockerBuilderPublisher {
                    dockerFileDirectory("/sparsh/")
                    cloud("docker")
        tagsString("server:v1")
                    pushOnSuccess(true)

                    fromRegistry {
                          url("sparshpnd23")
                          credentialsId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
                    }
                    pushCredentialsId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
                    cleanImages(false)
                    cleanupWithJenkinsJobDelete(false)
                    noCache(false)
                    pull(true)
              }
        }
        else {
              dockerBuilderPublisher {
                    dockerFileDirectory("/sparsh/")
                    cloud("docker")
        tagsString("phpserver:v1")
                    pushOnSuccess(true)

                    fromRegistry {
                          url("sparshpnd23")
                          credentialsId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
                    }
                    pushCredentialsId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
                    cleanImages(false)
                    cleanupWithJenkinsJobDelete(false)
                    noCache(false)
                    pull(true)
              }
        }
        }
        }

Task 2: Deployment

This Jenkins task will deploy the code using the suitable container & the Kubernetes deployment created above.

job("Deployment") {


      triggers {
        upstream {
          upstreamProjects("Production")
          threshold("SUCCESS")
        }  
      }


      steps {
        if(shell("ls /sparsh | grep html")) {


          shell("if sudo kubectl get pv server-pv-vol; then if sudo kubectl get pvc server-pv-vol-claim; then echo "volume present"; else kubectl create -f server-pv-vol-claim.yml; fi; else sudo kubectl create -f server-pv-vol.yml; sudo kubectl create -f server-pv-vol-claim.yml; fi; if sudo kubectl get deployments server-deploy; then sudo kubectl rollout restart deployment/server-deploy; sudo kubectl rollout status deployment/server-deploy; else sudo kubectl create -f web-deploy-server.yml; sudo kubectl create -f webserver_expose.yml; sudo kubectl get all; fi")       


      }


        else {


          shell("if sudo kubectl get pv phpserver-pv-vol; then if sudo kubectl get pvc phpserver-pv-vol-claim; then echo "volume present"; else kubectl create -f phpserver-pv-vol-claim.yml; fi; else sudo kubectl create -f phpserver-pv-vol.yml; sudo kubectl create -f phserverp-pv-vol-claim.yml; fi; if sudo kubectl get deployments phpserver-deploy; then sudo kubectl rollout restart deployment/phpserver-deploy; sudo kubectl rollout status deployment/phpserver-deploy; else sudo kubectl create -f web-deploy-phpserver.yml; sudo kubectl create -f webserver_expose.yml; sudo kubectl get all; fi")


        }
      }
    }

So, now go and check that whether your code is deployed or not, the Task 3 is all about testing.

Task 3 - Testing:

This task will test the code & if the code is found corrupt, it will automatically send an email to the mentioned email of the developer.

job("Testing") {

    steps {

        shell('export status=$(curl -siw "%{http_code}" -o /dev/null 192.168.99.102:2301); if [ $status -eq 200 ]; then exit 0; else python3 mail.py; exit 1; fi')
      }
    }

In monitoring terms, we have the best service running(Kubernetes), so as soon as something goes wrong, Kubernetes will launch replica pods.

And it might seem that this task was full of code, but when you go to the real world(the real Developer world, no one likes using GUI, everyone works on commands, and so doing this task will give you a pretty good idea about using commands, and it will make you more comfortable with commands.

So I hope you enjoyed doing this task, suggestions are welcome, see you next time.