Containerize your Jenkins "build executors" for distributed builds

Jenkins as a build/CI/automation tool helps to perform our day to day DevOps tasks, which involves builds, unit+integration testing, Code analysis etc.

For execution of above tasks, Jenkins provides a little but powerful "Build Executor"(thread) to run our Jenkins jobs.

Lets say I have 100 jobs that needs to get build at the same time (say parallel execution). Jenkins needs to spin up 100 threads for this execution which obviously will use extensive amount of underlying resources.

As per my experience, I have seen Jenkins getting hung up after executing 20 parallel build executors.

For this type of a scenario, why shouldn't we containerize our build executors??

Well that means at any point in time I can create a docker container of my executor and once the build completes, It gets teared down. And yes we already know the advantages of containers.

Thanks to Google Cloud for such a unique solution that gave me more control over my builds and deployments.

Ok!! So without wasting any time lets proceed......

Once Jenkins is up and running, install " Kubernetes Plugin " in Jenkins.

In Jenkins, click  Manage Jenkins in the left navigation. Then click  Configure System

Enter 0 for  # of executors. Setting this value to 0 ensures that all builds run in newly provisioned containers instead of the Jenkins master.

Scroll to the bottom of the screen, then click the  Add a new cloud drop-down and select  Kubernetes.

Enter kubernetes for the  Name field with other details as follows.

Second, Add a Kubernetes Pod Template. This section requires a slave-agent docker image that forms the crucial part of containerized build executors. We can take the official docker image of JNLP image " Docker images-Slave" or we can create our own docker image using dockerfile: reference Link: " GCP Dockerfile "

And third or optional step, you can configure "Volume" section after Kubernetes Pod template which need Docker related information