jenkins

his post is a follow up to our recent article focused on running?Jenkins Docker?in an HA configuration using Docker Swarm. ?Given the popularity of that post, we wanted to dive into Jenkins a little more and share some tips on speeding up Jenkins building. ?Why

CI/CD (Continuous Integration / Continuous Delivery) is one of the pillars of the modern DevOps tool chain. CI/CD, as exemplified by?Jenkins, automates the back-end of software development (building, tooling, and testing) prior to software release promotion. ?Jenkins is an open-source automation server created by?Kohsuke Kawaguchi?and written in Java. ?A Jenkins build?can be triggered by various means, the most common being a commit in a version control system like Git. ?Once a Jenkins build has been triggered, a test suite runs, enabling developers to automatically test their software for bugs prior to releasing to production. By some estimates, Jenkins has up to 70% of marketshare for all CICD tools, making its use today ubiquitous. But like all software, developers are always looking for ways to speed it up. ?If you are using Jenkins but want to accelerate your pipelines, this post answers the question “how do I speed up Jenkins builds”?

Background on Jenkins architecture

Jenkins architecture revolves around master / slave roles. Standard Jenkins deployment models were based initially on bare-metal servers and VMs, and like many aspects of today’s DevOps environments are now moving to containers.

The conventional use of storage in Jenkins has both pros and cons:

Pros:

Jenkins has a simple and stateless model.
Jenkins makes no assumptions regarding initial state.
Jenkins initiates every build from scratch.
Jenkins?can create n slaves when you need to scale.

Cons:

Jenkins has relatively poor utilization in scaleout models involving lots of slaves.
Each slave instance must build its pipeline from scratch, which typically involves a complete repository clone, followed by a complete compilation/build.
If you don’t build your pipeline from scratch, incremental builds can fail due to subtle changes in starting state?        

Where Portworx fits and wh

Portworx provides remarkable benefits for containerized Jenkins models. Following are Portworx-specific benefits for common use cases:

Shared volumes: In this model, there is one master and one slave. The advantage of using a Portworx shared volume is similar to the advantage of NFS: easy transfer of state between master and slave. Furthermore, this shared volume model works equally well, regardless of whether Jenkins is running containerized, in a VM, or on bare metal.
Faster incremental builds: Typically both slaves and their data are ephemeral, with data and state discarded when the slave exits. However, if the slave makes use of two different volumes (“build” and “artifact”), then a slave can exit while preserving its artifact repository, to help accelerate subsequent, or incremental builds.
Monolithic master: slaves are optional. Smaller Jenkins environments with fewer resources and requirements may opt to simply run only a single monolithic master. In the conventional model, all data/state on the master will be lost when the master exits. With Portworx, data persists in this model, allowing the master to exit, yet quickly restart when needed. When monolithic masters run in an AWS/Cloud, compute charges are only paid for (for example on “m4.16xlarge”) when needed.
Highly parallel / fully distributed: Without Portworx, the master delegates, and each slave does its own complete build and test cycles. With Portworx, only the master performs the build cycle, followed by multiple test cycles all performed in parallel by multiple slaves. In this most powerful model, the master takes multiple volume snapshots when it completes its build cycle, and then assigns a read/writeable snapshot for each slave to use as its own private volume for their respective test cycles.y