Simple CI/Deployment Approach with Jenkins
In this article i will try to explain a simple but effective CI/CD approach, detailing the why behind, with complementary examples and diagrams.
Off course there are lots of different approaches possible, depending on the company, projects, architecture and infrastructure, but this one tries to make simple but useful examples.
Pre Requisites
- Knowledge about AWS
- Knowledge about docker and lambda
- Knowledge about jenkins
Goals
- Have CI pipelines to test, build and push automatically without human interaction, only being triggered by a push to code repository (with bitbucket webhooks)
- Have centralized image repositories, set up on Prod account
- Have Deployment pipelines for each environment that will be a one click deployment where the version will be specified
- Separate between infrastructure management, and service/components version deployments/management
Assumptions and General definitions
- Workflow will be based on the repository practices, in this case, Git Flow, which means that we will have two immutable branches: master and develop
- Releases to deploy are tagged following semver
- Each environment (dev, staging, prod) will have their deployment pipelines in the same jenkins installation
- The ECS service will point to a fix tag, one per each environment
- The ECS Service will be setup so there is no downtime.
Approach
As said in the goals description we will separate between:
- CI pipelines
- Deployment pipelines
CI
The CI pipelines will be in charge of:
- Testing
- Compiling/Building
- Pushing
We won't implement a Continuos Deployment approach due to the QA needed approval.
This pipelines will:
- Be triggered by a push to master branch of code repository (bitbucket webhooks pointing to jenkins)
- Will run automatically
- Will use centralized images repository located on Prod environment
ECS Pipeline steps:
- Check: to see if version already exists
- Test: to run unit or integration tests (Not implement in the examples but is only an extra step)
- Build: build/compile
- Push: push to ECR
- Cleanup: to clean unused or untagged images
Lambda Pipeline steps:
- Check: to see if version already exists
- Test: to run unit or integration tests (Not implement in the examples but is only an extra step)
- Zip source code
- Upload to S3
Deployment
The deployment pipelines are the ones that defined which version will be running on each environment.
As said before each environment will have their own pipelines to deploy.
On every environment pipelines the steps will be the same:
- User will need to specify version to deploy
- Pipeline will check if the version exists on centralized repository
- If exists then is tagged as latest with the environment specific tag
- The ECS Service is restarted so new containers are started and old ones are killed, WITHOUT DOWNTIME
For Lambda Functions the process will be pretty similar but with some changes changes due to services specifications.
Steps for ECS:
- Tag: check if version exists and tag it to be current version
- Deploy: force ECS Service restart
Steps for Lambda:
- Deploy: point lambda function to specify zip version file
Diagrams
ECS CI/Deploy
Lambda CI/Deploy
Examples
Ok now is time for real Scripts examples, so here they are.
ECS CI: https://github.com/juanip84/jenkins-ci-deploy/blob/master/ECS/CI/Jenkinsfile
ECS Deploy: https://github.com/juanip84/jenkins-ci-deploy/blob/master/ECS/Deploy/Jenkinsfile
Lambda CI: https://github.com/juanip84/jenkins-ci-deploy/blob/master/Lambda/CI/Jenkinsfile
Lambda Deploy: https://github.com/juanip84/jenkins-ci-deploy/blob/master/Lambda/Deploy/Jenkinsfile
Base repository: https://github.com/juanip84/jenkins-ci-deploy
Conclusion
There's lots of possible ways of doing CI, CD, and Deployment, but i hope this one helps someone, and clarifies some concepts involved.
See you next time!
Fintech specialized SWE | Blockchain | Payments | xFT's | Web3
5 年Muy bueno che! bitbucket en particular ofrece un CI y unos pipelines muy buenos, dockerizables y que pueden correr cosas bastante custom, para un approach minimalista es una linda alternativa a tener una instancia con un jenkins dedicado, claro, con la desventaja que una eventual migración a github seria un poco mas costosa :p
DevOps Engineer @ Persefoni
5 年Excelente artículo Juan! Es un approach similar al que tenemos implementado nosotros ahora. Te consulto, están usando slaves de Jenkins para la ejecución de pipelines o sólo el máster de Jenkins con un par de executors? Gracias!
Engineering @ VieMed
5 年Muy bueno Juan Ignacio Paz?????
Hands-On Fractional CTO | GenAI, SaaS & Cloud
5 年Buenísimo Juan! Están usando algo de aws sam?
Tú Socio en innovación Digital
5 年el 1 Juan Ignacio Paz