GitOps for Kubernetes

This post was originally published here on Caylent.com.

Cloud-native applications are dominating the market with their improved performance and high efficiency. While there are more resources to support cloud-native applications running as microservices, managing complex cloud architecture is still a challenge. The more microservices you run, the more tasks you will have to deal with in order to keep the cloud environment healthy and running smoothly.

Automation becomes an obvious solution to the problem. Kubernetes in particular, is now supported by new approaches like Infrastructure as Code and a wealth of automation tools. Still, the CI/CD cycle of today demands more robust and agile architecture. This is where GitOps for Kubernetes comes in handy.

What Is GitOps?

GitOps is a new approach to Continuous Deployment that leverages Git as a single source of truth for declarative infrastructure and applications, providing both revision and change control. With GitOps, a system is run by submitting pull requests (and subsequent merges) to achieve the desired state of the system represented in a Git repository.

GitOps with Kubernetes provides next level of infrastructure-as-code and immutable infrastructure through declarative continuous delivery systems. The declarative style of configuration and active reconciliation model extends the platform’s core benefits for the deployment, monitoring, and lifecycle management of Kubernetes applications.

The framework is designed to integrate well with any CI/CD pipeline, even when you utilize third-party tools to manage your pipeline natively.

In fact, GitOps is more of a workflow or an approach than an actual tool. It is an approach that can help you optimize your CI/CD cycles without having to alter the way you develop and deploy microservices to the cloud.

Since GitOps is developed as a framework for the Continuous Delivery of cloud-native applications, you can rely on the framework to simplify certain processes within your pipeline. It integrates Git—a tool we are already familiar with—and CD tools seamlessly.

In essence, GitOps combines the traditional CI/CD pipelines with Git workflows, allowing for better end-to-end management of Kubernetes and app development. The two are treated as a unified process rather than separate ones.

GitOps is characterized by:

• Best practices across deployment, management, and monitoring of containerized clusters and applications
• A developer-centric experience for managing applications, where fully automated CI/CD pipelines/workflows using Git are used for development and operations
• Use of the Git revision control system to track and approve changes to the infrastructure and runtime environment of applications

Benefits of GitOps

From the definition, it is easy to see how GitOps can bring a lot of benefits to the table, starting with improved developer experience. Developers no longer have to worry about packaging their codes in containers and working with Kubernetes clusters. Instead, they can focus on their codes, relying on familiar tools like Git to handle everything else.

This is a definite improvement to developer experience. Even with DevOps specialists helping with the process, application deployment is still a bottleneck in many cases, simply because there is a need for well-packaged codes and microservices for easy deployment. GitOps with its unconventional approach solves that bottleneck.

Better stability of apps can also be expected when the approach is implemented. Git has a detailed logging system that allows for easy auditing and tracing, and you benefit from the same logging tool when you utilize GitOps as an approach. Changes to the Kubernetes cluster are recorded in the log, making cloud audits a lot easier to manage.

Even details such as errors and crashes are traceable. Details such as who commits the changes and the results of those changes are recorded every step of the way. Whether you are trying to comply with security best practices or pursuing SOC 2 compliance, the utilization of Git logs certainly makes maintaining a stable system easier. 

Most organizations have to invest substantially in their processes and systems in order to be compliant and auditable. With GitOps and Kubernetes, most of the compliance and auditability requirements can be satisfied with minimal effort. Read more about Achieving SOC 2 Compliance with us here.

There is also the fact that GitOps helps development teams improve productivity. One of the more prominent paradigms to emerge from the DevOps movement is the model of declarative systems and configuration. Simply put, with declarative models, you describe what you want to be achieved, as opposed to how to get there. 

The elimination of infrastructure-related tasks is a huge help too. Many developers—despite the traditional DevOps approaches—are not fully comfortable with managing their cloud infrastructure as code. There is still a level of complexity involved in the process.

Traditionally, infrastructure maintenance tends to become a bottleneck rather than a supporting function. Many development teams ended up dividing the tasks of managing infrastructure with the main CI/CD pipeline, causing further inefficiencies in the process. GitOps eliminates the bottleneck completely without altering the pipeline.

GitOps also contributes to the better security and standardization of a cloud environment. Code changes are pushed to Git before it is processed further. Additional security measures such as code checks against security standards can be put in place. It is also a common practice to integrate a staging server into the process.

A single source of truth is a role taken over by Git in this scenario. Whenever a security issue affects the system, you can always find the root cause by reviewing the Git repository. Alerts and logs are automatically sent to Git whenever changes are made to the Kubernetes cluster directly, adding a layer of security to the approach.

In addition, GitOps aids in the recovery of infrastructure environments by storing its declarative specifications of the environment under source control as a source of truth. By having a complete definition of what the environment should be (the single source of truth), it facilitates the re-creation of the environment in the event of a disaster.

Lastly, there is the fact that GitOps makes the whole system more reliable. You can fork and revert changes by controlling the Git repository. The same Git commands you use to manage codes and branches can be used to manage the entire cloud environment. When something goes wrong, rolling back to a certain snapshot is easy.

GitOps also adds complex automation to the mix. Everything that you can automate in Git can be implemented in GitOps. At the same time, the approach itself automates the management of your Kubernetes cluster through simple commands. It is a method that truly meets the needs of modern cloud-native applications.

Basic Principles of GitOps

Before we get to how you can implement GitOps as an approach, it is also necessary to take a closer look at the basic principles of this approach. 

1. Everything about the infrastructure is described declaratively. All server configurations are defined as facts?—and fully incorporated into the Git repo?—for maximum reliability. Declarative also means application declarations are versioned thoroughly.

2. System states are versioned in Git. This is one of the main reasons why recovering from an error with GitOps is very easy. You can roll back to a certain version in a consistent way. Rather than spending hours on disaster recovery, you only need to spend minutes.

3. Changes need to be approved, but approved ones can be implemented in the system. There is no need for a separate deployment workflow or pre-packaging of new codes before implementation. Everything happens automatically.

4. Correctness and security are maintained as parts of the process. There is no need to incorporate a separate security and integrity workflows into the CI/CD pipeline. Infrastructure as a Code is enhanced to a whole new level.

The GitOps Workflow

GitOps is beautifully simple in how it handles changes. The process starts with new codes being pushed to Git for further review. Once the new codes are cleared and approved, they are then merged to Git. This is where an action for deploying new codes is triggered.

Git will trigger the creation of new images by working together with your favorite CD pipeline tool. The process is fully automated, so you only need to define the parameters once. Combined with tools like AWS CodePipeline and AWS CodeCommit, the possibilities are endless. Not an AWS user? Integrate Google Cloud Build’s continuous deployment with Jenkins’ continuous integration tools. Or Cloud Build’s CI can link to Google’s Spinnaker CD tool too.

A popular tool to use is Flux. Flux works seamlessly with Amazon EKS for a truly scalable cloud environment. CodeBuild can be used to push new codes to ECR, after which Flux will automatically make the necessary changes to your EKS cluster.

What Is Flux?

Flux automates the deployment of containers to Kubernetes. It fills the automation void that exists between building and monitoring.

Flux’s main feature is the automated synchronisation between a version control repository and a cluster. If you make any changes to your repository, those changes are automatically deployed to your cluster.

Another feature is deployment automation for containers. It will continuously monitor a range of container registries and deploy new versions where applicable.

This is really useful for keeping the repository and, therefore, the cluster up to date. It also allows separate teams to have their own deployment pipelines since Flux is able to see the new image and update the cluster accordingly. This feature can be disabled though too and images can be locked to a specific version.

Integrations with Other Devops Tools

For configuration customization across environments and clusters, Flux comes with builtin support for Kustomize and Helm.

The Flux Workflow Diagram

(fluxcd/helm-operator-get-started, 2020)

Flux will monitor all of the container image repositories that you specify. It detects new images, triggers deployments, and automatically updates the desired running configuration of your Kubernetes cluster—and does so within the bounds of a configurable deployment policy.

How to Install Flux

If you’re using Kubernetes and Helm already then follow this in-depth tutorial by Flux for how to get started here. The tutorial is current for Flux version 1.18.0.

Conclusion

As you can see, the actual steps involved to set up Flux, deploy an app, give Flux access to it, and see modifications complete are very straight-forward. 

Many developers are already leveraging GitOps and realizing the benefits. It is an approach that will empower Kubernetes as a cloud platform even further. Expect to see GitOps being implemented across more development projects in the future.

Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.

References

GitHub. 2020. Fluxcd/Helm-Operator-Get-Started. [online] Available at: <https://github.com/fluxcd/helm-operator-get-started/blob/master/diagrams/flux-helm-operator-registry.png> [Accessed 11 March 2020].