Kubernetes & Data Protection Hits Mainstream with Container Storage Inteface (CSI) 1.17

In this quick blog, Nivas Iyer and I focus on the importance of the growth of containers and how to address the needs for protecting these valuable environments. Exploring why these environments from a consumer point of view gives the topic of Kubernetes a more human aspect. Think about the time right before the Covid-19 pandemic required us to do more from home. Kubernetes adoption was rapidly becoming a part of our daily lives without our even being aware of it. Kubernetes is at the heart of most digital transformation projects, helping companies, large and small to deliver amazing customer experience.

It helps everything along its digital journey; whether it is ordering that cup of coffee on the way to the office or streaming your favorite video content. There are so many different examples about of how Kubernetes adoption has really enriched our lives before, but now it has become ubiquitous.

Now, in the last few months, many of us might be ordering that coffee, dinner, groceries, all from our phones or tablets. This is Kubernetes in action. In a way, Kubernetes is helping us maintain social distancing, but also making it easier for us to manage navigating what our needs are. In a way we are forced to be a part of this new paradigm and it is helping us to be safe. Enabled by technology, we now have to think about using online freedoms in different ways, and we are even more dependent on availability and speed of access. How these constructs continue to expand and operate is a result of the flexibility that containers provide, and with robust data protection replication options; running new systems from dev/test to production really opens up a lot of companies to expand their online capacity to deal with the weight of online needs consumers demand even more now.

To understand what lies beneath a flexible environment, a little of how they evolved could be interesting to the Kubernetes audience. Containers allow abstractions. This allows developers to develop software with amazing agility. However, at some point, the environment needs to talk to the storage system, which could be an external storage or a hyper-converged storage system. The way this was done in the early days of Kubernetes was that the environment needed to interface with each storage system differently, sort of similar to how each cellphone manufacturer has its own charger interface; so Apple has its own charger and Samsung its own as well; and similarly it becomes very proprietary and unwieldy for those with multiple storage system options.

So the Kubernetes “community”[1] came up with something called CSI (Container Storage Interface), an interface that enables the environment to talk to all these different storage systems in a uniform way for automating the provisioning, attaching and mounting of block and file storage. In addition, Kubernetes storage interest groups identified snapshot operations as critical functionality for many stateful workloads, as it enables tools to back up the data.

Introduced to navigate this challenge is Kubernetes volume snapshot - beta 1.17, a feature which is now enabled by default and can be part of standard Kubernetes deployments. This feature is becoming widely viable across multiple distributions and, as they gradually adopt the Kubernetes 1.17 version for taking snapshots at the file-level system, this enhances a data protection solution that integrates with the CSI driver. So, this also includes the plain vanilla versions of the Kubernetes distribution, OpenShift, Diamante, Anthos, TKG, etc.

This is great for many enterprises, who are finally able to extend their enterprise grade data protection solution to Kubernetes environments.

We are seeing a number of companies releasing applications in monthly (or even more frequent) builds as part of the DevOps transformation. To accommodate this rapid pace of change, each build check-in includes a combination of application code version, application data schema changes and associated application definition and configuration.

In this rapid pace of change, the probability of failure is high and having a good data protection solution is essential to ensure an equally fast recovery to a stable state.

However, most data protection solutions focus primarily on the application data, and these lead to configuration drifts during rollbacks. It is important to protect the associated environment configuration spread across various Kubernetes objects, including ConfigMaps, Secrets, etc.

In addition, it is important to have the data protection solutions be available in a self-service mode for DevOps and application owners to manage their own rollbacks and data protection policies for the Kubernetes environments. The IT systems admin would still retain the full access using a common interface that manages the entire infrastructure, including bare metal, virtualized and containerized environments, to help with broader systemic failures.

In summary, Kubernetes environments are being adopted widely across organizations as they move to a microservices, agile and DevOps culture to help with rapid transformations. The benefits are being realized across companies recognizing that by building these environments and with their end users in mind; have a better experience. With Kubernetes 1.17 and CSI standards maturing, now is the time for enterprises to adopt them, and look for an enterprise-grade data protection solution to help with protecting these environments. For more information on Dell EMC Data Protection solutions that adhere to, developing and leading best practices for Kubernetes environments look no farther than right here.

Other blogs:

Gaining Speed – The Momentum around Data Protection for Kubernetes

How to Choose the Right Kubernetes Data Protection

Podcasts:

Power2Protect_EP002 - Data Protection, Cloud Native & Kubernetes: Part I

Power2Protect_EP011 - Data Protection, Cloud Native & Kubernetes: Part II

[1] https://github.com/container-storage-interface/spec/blob/master/spec.md