AKS Storage Provisioning

Storage is an important element for application deployment in any environment, on-prem or cloud. Even if we are running our application in a container some application may need storage to store the data they need for processing.

So if we are using Kubernetes to deploy the application we need to configure the underline storage that needs to run Pods. But the beauty of Kubernetes is it abstract the complex deployment of storage for the application. As a Developer he/she doesn’t need to worry how the storage are provisioned to application.

If you decide to deploy the application to AKS (Azur Kubernetes Service) some application may need temporary storage, some may need persistent volume, some may need fast SSD storage. Luckily Kubernetes can cover most of the storage option and some advance features too.

In this blog we will discuss following Kubernetes storage concepts.

• Volumes
• Persistent volumes
• Storage classes
• Persistent volume claims

Volumes

When deploying application to AKS its often to see provisioning storage to retrieve application data. In Kubernetes world Pods are treat as ephemeral, disposable resources, there for we have different approaches to use persistent data for application. A volume represents a way to store, retrieve, and persist data across pods and through the application lifecycle.

In AKS to provisioned persistent disk it use Azure Disks and Azure Files. There are few methods we can provisioned data disks to Pods, we can create disk manually and attached it to the pod when its provisioned or can use dynamic provisioned.

• Azure Disks — This can be use to create Kubernetes DataDisk resource. These disks can be premium SSD disk or Azure standard storage, backed by HDDs. For most production environment its best to use premium SSDs. Azure disks are mount to Pod as ReadWriteOnce, therefor its only available to a single node. For storage that need to access by multiple nodes simultaneously, use Azure Files.
• Azure Files — This can be used to mount an SMB 3.0 share backed by an Azure Storage account to pods. Files let you share data across multiple nodes and pods

Other than above two storage types Kubernetes support various storage types including Fiber, ISCSI etc. You can refer more about on supported storage from Kubernetes Storage documentation.

Following is an example Kubernetes manifest to attach a volume to Pod

apiVersion: v1
kind: Pod
metadata:
  name: mongodb-azure
spec:
  volumes:
  - name: mongodb-data
    azureDisk:
      diskName: mongovol
      diskURI: /subscriptions/<Sub-ID>/resourceGroups/<Resource-Group>/providers/Microsoft.Compute/disks/mongovol
      kind: Managed
      fsType: ext4
  containers:
  - image: mongo
    name: mongodb
    volumeMounts:
    - name: mongodb-data
      mountPath: /data/db
    ports:
    - containerPort: 27017
      protocol: TCP

Before we create the pod we need to provisioned the Azure Disk. Then add the disk ID to the diskURI parameter. Following is short demo how we can do it in Azure.

PersistentVolumes

Managing storage is a complex problem for compute resources. To make this easy Kubernetes introduce PersistentVolume subsystem an API for users and administrators that abstracts details of how storage is provided from how it is consumed. To do this we need to have few Kubernetes resources called StorageClass, PersistentVolume and PersistentVolumeClaim. PersistentVolumes are use full when creating ReplicaSet, Deployment and StatefulSets.

Cluster Administrator can create PersistentVolume by manually or dynamically created by the Kubernetes API server. If a pod is scheduled and requests storage that is not currently available, Kubernetes can create the underlying Azure Disk or Files storage and attach it to the pod. Dynamic provisioning uses a StorageClass to identify what type of Azure storage needs to be created.

A PersistentVolume is a piece of storage in the cluster that has been provisioned by an administrator. It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system.

A PersistentVolumeClaim is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g., can be mounted once read/write or many times read-only).

Provisioning PersistentVolumes (Manual)

When we create a PersistentVolume from manual, first we need to create an Azure Disk manually as we did in Volumes above. For create a Azure Disk refer this documentation

Following is the Kubernetes manifest for PersistentVolumes.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mongodb
spec:
  capacity:
    storage: 32Gi
  accessModes:
    - ReadWriteOnce
    - ReadOnlyMany
  persistentVolumeReclaimPolicy: Retain
  azureDisk:
      diskName: mongodb-pv
      diskURI: /subscriptions/<Sub-Number>/resourceGroups/<Resource-Group>/providers/Microsoft.Compute/disks/mongodb-pv
      kind: Managed
      fsType: ext4

After that we need to create a PersistentVolumeClaim to Pod

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mongodb-pvc 
spec:
  resources:
    requests:
      storage: 32Gi
  accessModes:
  - ReadWriteOnce
  storageClassName: ""

Next we need to attach this PersistentVolumeClaim to Pod.

apiVersion: v1
kind: Pod
metadata:
  name: mongodb-pvc
spec:
  containers:
  - image: mongo
    name: mongodb
    volumeMounts:
    - name: mongodb-data
      mountPath: /data/db
    ports:
    - containerPort: 27017
      protocol: TCP
  volumes:
  - name: mongodb-data
    persistentVolumeClaim:
      claimName: mongodb-pvc

Following is a demo for manually creating PV, PVC.

Provisioning PersistentVolumes (Dynamic)

When we use dynamic we do not need to provisioned the PersistentVolume upfront. To work with dynamic provisioned first we need to create a StorageClass. After creating StorageClass add StorageClass to the PersistentVolumeClaim. When we deploy the Pod it automatically provisioned the storage required for it. Following is the yaml manifest.

Yaml for the StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: premium-ssd
provisioner: kubernetes.io/azure-disk
parameters:
  kind: managed
  storageaccounttype: Premium_LRS

Yaml for the PersistentVolumeClaim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mongodb-pvc 
spec:
  resources:
    requests:
      storage: 32Gi
  accessModes:
  - ReadWriteOnce
  storageClassName: "premium-ssd"

Yaml for the Pod

apiVersion: v1
kind: Pod
metadata:
  name: mongodb-pvc
spec:
  containers:
  - image: mongo
    name: mongodb
    volumeMounts:
    - name: mongodb-data
      mountPath: /data/db
    ports:
    - containerPort: 27017
      protocol: TCP
  volumes:
  - name: mongodb-data
    persistentVolumeClaim:
      claimName: mongodb-pvc

Following is a demo for dynamic storage creation.

Additional References

• Azure AKS Storage Documentation
• Kubernetes Storage Documentation