StatefulSets - Resizing PVC

One of the most important features of a k8s STS is the ability to attach volumes to the pods and use them as a persistent disk.

Let's say you have deployed an STS with the following configuration:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: "nginx"
  podManagementPolicy: "Parallel"
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.k8s.io/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi        

Nice! You now have an STS that will provision a 1Gi volume for each pod you create.

BUT! What happens when you want to resize the volume and deploy the changes?

You will get an error from the k8s API because this field is immutable and cannot be changed by design.

How do you counter this issue ? Here's a simple trick:

1. Make sure the storage class has:

allowVolumeExpansion: true        

This will allow the volumes to be resized on the fly on the cloud side

1. Change the size to your desired value in the respective PVC and save the changes:

spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi # 1 > 5
  storageClassName: standard
  volumeName: pvc-xxx        

1. Restart the pods to trigger the expansion
2. Now the real magic begins ! Run the following:

kubectl -n your_namespace delete sts --cascade=orphan your_sts_name        

The above will delete the STS BUT, it will leave all the other resources that you have deployed. Pods/Services/Service Accounts/etc, will remain functioning, just without the "father" sts.

1. Change the size of the STS to your desired value, 5Gi in this case in your deployment pipeline and redeploy the STS
2. Now your STS is created and it will take control of them existing pods and it is now resized and will no longer fail the next deployment/pipeline/apply

I hope this "trick" will help you reizing your pvcs on the fly!

Keep calm and enjoy your PVCs!