Kubernetes-101: ConfigMaps and Secrets

This article is also available at my blog: mattias.engineer/k8s/configmaps-and-secrets/

When we discussed Pods in the first two articles in this series we saw how to set environment variables directly in our Pod manifest. This is fine for many situations. However, sometimes it is better to separate the configuration from the Pod manifest. This is especially true if you need to reuse the same set of environment variables in two or more Pods. This is where the ConfigMap resource comes in handy. We can create a ConfigMap containing the environment variables we need, and then we can refer to this ConfigMap in our Pods.

A special kind of configuration value is a secret, like a password to a database or a client secret for an OIDC connection. To handle secrets inside of a Kubernetes cluster there is a special kind of resource known as a Secret.

In this article we will go through how to use both ConfigMaps and Secrets.

ConfigMaps

There are a few different ways you can use the configuration stored in a ConfigMap. One common way is to set environment variables for a Pod using the values from a ConfigMap. Another common way is to mount them as Volumes in a Pod, we will see examples of this in the next article when we discuss Volumes. In this section we will concentrate on using a ConfigMap to set environment variables.

Imperatively creating ConfigMaps

I prefer to create my Kubernetes resources declaratively through the use of manifests. But for the fun of it, let us briefly see how we can create a ConfigMap in an imperative way using kubectl create configmap:

$ kubectl create configmap application-settings \
    --from-literal=header_color=blue \
    --from-literal=body_color=yellow

configmap/application-settings created        

Here we created a ConfigMap containing two key-value pairs, header_color=blue and body_color=yellow. We provided each key-value pair in the command using the --from-literal flag repeatedly.

If we instead have a file containing a set of key-value pairs we can create a ConfigMap using that file. For example, imagine we have a file named config.env with the following content:

header_color=blue
body_color=yellow        

Then we can create a ConfigMap from this file using the following command:

$ kubectl create configmap application-settings --from-env-file ./config.env

configmap/application-settings created        

The end result is identical to the first command using --from-literal repeatedly. That’s enough imperative commands for now.

Declaratively creating ConfigMaps

Let us instead create the previous ConfigMap using a declarative approach with a Kubernetes manifest:

# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: application-config
data:
  header_color: "blue"
  body_color: "yellow"        

Similar to the other manifests we have seen the ConfigMap manifest has an .apiVersion, a .kind, and .metadata.name. However, the .spec part that we have come to know and love, is missing. Instead we have .data which is where we can define our key-value pairs of settings that this ConfigMap contains. Apart from the .data property a ConfigMap could also have a .binaryData field. The difference between .data and .binaryData is that the .data property has settings in clear text while the .binaryData property has settings in base64-encoded binary data.

We can use kubectl apply to create the ConfigMap from our manifest:

$ kubectl apply -f configmap.yaml

configmap/application-config created        

We can list all of our ConfigMaps with kubectl get configmaps:

$ kubectl get configmaps

NAME                 DATA   AGE
application-config   2      3s        

The output tells us that the ConfigMap named application-config contains 2 items (our two key-value pairs). If we want to shorten the previous command we would use the short form for configmaps which is cm:

$ kubectl get cm

NAME                 DATA   AGE
application-config   2      31s        

In the previous article in this series we discussed Namespaces. Let’s see if we have ConfigMaps in any other Namespace by adding the -A flag (or --all-namespaces) to the previous command:

$ kubectl get cm -A

NAMESPACE         NAME                                 DATA   AGE
default           application-config                   2      54s
default           kube-root-ca.crt                     1      6d23h
kube-node-lease   kube-root-ca.crt                     1      6d23h
kube-public       cluster-info                         1      6d23h
kube-public       kube-root-ca.crt                     1      6d23h
kube-system       coredns                              1      6d23h
kube-system       extension-apiserver-authentication   6      6d23h
kube-system       kube-proxy                           2      6d23h
kube-system       kube-root-ca.crt                     1      6d23h
kube-system       kubeadm-config                       1      6d23h
kube-system       kubelet-config                       1      6d23h
security          kube-root-ca.crt                     1      6d23h        

We see a lot of ConfigMaps. These ConfigMaps came with my Minikube cluster, and we should not edit or delete them unless we believe we have a reason to do so. Remember how everything in Namespaces named kube-* are generally internal for Kubernetes to function. The number of ConfigMaps you see in this list will depend on what type of Kubernetes cluster you are using.

If we want to see additional details about a given ConfigMap we can use kubectl describe configmap:

$ kubectl describe configmap application-config

Name:         application-config
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
header_color:
----
blue
body_color:
----
yellow

BinaryData
====

Events:  <none>        

The output has a strange format, but we can see the settings we provided in our manifest. We could also run kubectl get configmap on a specific ConfigMap:

$ kubectl get configmap application-config

NAME                 DATA   AGE
application-config   2      2m        

Time for a brief kubectl intermission.

Whenever we do a kubectl get command we could add the -o flag (or --output) with the value of yaml to get the output in a YAML format that is close to what we have in our manifest file. If we do this for our ConfigMap we get this:

$ kubectl get configmap application-config -o yaml

apiVersion: v1
data:
  favorite_word: extrapolation
  word_of_the_day: hello
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"header_color":"blue","body_color":"yellow"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"application-config","namespace":"default"}}
  creationTimestamp: "2022-12-22T18:55:16Z"
  name: application-config
  namespace: default
  resourceVersion: "28290"
  uid: cf85c829-49b4-4da0-ae16-51c113afb953        

If we compare this output to the manifest we applied we see that there are a few additional properties here, specifically nested under .metadata. These are not properties that you should set yourself, Kubernetes sets them for us and uses them to keep track of the object state (except for the .metadata.namespace property which we could set to the Namespace we wish).

As a last example in this section we will delete a ConfigMap using kubectl delete:

$ kubectl delete configmap application-config

configmap "application-config" deleted        

Using a ConfigMap in a Pod

There is no point in having a ConfigMap unless we use it for something. Here we’ll see a simple example of how to use a ConfigMap to set environment variables in a Pod. We will reuse the ConfigMap from above, and combine it with a Pod manifest (I will use a Pod, not a Deployment, for simplicity):

# application.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: application-config
data:
  header_color: "blue"
  body_color: "yellow"
---
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
    - name: web
      image: nginx:latest
      ports:
        - containerPort: 80
      env:
        - name: HEADER_COLOR
          valueFrom:
            configMapKeyRef:
              name: application-config
              key: header_color
        - name: BODY_COLOR
          valueFrom:
            configMapKeyRef:
              name: application-config
              key: body_color        

We can apply this manifest to create our Pod and ConfigMap objects:

$ kubectl apply -f application.yaml

configmap/application-config created
pod/web created        

To verify that the environment variables have been set correctly we will use kubectl exec to run the env command inside of out Nginx Pod:

$ kubectl exec -it web -- env | grep _COLOR

HEADER_COLOR=blue
BODY_COLOR=yellow        

It worked! If you need a refresher on the kubectl exec command you can read the second article in this series on Pods here.

Secrets

Secrets are similar to ConfigMaps and can be used in similar ways. A Secret generally contains a password, a token, or some sort of key. They should not be stored in plain text inside of ConfigMaps. Working with Secrets is very similar to working with ConfigMaps, with some minor differences.

Imperatively creating Secrets

Let us first look at how to create a Secret with an imperative command, similar to how we did for ConfigMaps. The command to use is kubectl create secret:

$ kubectl create secret generic db-secrets \
   --from-literal=db_username="admin" \
   --from-literal=db_password="s3cr3tp4ssw0rd"

secret/db-secrets created        

I specify that I create a generic Secret. There are a few different kinds of Secrets, as shown in the table in the official documentation. Where in that table is the generic Secret type? Unfortunately the naming is a bit inconsistent. The Opaque type is the same as the generic type. This is probably the most common Secret type you will use, and it is the only one we will see in this article.

Declaratively creating Secrets

A manifest for a Secret is shown below:

# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: db-secrets
type: Opaque
data:
  db_username: YWRtaW4=
  db_password: czNjcjN0cDRzc3cwcmQ=        

This looks similar to what the manifest for a ConfigMap looked like. One glaring difference seems to be in the values for the keys in .data. The values are provided in base64-encoded format. How did I obtain these base64-encoded values you ask? I did the following:

$ echo -n "admin" | base64

YWRtaW4=

$ echo -n "s3cr3tp4ssw0rd" | base64

czNjcjN0cDRzc3cwcmQ=        

If this step feels like to much work you could replace .data with .stringData in the manifest, and then provide the values in plaintext. No matter what way you do this, remember that this Secret manifest should not be committed to your source repository as is. You could add the Secret with a dummy value, then update the Secret value to the correct value when you apply your manifest, either in a CI/CD pipeline or manually in your terminal.

When we have our Secret manifest we can work with the Secret as with any other Kubernetes object, we’ll see the usual examples of kubectl get, and kubectl describe, below. First we list our newly created Secret:

$ kubectl get secret db-secrets

NAME         TYPE     DATA   AGE
db-secrets   Opaque   2      12s        

We could also list all available Secrets in all Namespaces:

$ kubectl get secrets -A

NAMESPACE   NAME                                     TYPE                 DATA   AGE
default     db-secrets                               Opaque               2      96s        

Unfortunately I did not have any other Secret in my cluster.

We could describe a Secret to get additional details:

$ kubectl describe secret db-secrets

Name:         db-secrets
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
db_username:  5 bytes
db_password:  14 bytes        

Let us delete our Secret to end this section, we use kubectl delete for this:

$ kubectl delete secret db-secrets

secret "db-secrets" deleted        

Using a Secret in a Pod

Similarly to what we did for a ConfigMap we will now use a Secret in a simple application. The application manifest is shown below:

# application.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: db-secrets
type: Opaque
data:
  db_username: YWRtaW4=
  db_password: czNjcjN0cDRzc3cwcmQ=
---
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
    - name: web
      image: nginx:latest
      ports:
        - containerPort: 80
      env:
        - name: DB_USERNAME
          valueFrom:
            secretKeyRef:
              name: db-secrets
              key: db_username
        - name: DB_PASSWPRD
          valueFrom:
            secretKeyRef:
              name: db-secrets
              key: db_password        

We can apply this manifest to create our Pod and Secret objects:

$ kubectl apply -f application.yaml

secret/db-secrets created
pod/web created        

To verify that the environment variables containing our secrets have been set correctly we will use kubectl exec to run the env command inside of out Nginx Pod:

$ kubectl exec -it web -- env | grep DB_

DB_USERNAME=admin
DB_PASSWORD=s3cr3tp4ssw0rd        

It worked!

How to work with Secrets in production?

I briefly mentioned that we should not commit our secret values to our git repository. So how do you work with Secrets in production? I would argue that the simplest way is the following if you want a semi-manual approach:

1. Create a manifest that represents your Secret
2. Provide dummy values for your Secret keys
3. Apply the manifest to create the Secret in your cluster using your CI/CD pipeline or some other automation
4. Update the value of the Secret directly in your cluster using kubectl edit secret <secret-name>

If you are using a CI/CD workflow you could use an approach like the following:

1. Store your secret values in the CI/CD secret store (e.g. GitHub Actions secrets)
2. Build your Secret manifest using some templating tool, and include the secret values from the secret store
3. Apply the manifests to create the Secret in your cluster from your CI/CD workflow

There are other options for secret values as well, for instance you could store them in a third-party tool such as Hashicorp Vault. There is also an option of actually storing your secret values in your git repository in an encrypted form using Bitnami sealed secrets.

Summary

We have now seen how to create and use both ConfigMaps and Secrets for our applications in Kubernetes. We looked at both imperative and declarative ways of working with both objects. We ended by briefly discussing how to handle Secrets in a production environment.

In the next article we will visit the topic of Volumes. Volumes provide storage for our Pods. We will also encounter PersistentVolumes and PersistentVolumeClaims.