Building a Kubernetes Cluster with Raspberry Pis: A Tech DIY Project
In December 2023, I began an exciting DIY project: constructing a Kubernetes (K8S) cluster using Raspberry Pi s. This venture was driven by my curiosity to delve deeper into the workings of K8S and the opportunity to utilize my accumulated PTO in a productive way! Use it or lose it right?!?
My Setup
I used the following hardware for my cluster:
Each with at least a 32GB microSD - I prefer larger for more storage capacity (images, persistence and state). The Pi 4 has 128GB
For housing the Pis, I chose a stackable Pi case (GeeekPi Cluster Case) - I don't see why I couldn't stack multiple cases if/when I expand my cluster in the future.
They are finally ramping up production of the PI5s so I'll be adding one in the near future.
Networking Components
Setting Up the Cluster
I followed this comprehensive guide to set up my cluster using MicroK8s on Ubuntu . Once the basic cluster was up and running my next task was ensuring that I could interact with the cluster from an external host (ie my laptop).
Once configured, it is time to get my laptop set up to connect.
# ssh onto the cluster's master node
ssh <user>@<master node host or ip>
# retrieve the kube config
sudo microk8s config
copy the output and write it to ~/.kube/config on your machine.
It will look something like (I truncated the cert data for brevity and I'm not about to publish these sensitive values) this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBD...
server: https://192.168.1.181:16443
name: microk8s-cluster
contexts:
- context:
cluster: microk8s-cluster
user: admin
name: microk8s
current-context: microk8s
kind: Config
preferences: {}
users:
- name: admin
user:
client-certificate-data: LS0tLS1CRUd...
client-key-data: LS0tLS1CRUdJ...
Ideally you wouldn't want to use your admin config, you should configure RBAC, but each use case and therefore setup with differ, now you can use kubectl to interact with your cluster from your laptop. This use case and setup are super basic.
I also installed kubectx and kubens on my laptop to make my kubectl commands easier and less verbose.
# get a list of clusters
kubectx
# set the context to your microk8s raspberry pi cluster
kubectx microk8s
# list namesages
kubens
# set a namespace to use so you don't have to include -n <namespace> on
# kubectl commands
kubens kube-system
kubectl get svc # list all services in the kube-system namespace
OCI Registry
To store helm charts and docker images, I set up an OCI registry on my home Synology NAS using the official Docker, Inc image from Docker Hub. I then registered this registry on each node, allowing the cluster to pull internally hosted images - assuming my nas ip is 192.168.1.1 and the registry is listening to port 5000.
#ssh onto each node to add an insecure registry by hostname and ip
# assuming insecure registry at ip 192.168.1.1 with port 5000
sudo vi /var/snap/microk8s/current/args/certs.d/192.168.1.1:5000/hosts.toml
paste the following
server = "https://192.168.1.1:5000"
[host."https://192.168.1.1:5000"]
capabilities = ["pull", "resolve"]
now do the same with your hostname:
sudo vi /var/snap/microk8s/current/args/certs.d/registry-host:5000/hosts.toml
# paste in the hosts.toml file
server = "https://registry-host:5000"
[host."https://registry-host:5000"]
capabilities = ["pull", "resolve"]
Load Balancer
To avoid the complexities of port forwarding and setting up ingresses, I installed a Bare Metal LB, “metallb,” assigning a virtual IP range (10.0.118.0-10.0.118.255). Why 10.0.118.XXX you ask? K=11 and 8 : K8 (see what I did there?) This exposes any load balanced service on an IP in that range.
In addition I needed to set up a static route on my home's router: I have all 10.0.118 ips routed to my master node at 192.168.1.181. (Some day I'll set up a homebrew DNS to give a name to our k8s ips, but that is for another day) - each router will be different, consult your user manual/manufacture documentation for how to do this on your router.
Dashboard
The final touch was installing the Kubernetes dashboard.
The kube dashboard is a microk8s addon or it can be installed via helm or kubectl - I chose the addon.
# from master node
sudo microk8s enable dashboard
To access it simply run on your machine:
kubectl proxy
or better yet let's expose it through a LoadBalancer service (kubernetes-dashboard-ui) so we don't have to mess with the proxy!
Write the following to a file: ./kubernetes-dashboard-ui.yaml
kind: Service
apiVersion: v1
metadata:
name: kubernetes-dashboard-ui
labels:
k8s-app: kubernetes-dashboard
spec:
ports:
- protocol: TCP
port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
type: LoadBalancer
apply to your cluster and get the LoadBalancer EXTERNAL-IP:
# set namespace if not already set
kubens kube-system
kubectl apply -f kubernetes-dashboard-ui
kubectl get svc kubernetes-dashboard-ui
I can now access my k8s dashboard at https://10.0.118.0/ but I still need to authenticate.
I can get my token by running and copying the token attribute: majority of which is redacted in gray. It starts with eyJhbGci... and ends with ...lxYjg. Copy and paste to log in (like with cluster config, the ideal configuration would use RBAC to give individual subjects access to only resources that they are authorized to access)
kubectl describe secret microk8s-dashboard-token
Conclusion
You now have your own K8S cluster powered by some raspberry pis. Now is your opportunity to deploy your own stuff to it - blogs, databases, services, IoT services - the sky is the limit!
This project not only honed my technical skills but also gave me a deeper understanding of Kubernetes in a practical, hands-on manner. It's a perfect example of how technology enthusiasts can leverage their spare time to create something truly remarkable and functional.
My next article with talk about deploying my first app - a generative AI chat bot fully hosted on my Raspberry Pi k8s cluster! This will be a zero code article - minus some docker kung-fu and kubernetes yaml files.
Do you have other ideas on how to configure your own K8S cluster?
Was this article helpful?
I invite you to share your thoughts, ideas or advice below!