Repeatable KubeCon Live Demos - without worry

A recurring joke among speakers at the reception prior to #kubecon & Cloud Native Con North America 2022 was "I'm going to do a live demo - wish me luck".

A good portion of people I spoke to had mentioned other strategies, like recording their demos prior or mapping them into slides. Neither of those is wrong and the presentations definitely still provide value to attendees - but I was determined to do a live demo against my local kubernetes cluster, while making modifications to workloads that require a re-deployment, in order to prove out a #continuouscompliance workflow around #lula.

Cluster Bootstrapping

The first step was establishing what my local cluster would look like and how it would be repeatably feed the artifacts it needed to operate.

Not being new to the term "air gap", I kind-of knew where to look without much toil. Air Gap is commonly denoted by software systems as to what their runtime looks like when there is little-to-no connectivity out to the internet, and for KubeCon, I wanted to have the goal of being able to repeatably run my demonstration without wifi enabled.

"Rancher Desktop is an app that provides container management and Kubernetes on the desktop. It is available for Mac (both on Intel and Apple Silicon), Windows, and Linux." - https://docs.rancherdesktop.io/

Not only that, if we read through the docs, we'll see that:

"If Rancher Desktop has been installed on a machine initially with networked access, it can be run subsequently on that machine after network connectivity has been turned off."

So if I had internet at some point prior to the event (IE the week prior at my home), I could essentially cache all requirements for running #rancher Desktop #kubernetes with the ability to reset the cluster and not need any external connection (Thank you K3S for building #airgap as a 1st class citizen).

I enabled kubernetes under Rancher Desktop - Preferences and selected V1.24.3 as my version of kubernetes to download and run. Once it came online, I disconnected from internet access and Reset Kubernetes under the troubleshooting tab to test that I could re-provision the local cluster without the need for internet.

Confident that I could rely on the cluster offline, I moved onto bootstrapping the demo.

Demo Bootstrapping

The demonstration I sought to provide was a live-cluster running locally on my machine, with #istio service mesh deployed and actively injecting sidecar proxies. This would then allow me to deploy generic web applications and be able to demonstrate that they were/were-not compliant with rulesets that Lula was validating as compliant with #security controls.

Offline Istio Package

This deployment of istio was going to be very generic and bare-bones configuration. I wanted to have enough of istio deployed to have a functional mesh via the sidecar architecture to represent mTLS traffic between pods that communicate.

So what did I need? Following Istio docs, I needed:

1. Istio base Orchestration
2. Istiod Orchestration
3. Required Images

Not only these, but I needed a reproducible way to deploy these without internet connectivity. I'd be lying if I said I didn't already know how to do this. Zarf is purpose built for air gap software delivery. I'd have a much longer article if Included all of the Zarf capabilities here, but the main gist is that Zarf handles both the packaging of artifacts (above) into a single archive, as well as simplifying the deployment of those artifacts to a kubernetes cluster in the air gap (and much much more).

From here I created a manifest for the #zarf package of istio as seen Here, and performed a zarf package create to collect all of those artifacts into a local archive.

This means I can now reproducibly deploy istio into my cluster with zarf package deploy and not need any external connectivity (more on the deploy of this in a minute).

Offline Demo Package

Along with istio, I needed a web application I could package and deploy easily to demonstrate both the in-compliance and out-of-compliance states for #lula to validate. I chose nginx for simplicity and packaged a deployment manifest and required image with my zarf manifest seen Here.

With that, I now have a cluster that can be provisioned offline, a package for deploying istio offline, and a workload to represent a test scenario that can be deployed offline. Now we're ready to present at KubeCon!

Demonstration

Setup

Prior to the presentation, I setup my cluster by reseting kubernetes as outlined above. This put me in a fresh state. I then performed the following steps:

1. zarf init - this deploys the zarf runtime (registry and mutating webhook agents) into the cluster - also from an offline package.
2. zarf package deploy zarf-package-istio-package-arm64.tar.zst - this deployed the istio #servicemesh

Live-Demo

Then came the time to present. Looking at the rules for my #oscal component definition to be verified, we're checking to see if all pods in a specific namespace contain the istio sidecar #proxy.

The zarf package for this workload contains a variable that I can set at runtime with zarf in order to modify imperatively.

To represent the compliant scenario, I ran:

• zarf package deploy zarf-package-compliance-demo-arm64.tar.zst --confirm --set INJECT=true

This ensures the sidecar would be injected and running lula execute oscal-component.yaml was able to demonstrate that my live cluster configuration for this workload was #compliant with the security control (AC-4 from #NIST 800-53).

In order to further demonstrate future capabilities for continuous compliance, we want to ensure we can validate a non-complaint configuration for kubernetes. I then ran:

• zarf package deploy zarf-package-compliance-demo.tar.zst --confirm --set INJECT=false

This performs a rolling-update to the deployment and the pod, which now opts-out of istio sidecar injection, no longer has the istio sidecar proxy. Re-running the same command lula execute oscal-component.yaml produced #noncompliance against the same ruleset.

Summary

There is a lot of good material I am glossing over here, simply because I wanted to focus on demonstration logistics over the demonstration itself. Planning to dive into the demonstration itself more, as well as what being a first-time speaker at KubeCon required and meant to me leading up to the event.

Please see my demo repository for more specific information.