Kubeflow Roadmap - Replacing Ksonnet with Kustomize or Helm ?

As a Kubeflow Product Manager, I often receive questions on the Kubeflow roadmap. I thought that others might like this recent discussion from some Community members.

Kubeflow User Question - Given a choice, I’d prefer not to use Helm, because if I’m going to ask the ops folks for privileged access to setup Tiller I might as well try to setup an Operator that not only allows me to deploy, but also to manage the state of Kubeflow in a namespace. I noticed that there already are operators for Tensorflow and Pytorch in the Kubeflow Github repository, but nothing for Kubeflow itself.

One Kubeflow Contributor's View - You are raising some very interesting points. I'll go through your specific points and then attempt to summarize our current direction and focus.

Helm charts are seemingly easy for simplier, first deployments, but can very quickly become a messy collection of unreadable templates and black-box K8s object definitions being controlled via values.yaml files.

Similarly, its single, monolithic giant sudo-like Tiller is a major showstopper for deployment in security-conscious environments.

From our perspective, the Kubeflow Community continues to move towards Kustomize. Kustomize delivers service definitions as flat YAML manifests, which the DevOps engineer can tune to their environment by writing extra "kustomization" overlays, again standard YAMLs. All overlays are meant to be kept under version control.

Simple YAML manifests are much easier to audit than custom Helm charts, which is very useful when working in an on-prem, locked-down cluster.

Similarly, support for namespace-based isolation is very important for Kubeflow in enterprise environments.

In the older Kubeflow 0.4 version, all components are meant to be run in the same namespace, hence no isolation, and JupyterHub is the main UI for users to spawn notebooks. In 0.5, Kubeflow replaced JupyterHub with a very thin, Kubernetes-native UI that spawns instances of a Notebook CRD directly. This removes much of JupyterHub's complexity, and allows for easy execution of multiple notebook servers concurrently.

Moreover, 0.5 is also the first version with native support for namespace-based user isolation. The "multi-user" Customer User Journey (CUJ) brings support for a new "Profiles" K8s Custom Resource. This enables dynamic per-user creation of namespaces, so each user can run isolated by default.

The current implementation of the multi-user CUJ focuses on Google IAP for authentication. The Community received great input at the Kubeflow Contributor's Summit at Google in Sunnyvale on March 12 and 13. We understand the need for integration with on-prem enterprise-grade authentication, i.e. in the form of LDAP / Active Directory. We will be driving this effort as part of the multi-user and on-prem CUJs during the upcoming 0.6 release cycle.