An Overview to Kubernetes Client Library

The client-go is the official client library for the Kubernetes programming interface, designed to interact with Kubernetes clusters. This typical web service client library provides a comprehensive set of tools and abstractions that facilitate the development of applications and tools for managing Kubernetes resources.

With client-go, you can interact with the API server's HTTP interface using REST verbs such as Create, Get, List, Update, Delete, Patch, and Watch. The API server exposes a RESTful HTTP API that communicates using JSON or Protocol Buffers (protobuf). This design primarily supports internal cluster communication for performance optimization.

You can find client-go on GitHub: kubernetes/client-go .

Key Features

1. Kubernetes API Interaction: Supports CRUD operations on Kubernetes resources.
2. Typed Clients: Provides typed clients for different resources (e.g., Pods, Deployments).
3. Informers and Listers: Enables watching for resource changes and maintains an in-memory cache.
4. Authentication and Configuration: Simplifies authentication with the Kubernetes API.
5. Rate Limiting: Helps manage the load on the API server by limiting the number of requests.

Getting Started with client-go

1. Set Up Your Go Environment:

Make sure you have Go installed on your machine. If you have not installed, refer to https://go.dev/doc/install to install.

After installing the Go, you can set up your Go project with:

go mod init my-k8s-tool
        

2. Install client-go

Add client-go to your project by running:

% go get k8s.io/client-go@latest
go: downloading k8s.io/client-go v0.31.2
go: added k8s.io/client-go v0.31.2
        

You may also need to add the Kubernetes API machinery:

% go get k8s.io/apimachinery@latest

go: downloading k8s.io/apimachinery v0.31.2        

3. Create a clientset

To interact with the Kubernetes API, you first need to create a clientset. Here’s an example of how to do this:

package main

import (
    "context"
    "flag"
    "fmt"
    "os"

    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
)

func main() {
    // Load the kubeconfig file
    kubeconfig := flag.String("kubeconfig", "~/.kube/config", "path to the kubeconfig file")
    flag.Parse()

    // Create a config from the kubeconfig file
    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error building kubeconfig: %v\n", err)
        os.Exit(1)
    }

    // Create the clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error creating clientset: %v\n", err)
        os.Exit(1)
    }

    // Example: List all Pods in the kube-system namespace
    pods, err := clientset.CoreV1().Pods("kube-system").List(context.TODO(), metav1.ListOptions{})
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error listing pods: %v\n", err)
        os.Exit(1)
    }

    for _, pod := range pods.Items {
        fmt.Println(pod.Name)
    }
}
        

• The kubernetes package contains the clientset to access Kubernetes API.
• The tools/clientcmd from client-go contains tools to read and parse the kubeconfig (i.e., the client configuration with server name, credentials, etc.)
• That kubeconfig is read and parsed using clientcmd.BuildConfigFromFlags.
• From clientcmd.BuildConfigFromFlags we get a rest.Config which you can find in the k8s.io/client-go/rest package. This is passed to kubernetes.NewForConfig in order to create the actual Kubernetes client set. It’s called a client set because it contains multiple clients for all native Kubernetes resources.
• In the following example code, we select the core group in v1 with clientset set.CoreV1() and then list all the pods in the "kube-system" namespace

 pods, err := clientset.CoreV1().Pods("kube-system").List(context.TODO(), metav1.ListOptions{})        

Note that only the last function call, List, actually accesses the server. Both CoreV1 and Pods select the client and set the namespace only for the following List call (this is often called the builder pattern, in this case to build a request. If you need an explanation of the builder design pattern, please refer to https://www.geeksforgeeks.org/builder-design-pattern/ )

• The List call sends an HTTP GET request to /api/v1/namespaces/kube-systems/pods/ on the server, which is set in the kubeconfig. If the Kubernetes API server answers with HTTP code 200, the body of the response will carry a list of encoded pod objects, either as JSON—which is the default wire format of client-go—or as protocol buffers.
• You can enable protobuf for native Kubernetes resource clients by modifying the REST config before creating a client from it:

config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig) config.AcceptContentTypes = "application/vnd.kubernetes.protobuf,
application/json" config.ContentType = "application/vnd.kubernetes.protobuf"
clientset, err := kubernetes.NewForConfig(config)        

4. HTTP Verbs for Interacting with Kubernetes Resources

The Kubernetes API server exposes an HTTP REST interface, allowing clients to interact with resources using standard HTTP verbs.

• HTTP GET: Retrieves data for a specific resource (e.g., a particular Pod) or a collection of resources (e.g., all Pods in a namespace).
• HTTP POST: Creates a new resource, such as a Service or a Deployment.
• HTTP PUT: Updates an existing resource, for example, changing the container image of a Pod.
• HTTP PATCH: Partially updates an existing resource. To learn more about strategies for patching resources, refer to the Kubernetes documentation on using a JSON merge patch to update a Deployment.
• HTTP DELETE: Deletes a resource permanently.

Using the clientset, you can perform these HTTP operations on various Kubernetes resources through the API server.

• Listing Resources (HTTP GET)

To list all Pods in the "kube-system" namespace, you can use ListOptions:

pods, err := clientset.CoreV1().Pods("kube-system").List(context.TODO(), metav1.ListOptions{})
        

• Create a Resource(HTTP PUT ):

Use CreateOptions to create a pod in the "example" namespace

pod := &v1.Pod{...} // Define your Pod spec here
_, err = clientset.CoreV1().Pods("example").Create(context.TODO(), pod, metav1.CreateOptions{})
        

• Update a Resource(HTTP UPDATE):

UpdateOptions to update "my-pod" in the "example" namespace with a new image

pod, err := clientset.CoreV1().Pods("example").Get(context.TODO(), "my-pod", metav1.GetOptions{})
pod.Spec.Containers[0].Image = "new-image"
_, err = clientset.CoreV1().Pods("default").Update(context.TODO(), pod, metav1.UpdateOptions{})
        

• Delete a Resource(HTTP DELETE)

DeleteOptions to delete "my-pod" in the example namespace

err = clientset.CoreV1().Pods("example").Delete(context.TODO(), "my-pod", metav1.DeleteOptions{})
        

5. Watches

Use the watch.Interface to monitor changes to resources. However, using informers is preferred for caching and efficient resource management.

// Interface can be implemented by anything that knows how to watch and // report changes.
type Interface interface {
// Stops watching. Will close the channel returned by ResultChan(). Releases // any resources used by the watch.
Stop()
// Returns a chan which will receive all the events. If an error occurs // or Stop() is called, this channel will be closed, in which case the // watch should be completely cleaned up.
ResultChan() <-chan Event
}        

The result channel of the watch interface returns three kinds of events:

// EventType defines the possible types of events.
type EventType string
const ( Added
Modified Deleted Error
)
// Event represents a single event to a watched resource. // +k8s:deepcopy-gen=true
type Event struct {
Type EventType
EventType = "ADDED" EventType = "MODIFIED" EventType = "DELETED" EventType = "ERROR"
//
//
//
//
//
//
Object runtime.Object
}        

6. Use Informers and Caching

If you need to watch for changes to resources, use informers. Here’s a basic example of setting up an informer to watch Pods:

import (
    "k8s.io/client-go/tools/cache"
    "k8s.io/client-go/kubernetes/scheme"
)

func main() {
    // After creating the clientset

    // Create a shared informer
    informer := cache.NewSharedInformer(
        cache.NewListWatchFromClient(clientset.CoreV1().RESTClient(), "pods", v1.NamespaceAll, fields.Everything()),
        &v1.Pod{},
        0, // No resync (0 means it won't resync periodically)
    )

    informer.AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc: func(obj interface{}) {
            pod := obj.(*v1.Pod)
            fmt.Printf("Pod added: %s\n", pod.Name)
        },
        UpdateFunc: func(oldObj, newObj interface{}) {
            pod := newObj.(*v1.Pod)
            fmt.Printf("Pod updated: %s\n", pod.Name)
        },
        DeleteFunc: func(obj interface{}) {
            pod := obj.(*v1.Pod)
            fmt.Printf("Pod deleted: %s\n", pod.Name)
        },
    })

    stopCh := make(chan struct{})
    defer close(stopCh)
    go informer.Run(stopCh)

    // Block forever
    select {}
}
        

The client set main interface in k8s.io/client-go/kubernetes/typed for Kubernetes- native resources looks like this:

type Interface interface {
	Discovery() discovery.DiscoveryInterface
	AdmissionregistrationV1() admissionregistrationv1.AdmissionregistrationV1Interface
	AdmissionregistrationV1alpha1() admissionregistrationv1alpha1.AdmissionregistrationV1alpha1Interface
	AdmissionregistrationV1beta1() admissionregistrationv1beta1.AdmissionregistrationV1beta1Interface
	InternalV1alpha1() internalv1alpha1.InternalV1alpha1Interface
	AppsV1() appsv1.AppsV1Interface
	AppsV1beta1() appsv1beta1.AppsV1beta1Interface
	AppsV1beta2() appsv1beta2.AppsV1beta2Interface
	AuthenticationV1() authenticationv1.AuthenticationV1Interface
	AuthenticationV1alpha1() authenticationv1alpha1.AuthenticationV1alpha1Interface
	AuthenticationV1beta1() authenticationv1beta1.AuthenticationV1beta1Interface
	AuthorizationV1() authorizationv1.AuthorizationV1Interface
	AuthorizationV1beta1() authorizationv1beta1.AuthorizationV1beta1Interface
	AutoscalingV1() autoscalingv1.AutoscalingV1Interface
	AutoscalingV2() autoscalingv2.AutoscalingV2Interface
	AutoscalingV2beta1() autoscalingv2beta1.AutoscalingV2beta1Interface
	AutoscalingV2beta2() autoscalingv2beta2.AutoscalingV2beta2Interface
	...
}        

To understand more about informer and cache, please check out https://firehydrant.com/blog/stay-informed-with-kubernetes-informers/

Versioning and Compatibility

Kubernetes APIs are versioned. It's important to note that using a client version that doesn't match the API group version supported by the API server will result in failure. Clients are hardcoded to a specific version, so developers must select the appropriate version for their application. it is important to ensure that your client-go version is compatible with the cluster's API version. Features marked as alpha or beta may not be available in older server versions.

API Machinery

There is another important repository called API Machinery, which is used as k8s.io/apimachinery in Go. This repository provides the generic building blocks needed to implement a Kubernetes-like API. While API Machinery is not limited to container management, it plays a vital role in Kubernetes.

For example, API Machinery could be used to build APIs for an online shop or any other domain-specific service. Many API Machinery packages appear frequently in Kubernetes-native Go code. One significant package is k8s.io/apimachinery/pkg/apis/meta/v1, which contains generic API types like ObjectMeta, TypeMeta, GetOptions, and ListOptions.

Conclusion

client-go is a powerful library for building tools that interact with Kubernetes clusters. By following the steps outlined above, you can begin creating tools to manage Kubernetes resources. Don't forget to explore the official client-go documentation to dive deeper into its advanced features and best practices.