Day12 Daemonset , Jobs & CronJobs

A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.

Some typical uses of a DaemonSet are:

1. Log Collection and Monitoring: Deploying log collectors, such as Fluentd or Filebeat, as DaemonSets ensures that logs are collected from every node in the cluster, enabling centralized monitoring and analysis.
2. Network Agents: DaemonSets are useful for deploying network agents, such as Calico or Weave, that provide networking and security functionalities to all nodes within the cluster.
3. Resource Monitoring: Running resource monitoring agents, such as Prometheus Node Exporter, as DaemonSets allows you to collect node-level metrics and monitor resource utilization across the cluster.
4. System-Level Services: DaemonSets can be used to deploy system-level services like cluster-wide DNS resolvers, storage provisioners, or any other process that needs to be present on every node.

In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon. A more complex setup might use multiple DaemonSets for a single type of daemon, but with different flags and/or different memory and cpu requests for different hardware types.

Deploying a DaemonSet

Deploying a DaemonSet is just like a Deployment resource. You define the desired state of the DaemonSet by specifying the pod template, which includes the container image, labels, and any required volumes or configurations. Once you apply the DaemonSet manifest, Kubernetes takes care of scheduling and ensuring that a pod is running on each node.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd-elasticsearch
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
spec:
  selector:
    matchLabels:
      name: fluentd-elasticsearch
  template:
    metadata:
      labels:
        name: fluentd-elasticsearch
    spec:
      tolerations:
      # these tolerations are to have the daemonset runnable on control plane nodes
      # remove them if your control plane nodes should not run pods
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      containers:
      - name: fluentd-elasticsearch
        image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
      # it may be desirable to set a high priority class to ensure that a DaemonSet Pod
      # preempts running Pods
      # priorityClassName: important
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
        

Jobs

A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. Suspending a Job will delete its active Pods until the Job is resumed again.

A simple case is to create one Job object to reliably run one Pod to completion. The Job object will start a new Pod if the first Pod fails or is deleted (for example due to a node hardware failure or a node reboot).

You can also use a Job to run multiple Pods in parallel.

  apiVersion: batch/v1
  kind: Job
  metadata:
    name: hello-world
  spec:
    template:
      spec:
        containers:
        - name: hello
          image: busybox
          command: ["echo", "Hello, World!"]
        restartPolicy: Never
    backoffLimit: 4
        

CronJobs

CronJob is meant for performing regular scheduled actions such as backups, report generation, and so on. One CronJob object is like one line of a crontab (cron table) file on a Unix system. It runs a Job periodically on a given schedule, written in Cron format.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox:1.28
            imagePullPolicy: IfNotPresent
            command:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure
        

Cron job is to schedule the task just like cronjob in linux

Where first star represent minute , hour, day of the month, month, and day of week respectively.

example: