Deploying Prometheus and Grafana over Kubernetes

Task:

• Creating Docker images for Prometheus & Grafana.
• Deploying Prometheus & Grafana as pods on top of Kubernetes by creating resources Deployment, ReplicaSet, Pods or Services.
• Making their data persistent using PVC. 
• Exposing both pods to Outside world

Creating Docker images for Prometheus & Grafana

Grafana

FROM centos:latest
RUN yum install wget -y
RUN wget https://dl.grafana.com/oss/release/grafana-7.0.3-1.x86_64.rpm
RUN yum install grafana-7.0.3-1.x86_64.rpm -y
WORKDIR /usr/share/grafana
CMD /usr/sbin/grafana-server start && /usr/sbin/grafana-server enable && /bin/bash

I created grafana image on top of centos using Dockerfile and uploaded the image to DockerHub, so that we can directly download the image from there.

Prometheus

Lets do the same for Prometheus image

FROM centos


RUN yum install wget -y

RUN wget https://github.com/prometheus/prometheus/releases/download/v2.19.0/prometheus-2.19.0.linux-amd64.tar.gz

RUN tar -xzf prometheus-2.19.0.linux-amd64.tar.gz

RUN mkdir /prom_data

CMD ./prometheus-2.19.0.linux-amd64/prometheus --config.file=/prometheus-2.19.0.linux-amd64/prometheus.yml --storage.tsdb.path=/prom_data && tail -f /dev/null

Creating Deployment

Prometheus

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prom-deploy
  labels:
    app: prometheus
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      name: prom-deploy
      labels:
        app: prometheus
    spec:
      volumes:
        - name: prom-pv-vol
          persistentVolumeClaim:
            claimName: prom-pv-volclaim
        
      containers:
        - name: prometheus
          image: gauravmehta13/prometheus:v2
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9090
          volumeMounts:
            - mountPath: "/metrics"
              name: prom-pv-vol

Grafana

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana
  labels:
    app: grafana
spec:
  selector:
    matchLabels:
      app: grafana
  template:
    metadata:
      name: grafana
      labels:
        app: grafana
    
    spec:
      containers:
        - name: grafana
          image: gauravmehta13/grafana:v1
      imagePullPolicy: IfNotPresent

      restartPolicy: Always

PVC

apiVersion: v1


kind: PersistentVolumeClaim


metadata:
  name: prom-pv-volclaim


spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Kustomization

I used Kustomization to run all the yml files in a sequence.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - pvc.yml
  - prometheus.yml
  - grafana.yml

We have to use kubectl create -k . for running kustomization file

Let's check the running pods

Now, we have to expose these pods using type NodePort

#For exposing to outside world
kubectl expose deployment grafana --port=3000 --type=NodePort
kubectl expose deployment prom-deploy --port=9090 --type=NodePort

After this we can check the ports using kubectl get svc

Configuring Grafana

• The default username and password are admin
• After that we will get a screen to change password

• Now, We have to go to Data Source and select Prometheus and provide its url.

Now, We are ready to Monitor

We can can now view the real-time Metrics information visuals through Grafana using PromQl. We also have created Persistent storage so that even if the Pods crash due to some reason, Prometheus will automatically retrieve the data back.