Develop an Infrastructure as Code solution for deploying a scalable web application stack consisting of containerized components

● Web Server: Containerized Any HTTP Server (eg. node.js)

● Database Server: Containerized MySQL or any NoSQL

● Application Server: Containerized lightweight Python or Node.js application

Lets get started, with creating a demo-app consisting of web-server, db-server and app-server with basic codes. After that add Dockerfile for every service, to implement the best practices for containerization, for optimization and container image management. Here is the basic architecture of directory.

To simplify the management and orchestration of multiple Docker containers that work together to form a complete application or system. I created one docker-compose file.

version: '3'

services:
  app-server:
    build:
      context: ./app-server
    image: sailesh081/demo-app-devops-app-server:latest
    ports:
      - "3000:3000"
      
  web-server:
    build:
      context: ./web-server
    image: sailesh081/demo-app-devops-web-server:latest
    ports:
      - "5000:5000"

  db-server:
    build:
      context: ./db-server
    image: sailesh081/demo-app-devops-db-server:latest
    ports:
      - "1433:1433"        

Now, run docker-compose up -d to start and run a multi-container application defined in a Docker Compose file in detached mode.

As i have docker desktop installed so can be checked from docker desktop too,

We can check this container from browser too by running localhost:5000

By utilizing the container orchestration tool eg: Kubernetes, for managing the containerized infrastructure with implementing infrastructure provisioning using a local virtualization tool called minikube, where minikube is running inside docker vm.

By implementing auto-scaling capabilities for handling increased demand on the web application i have written kubernetes manifest and service file to deploy our application inside multiple pods with proper considerations.

Under deployments folder i have created,

app-server-deployment.yml file which contains deployment code for app-server.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app-server
  template:
    metadata:
      labels:
        app: app-server
    spec:
      containers:
      - name: app-server
        image: sailesh081/demo-app-devops-app-server:latest
        ports:
        - containerPort: 3000        

Similarly, for db-server-deployment.yml file for database,

apiVersion: apps/v1
kind: Deployment
metadata:
  name: db-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: db-server
  template:
    metadata:
      labels:
        app: db-server
    spec:
      containers:
      - name: db-server
        image: sailesh081/demo-app-devops-db-server:latest
        ports:
        - containerPort: 3306
        

web-server-deployment.yml file,

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-server
  template:
    metadata:
      labels:
        app: web-server
    spec:
      containers:
      - name: web-server
        image: sailesh081/demo-app-devops-web-server:latest
        ports:
        - containerPort: 8000        

Similarly, for the kubernetes services i have created 3 separate service to expose and manage containers by enabling Service Discovery with providing different access mode as type Load Balancer or NodePort.

app-server-services.yml,

apiVersion: v1
kind: Service
metadata:
  name: app-server-loadbalancer
spec:
  selector:
    app: app-server
  ports:
    - protocol: TCP
      port: 3000
      targetPort: 3000
  type: LoadBalancer

---

apiVersion: v1
kind: Service
metadata:
  name: app-server-nodeport
spec:
  selector:
    app: app-server
  ports:
    - protocol: TCP
      port: 3000
      targetPort: 3000
  type: NodePort        

db-server-services.yml,

apiVersion: v1
kind: Service
metadata:
  name: db-server-loadbalancer
spec:
  selector:
    app: db-server
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306
  type: LoadBalancer

---

apiVersion: v1
kind: Service
metadata:
  name: db-server-nodeport
spec:
  selector:
    app: db-server
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306
  type: NodePort        

web-server-services.yml,

apiVersion: v1
kind: Service
metadata:
  name: web-server-loadbalancer
spec:
  selector:
    app: web-server
  ports:
    - protocol: TCP
      port: 5000
      targetPort: 5000
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: web-server-nodeport
spec:
  selector:
    app: web-server
  ports:
    - protocol: TCP
      port: 5000
      targetPort: 5000
  type: NodePort        

We can apply all these manifest and service file by running the following commands in respective directory.

kubectl apply -f app-server-deployment.yml        

kubectl apply -f app-server-services.yml        

kubectl apply -f db-server-deployment.yaml        

kubectl apply -f db-server-services.yml        

kubectl apply -f web-server-deployment.yaml        

kubectl apply -f web-server-services.yml        

Here is the results attached after running them,

As i have minikube as running on local vm you can check it through kuberrnetes dashboard too by running cmd,

minikube dashboard

To ensure high availability and fault tolerance in your Docker application. This means that if one container fails or stops running, another replica can immediately take its place to maintain the service. This ensures that your application remains accessible and functional even in the event of a failure.I have configured with 3 replica set when deploying the app so we can view 9 services for each 3 pods active.

As the application is deployed we need to have proper monitoring and logging to maintain the

health of the application. So, i configured grafana and prometheus on kubernetes using Helm,

• Grafana as data visualization tool for container orchestration platform.

• Prometheus as a data source which stores data in a time series database.

• And Helm chart for prometheus and grafana to arrange and manage the pods and

services in our cluster.

To configure this, Start minikube and install prometheus repository,

Add prometheus repo:

 helm repo add prometheus-community https://prometheus-community.github.io/helm-charts        

Install Helm Chart for prometheus

helm install prometheus prometheus-community/prometheus        

As it is installed, now we have to expose the prometheus-server service via NodePort because

it will be installed as clusterIP where only from kubernetes cluster can have communication so

to check from our host machine we need to expose.

kubectl expose service prometheus-server --type=NodePort --target-port=9090 --name=prometheus-server-np        

Now we can access Prometheus by UI also by running below cmd,

minikube service prometheus-server-np --url        

Similarly, install Grafana;

Add Grafana Helm repo:

helm repo add grafana https://grafana.github.io/helm-charts        

Install grafana:

helm install grafana grafana/grafana        

Expose Grafana service via NodePort in order to access Grafana UI;

kubectl expose service grafana --type=NodePort --target-port=3000 --name=grafana-np        

get grafana credentials

kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo        

Access grafana from Url

minikube service grafana-np --url        

After login, add data source as prometheus, use prometheus-server:80 as url because it is

running on local machine.

After adding data source import prometheus dashboard, I use community based Grafana

dashboard with id 6417 (you can use any available id)

After login, add data source as prometheus, use prometheus-server:80 as url because it is

running on local machine.

After adding data source import prometheus dashboard, I use community based Grafana

dashboard with id 6417

Finally, we can see the final dashboard of the grafana where we can see different sections with

• Cluster health

• Deployments

• Pods

• Containers etc...

Up above we configure everything using manually. Now, we can configure all these using Infrastructure as a Code, For this i used Terraform. Here is the folder structure for terraform.

For terraform code you can check on github link

By this, We can ensure that

Infrastructure as Code scripts or configuration files for provisioning the containerized web application stack on how to

• Set up and configure the containerized infrastructure.
• Deploy the containerized web application stack.
• Implement a CI/CD pipeline for automated testing, building, and deploying the containerized web application stack.
• Integrate logging and monitoring solutions specifically tailored for containerized environments.
• Utilize container registry services for managing container images.

Github code link: https://github.com/saileshkhadka/app-devops-microservices-deploy

Happy Learning !!