Integrating AWS Services using AWS Eks

Task Description :- Implementing EKS Cluster with different AWS Services Like EC2 , EFS, ELB,EBS and Monitoring Cluster Using Graphana and Prometheus . Creating Fargate Cluster and running pods on fargate cluster.

What is EKS:- EkS is a fully managed kubernetes service provided by AWS. EKS stands for elastic kubernetes service. In this service Kubernetes master is fully managed by AWS we have to only manage our worker/slave node. Obiously we want to deploy our Application so we need to manage worker node. Good thing is that it is provided by Amazon so we can integrate EKS with EC2,EFS,ELB,EBS and many more services of AWS and we have high availability of K8S cluster as master is managed by AWS .

First we have to create AWS EKS Cluster :-

For this we will Install one automation tool that is Eksctl this tool was created by Weaveworks. You can search on Google for Eksctl installation. It will create EKS cluster for us and behind the seen it creates the CloudFormation stack So you can see this stack on cloudformation Console.

Why we need EKSCTL ?

We need this tool as we want automation. We have to plan for the cluster like how many nodes we want and what will be the type of instance. Although you can launch it from WebUI also but as a technical man we want to see what's happening behind the seen so we used CLI.EKSCTL is a Cli tool and it is also now the official tool of AWS.

STEP BY STEP PROCEDURE:-

First We have to create one YAML code in which we have to create two node groups. In first node group We will request for On-demand instances that are costly and in second node group we want Spot instances which are cheaper than On-demand. We have to attach a key-pair so that we can login to instance.

Now to Run this Code we have to run following command :-

# eksctl create cluster -f cluster.yml (This process may take some time)

You can watch this process in cloudformation tab also

Here Behind the seen they are launching our desired instances and Kubenetes Master that will not be Visible for us because that is managed by AWS but we can see EC2 instances in the EC2 tab.

After Suceessfull completion of above Steps following will be shown on the Output Screen.

You can also check by running #eksctl get cluster

Now we have to update K8S configuration file.Sometimes it might be automatically update, if zone is same as you have mentioned in AWS configure command or you can update it by running this command =>

#aws eks update-kubeconfig --name yogicluster

Now we can run KubeCTL command and create pods.

Now,if we want that our client can see our website ,we can expose it using LoadBalancer and it will automatically create one loadbalancer for our website as all K8s services are managed by EKS and EKS is tightly coupled with other AWS services

You can see below that there is no LoadBalancer but as we run the command LoadBalancer will automatically created and we can see our website.

We can also go inside this container by runing K8s exec command and see what's inside the pod. #kubectl exec -it mylwweb-5d945b8578-m8xvc -- bash 

We can also create the Persistent storage for this we have to write code given below:-

Here PVC will not be created because of the Storage Class policy. Untill the PVC will not attach to any pod they will not create it. If you'll describe the storage class then you will observe that policy is already set to WaitForFirstConsumer .So,if any pod will use this PVC Storage Classs will get the storage from EBS.

So,if we want to create our PVC we have to edit our deployment and Attach PVC to POD , then we can observe PVC is created successfully.Now, we have to claim the volume and Mount it on /var/www/html .

So,now in above image you can see that PVC is successfully created .

Now,we can create a Wordpress website easily for this we will mount one PVC to /var/lib/mysql so that if anyhow pod got deleted we will not loose our database data and will also create one more pvc and mount it to /var/www/html so we will not loose our website data like username and password . We will also create one Box called Secret in which we will store our credentials like MYSQL_PASSWORD and refer this in the code so our credential will be safe.We will also create one Kustomize file . Kustomize is a standalone tool to customize Kubernetes objects through a kustomization file and our entire setup will be done .

In One click our entire setup will be ready.

In PVC, we have to change one thing that is ReadWriteMany so that at the same time many clients can acces it and write data on it means, create posts. But EBS does not support ReadWriteMany so we will solve this issue by using EFS later.

Now we can monitor these pods by using Prometheus and Graphana for this we need Helm that is K8s Package Manager.It will install Prometheous automatically and automatically they will start monitoring our pods.

First,we have to install Helm in our local system and we also have to install Tiller in our system.If you install v3.0+ of Helm it will automatically configure it.Only you have to set Admin privilllage of Kube-System so that it can launch pods.

We will create one NameSpace that's called Prometheus and then install Prometheus

Now We can Use PortForwarding to see our Prometheus.

Here,in the above image we can monitor our node and port.we can also see CPU and Storage usage in Prometheous and if we want to see it in Graphical form we will install Grafana.

Now we will provide Data Source to Graphana that is on our Promethus server so either you provide ip or hostname.

Now, we will import one dashboard Cluster Monitoring for Kubernetes.We will copy it's ID (10000) and paste in import tab. Now we can see a beautifull dashboard.

Now, if we want to Use EFS Storage so we have to do the following things:-

As of now our SC is using EBS provisioner so we have to creater EFS provisioner first and provide security and admin role. Now, we have to create one storage class .

Note:- We have to create EFS in the same VPC and provide the same security groups on which the instances are running on.

Now,we will edit our EFS and change the id of EFS and DNS name.

But,there is one problem, our container will not create so we have to install one utility (NFS-utilities) amazon-efs-utills in all the nodes after logging in via ssh and finally our pod will be created.

Finally, you can see our wordpress site is launched!!!

What is Fargate Service ?

AWS Fargate Service is a Serverless service which we can Integrate with EKS . By using this service we can minimize our Cost because EKS manage Manager and here no slave will be created previously.As the demand come up on the fly node will be launched with docker and all other dependencies and also our pod will be created.Due to some reasons we can not create Fargate profile in Mumbai region so we have to choose any other region like Singapore.

Now if we run #kubectl get nodes command we can see two nodes are there but you can not see them in E2 tab .

As soon as we launch one pod, Node will be automatically created.

Github Code link :- Eks Task code