Autopilot Kubernetes Cluster
Rahul Udayshankar Sinha
![Rahul Udayshankar Sinha]()
Rahul Udayshankar Sinha
FinTech | Cloud Architecture | Site Reliability Engineering | DevOps | Infrastructure as Code | Kubernetes | Data Analytics | Linux | Security & FinOps | Ultra Low Latency Trading Tech
Autocluster GKE Installation Steps:
2. Create Cloud NAT gateway with Standard Service Tier
3. Advanced Cloud NAT Gateway Configuration
4. Started creating GKE Cluster in Autopilot mode
5. GKE Autopilot cluster - Basic Configuration
6. 5. GKE Autopilot cluster - Network configuration
Connect to GKE Autopilot Cluster
领英推荐
rahul@rahul-intel-nuc:~$ sudo addgroup sre --gid 5000
Adding group `sre' (GID 5000) ...
Done.
rahul@rahul-intel-nuc:~$ sudo adduser sre --gid 5000 --uid 5000
Adding user `sre' ...
Adding new user `sre' (5000) with group `sre (5000)' ...
Creating home directory `/home/sre' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for sre
Enter the new value, or press ENTER for the default
Full Name []: sre
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
Adding new user `sre' to supplemental / extra groups `users' ...
Adding user `sre' to group `users' ...
rahul@rahul-intel-nuc:~$ cat /etc/group | grep 5000
sre:x:5000:
rahul@rahul-intel-nuc:~$ cat /etc/passwd | grep 5000
sre:x:5000:5000:sre,,,:/home/sre:/bin/bash
rahul@rahul-intel-nuc:~$ su - sre
Password:
sre@rahul-intel-nuc:~$
2. Connect to GKE Cluster by generating kubeconfig entry
sre@rahul-intel-nuc:~$ gcloud auth login
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555..ode_challenge_method=S256
Enter authorization code: 4/0AfJohXk3i7jHs..
You are now logged in as [[email protected]].
Your current project is [None]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
sre@rahul-intel-nuc:~$ gcloud config set project sara-project
WARNING: You do not appear to have access to project [sara-project] or it does not exist.
Are you sure you wish to set property [core/project] to sara-project?
Do you want to continue (Y/n)? Y
Updated property [core/project].
sre@rahul-intel-nuc:~$ gcloud container clusters get-credentials sara-autopilot-private-cluster-1 --region us-central1 --project sara-project-401601
Fetching cluster endpoint and auth data.
kubeconfig entry generated for sara-autopilot-private-cluster-1.
sre@rahul-intel-nuc:~$
3. Get number of nodes in GKE Autolpilot Cluster
sre@rahul-intel-nuc:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
gk3-sara-autopilot-priva-default-pool-47750026-sxwn Ready <none> 42m v1.27.3-gke.100
gk3-sara-autopilot-priva-default-pool-8580b4db-6rfp Ready <none> 42m v1.27.3-gke.100
sre@rahul-intel-nuc:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gk3-sara-autopilot-priva-default-pool-47750026-sxwn Ready <none> 42m v1.27.3-gke.100 10.128.0.35 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
gk3-sara-autopilot-priva-default-pool-8580b4db-6rfp Ready <none> 42m v1.27.3-gke.100 10.128.0.34 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
sre@rahul-intel-nuc:~$
4. Deploy nginx to Autopilot GKE cluster
sre@rahul-intel-nuc:~$ mkdir kube-manifest
sre@rahul-intel-nuc:~$ cd kube-manifest/
sre@rahul-intel-nuc:~/kube-manifest$ vim 01-deployment.yaml
sre@rahul-intel-nuc:~/kube-manifest$ vim 02-service.yaml
sre@rahul-intel-nuc:~/kube-manifest$ cd ..
sre@rahul-intel-nuc:~$ cat kube-manifest/01-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: #Dictionary
name: sara-app-deployment
spec: # Dictionary
replicas: 5
selector:
matchLabels:
app: sara-app
template:
metadata: # Dictionary
name: sara-app-pod
labels: # Dictionary
app: sara-app # Key value pairs
spec:
containers: # List
- name: sara-app-container
image: nginx:stable-alpine3.17-slim
ports:
- containerPort: 80
resources:
requests:
memory: "128Mi" # 128 MebiByte is equal to 135 Megabyte (MB)
cpu: "200m" # `m` means milliCPU
limits:
memory: "256Mi"
cpu: "400m" # 1000m is equal to 1 VCPU core
sre@rahul-intel-nuc:~$ cat kube-manifest/02-service.yaml
apiVersion: v1
kind: Service
metadata:
name: sara-app-lb-service
spec:
type: LoadBalancer # ClusterIp, # NodePort
selector:
app: sara-app
ports:
- name: http
port: 80 # Service Port
targetPort: 80 # Container Port
sre@rahul-intel-nuc:~$ kubectl apply -f kube-manifest/
Warning: autopilot-default-resources-mutator:Autopilot updated Deployment default/sara-app-deployment: adjusted resources to meet requirements for containers [sara-app-container] (see https://g.co/gke/autopilot-resources)
deployment.apps/sara-app-deployment created
service/sara-app-lb-service created
sre@rahul-intel-nuc:~$
5. Get the deploy , pod and service created in Autopilot GKE Cluster
sre@rahul-intel-nuc:~$ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
sara-app-deployment 2/5 5 2 25s
sre@rahul-intel-nuc:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sara-app-deployment-9bbcfdbb9-68vkp 1/1 Running 0 31s
sara-app-deployment-9bbcfdbb9-8qldm 0/1 Pending 0 31s
sara-app-deployment-9bbcfdbb9-m2982 0/1 Pending 0 31s
sara-app-deployment-9bbcfdbb9-t4rjz 0/1 Pending 0 31s
sara-app-deployment-9bbcfdbb9-z8npq 1/1 Running 0 31s
sre@rahul-intel-nuc:~$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 34.118.224.1 <none> 443/TCP 50m
sara-app-lb-service LoadBalancer 34.118.226.191 35.232.181.29 80:31602/TCP 42s
sre@rahul-intel-nuc:~$ curl https://35.232.181.29
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a >nginx.org</a>.<br/>
Commercial support is available at
<a >nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
sre@rahul-intel-nuc:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sara-app-deployment-9bbcfdbb9-68vkp 1/1 Running 0 77s
sara-app-deployment-9bbcfdbb9-8qldm 0/1 Pending 0 77s
sara-app-deployment-9bbcfdbb9-m2982 0/1 Pending 0 77s
sara-app-deployment-9bbcfdbb9-t4rjz 0/1 Pending 0 77s
sara-app-deployment-9bbcfdbb9-z8npq 1/1 Running 0 77s
sre@rahul-intel-nuc:~$
6. GKE Autopilot cluster automatically triggers Cluster Scale up
sre@rahul-intel-nuc:~$ kubectl describe pod sara-app-deployment-9bbcfdbb9-8qldm
Name: sara-app-deployment-9bbcfdbb9-8qldm
Namespace: default
Priority: 0
Service Account: default
Node: gk3-sara-autopilot-private-clu-pool-1-0743bc71-9sc9/10.128.0.36
Start Time: Tue, 24 Oct 2023 13:13:08 +0800
Labels: app=sara-app
pod-template-hash=9bbcfdbb9
Annotations: <none>
Status: Pending
SeccompProfile: RuntimeDefault
IP:
IPs: <none>
Controlled By: ReplicaSet/sara-app-deployment-9bbcfdbb9
Containers:
sara-app-container:
Container ID:
Image: nginx:stable-alpine3.17-slim
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Limits:
cpu: 250m
ephemeral-storage: 1Gi
memory: 512Mi
Requests:
cpu: 250m
ephemeral-storage: 1Gi
memory: 512Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jgggg (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-api-access-jgggg:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: kubernetes.io/arch=amd64:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 99s (x2 over 100s) gke.io/optimize-utilization-scheduler 0/2 nodes are available: 2 Insufficient cpu. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod..
Normal TriggeredScaleUp 92s cluster-autoscaler pod triggered scale-up: [{https://www.googleapis.com/compute/v1/projects/sara-project-401601/zones/us-central1-c/instanceGroups/gk3-sara-autopilot-private-clu-pool-1-0743bc71-grp 0->1 (max: 1000)}]
Normal Scheduled 18s gke.io/optimize-utilization-scheduler Successfully assigned default/sara-app-deployment-9bbcfdbb9-8qldm to gk3-sara-autopilot-private-clu-pool-1-0743bc71-9sc9
sre@rahul-intel-nuc:~$
7. As Scale completed, Pods are assigned to the GKE nodes and all the nodes are ready
sre@rahul-intel-nuc:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sara-app-deployment-9bbcfdbb9-68vkp 1/1 Running 0 115s
sara-app-deployment-9bbcfdbb9-8qldm 0/1 ContainerCreating 0 115s
sara-app-deployment-9bbcfdbb9-m2982 0/1 ContainerCreating 0 115s
sara-app-deployment-9bbcfdbb9-t4rjz 0/1 ContainerCreating 0 115s
sara-app-deployment-9bbcfdbb9-z8npq 1/1 Running 0 115s
sre@rahul-intel-nuc:~$ kubectl top pods
NAME CPU(cores) MEMORY(bytes)
sara-app-deployment-9bbcfdbb9-68vkp 0m 2Mi
sara-app-deployment-9bbcfdbb9-z8npq 0m 2Mi
sre@rahul-intel-nuc:~$ kubectl get pods -w
NAME READY STATUS RESTARTS AGE
sara-app-deployment-9bbcfdbb9-68vkp 1/1 Running 0 2m29s
sara-app-deployment-9bbcfdbb9-8qldm 1/1 Running 0 2m29s
sara-app-deployment-9bbcfdbb9-m2982 1/1 Running 0 2m29s
sara-app-deployment-9bbcfdbb9-t4rjz 1/1 Running 0 2m29s
sara-app-deployment-9bbcfdbb9-z8npq 1/1 Running 0 2m29s
sre@rahul-intel-nuc:~kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
sara-app-deployment 5/5 5 5 2m43s
sre@rahul-intel-nuc:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gk3-sara-autopilot-priva-default-pool-47750026-sxwn Ready <none> 52m v1.27.3-gke.100 10.128.0.35 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
gk3-sara-autopilot-priva-default-pool-8580b4db-6rfp Ready <none> 52m v1.27.3-gke.100 10.128.0.34 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
gk3-sara-autopilot-private-clu-pool-1-0743bc71-9sc9 Ready <none> 2m27s v1.27.3-gke.100 10.128.0.36 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
sre@rahul-intel-nuc:~$
8. As the application is deleted, GKE Autopilot cluster scales down to 1 node
sre@rahul-intel-nuc:~$ kubectl delete -f kube-manifest/
deployment.apps "sara-app-deployment" deleted
service "sara-app-lb-service" deleted
sre@rahul-intel-nuc:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gk3-sara-autopilot-private-clu-pool-1-0743bc71-9sc9 Ready <none> 7m54s v1.27.3-gke.100 10.128.0.36 <none> Container-Optimized OS from Google 5.15.109+ containerd://1.7.0
sre@rahul-intel-nuc:~$