The Intuition Behind The Microservice Architecture Series - Part 2 - How We use Kubernetes for Deployment

Introduction

Kubernetes is relatively new. However, the underlying system i.e. Borg is more than a decade old. Google open sourced Kubernetes close to 4 years ago in July, 2015. As is typical with any new software, there are a number of new terminologies / concepts one needs to understand. Let us try to examine the list of these terms and see if we can understand what they mean.

A. Containers

It is the first term that comes to mind when a Container Deployment Platform like Kubernetes or Docker Swarm is discussed. What does the term mean in real life? A Container in real life is meant to contain other items. I can bet that all of us have seen one, I mean Containers. Imagine enjoying ourselves, waiting behind a railroad , watching a goods train passing slowly with 175 containers on it. 

What is the advantage of a Container? US Companies import millions of tonnes of items from other countries. Without Containers, imagine the workers at shipping terminals loading different items such as furnitures, steel products, toys, food items and others slowly and carefully on the ship. The process will be incredibly time consuming. In fact, the shipping industry was that complex (and slow) before shipping Containers made things easy in the 50s. With Containers however, things such as toys, furnitures, steel products, food items and other staff can easily be packed , loaded and unloaded from trucks, to ships, to trains, to trucks again and finally to Retailer's warehouse. This is our already known experience whether we know or heard about or love or hate Containers in software!

A software Container is a Container that carries software like a Furniture Container shipped from China contains Furniture. The advantage is the same. A Software Container , due to the solid packaging principles, can be offloaded i.e. deployed to personal laptops, integration environment, QA environments, staging and production alike. The Container itself comes with everything that the software needs to run, from the Operating System , the libraries the application uses along with the application itself. All we have to install in the various environment i.e. dev / prod / qa, is basic software that can support running the container. One such popular software is called , you guessed it, Docker. You can imagine that Docker is to the Software Container as a physical Ship is to a collection of physical container. Software Containers run on top of Docker as physical Container get transported across the oceans on top of Ships.

B. Declarative Deployment

What is a good example of declarative work? If we can imagine the difference between cooking our own dinner and eating out in a restaurant, it will be more clear. In the restaurant, we are specifying what we want to eat, not cooking ourselves. Kubernetes makes it possible for us to just specify how we want our Containers to be deployed, how much CPU , how much memory, how many replica instances we want Kubernetes to run for us. Here Kubernetes is playing the same role , the restaurant waiter / waitress and the Chef plays when we eat out. Similar to the Waiters/Waitresses asking us how much spice we want in our Chicken (Or it could be Veg) Burger / Pizza / Biriyani and how many plates we would like to order.

People who are familiar with the very popular SQL language will be familiar with declarative way of doing things. 

When we say

SELECT FIRST_NAME, LAST_NAME FROM EMPLOYEES;

The underlying system does awesome lot of work to get us the first and last names of employees from the employees table. Similarly, when we ask Kubernetes through declarative syntax to deploy our containers and maintain say, 4 instances all the time, we are delegating a lot of heavy lifting to the work horse that is Kubernetes.

Without understanding the details of Kubernetes, if we can realize how much fun we get from declarative way of doing things i.e. eating out and not cooking ourselves, we can understand what Kubernetes is bringing to the table. If we are still not clear about the heavenly pleasure of declarative work, we can try doing what Kubernetes does for us, ourselves but solely at our own risk.

C. Pods

Our industry has an uncanny habit of reusing english words and making the original meaning forgotten. How many people in our industry know that originally a Pod meant (I guess it still means) a group of live Whales (in an ocean). However, if we examine the original meaning of the word, we will realize why the Kubernetes creators choose this word for Kubernetes. In a live Pod, Whales stay close to one another. In a Kubernetes Pod, Software Containers stay close to one another. Kubernetes ensures that all containers in a single Pod will always be deployed to a single host and can call one another using localhost. 

There is a key difference between Whales in Pods and Containers in Kubernetes Pods, though. We will serve ourselves better to keep in mind that unlike the super fat Whales in live Pods (baby whales weigh more than several hundred pounds) , containers running within Kubernetes Pods are meant to be super lightweight.

D. Master Node and API Server

A healthy system , human or otherwise never runs depending on a single human or machine. A large restaurant has multiple Waiters / Waitresses, Chefs and other staff. Likewise Kubernetes is also made from various subsystems / modules running in a number of machines / nodes. One of the most important of these subsystems is the Master Node(s). If we are just learning Kubernetes , one Master Node is enough to explore. However, if we are thinking about a highly available production system, a Master Node that runs in multiple machines, typically 3 or 5 is more normal. Like a large busy restaurant has multiple Waiting staff, who collaborate between the ordering customers and the order executing Chefs, Kubernetes Master Nodes, more specifically the API Server that runs within the Master Node, interfaces between the ordering Kubernetes client (you and me) and the rest of system.

E. Worker Node and Scheduler

These are the Chefs in the Kubernetes restaurant! They do all the work receiving orders from the API Server and the Scheduler. The Master Node has a track of how many worker nodes it can delegate to and when it has not heard from a certain Worker Node, gets it restarted. When the API Server gets a new task of running a new set of Pods, it checks with the Kubernetes Scheduler (The Master Chef in the restaurant kitchen scheduling customer orders to Chefs who have bandwidth). The Kubernetes Scheduler determines the best worker node that has enough bandwidth in terms of CPU and memory to run these new pods. Once the Scheduler finds the best fitting worker nodes, it schedules them to run on that worker node.

F. Replication Controller

Imagine a restaurant has outsourced its waiting staff from a staffing company and asked for a 15 standing member during business hours. The staffing company has placed a 15 member crew in that restaurant. However, if one these people falls sick or quits, it is the staffing company that will promptly replace the missing member with a new one. In this case, the staffing company constantly watches and makes sure the required number of waiting staff are always present in the restaurant. 

The Kubernetes Replication Controller does the same job in slightly a different way. Each Container running inside Pods has to provide a readiness and liveliness APIs (at times they could be the same API) for Kubernetes to determine its health. When the liveliness API of a Container / Pod remains unresponsive for a certain time, Kubernetes kills it. The Replication Controller watches two things constantly, the desired number of pods, say 5 and the active pods in this case, say 4. The moment it (Replication Controller) realizes the active state has fallen short of the desired state, it calls the API Server to create a new Pod in the Kubernetes database etcd. The Scheduler then watches that a new Pod is created in the Kubernetes database but does not physically exist yet, it finds a new host to create a physical pod instance. This way the Kubernetes Replication Controller, and the Scheduler uses the etcd database to maintain that active state remains the same with the desired state.

G. The etcd Database

Chefs in a restaurant has memory or notebook to take orders before they execute them. As Kubernetes executes using Computers who can not remember things if they are restarted, Kubernetes needs a database to store staff it needs to remember upon restart. etcd is that clustered database kubernetes uses to store its data. When we say clustered, we mean data within the etcd database is replicated i.e. stored in 3 different data nodes, and one of the etcd data nodes crashing will not cause data loss.

H. Service

Kubernetes Pods are like Chefs in a Restaurant in way. When we eat out, we cannot directly talk to the Chefs themselves. True, Chefs can communicate on their own but do not. We need to go through the waiting staff, who rotate (load balances) our orders for execution among the available Chefs. Kubernetes Service is like the Waiting Staff for the external clients. They take orders from external client, and distributes the orders / api calls to the available Chefs i.e. Pods. Restaurant might replace Chefs internally as Kubernetes replaces Pods but the Service remains available for clients. Like newly available Chefs let the Waiting staff know about their availability, newly coming up Pods, make the Kubernetes Service know (again through the etcd database and the API Server) that they are available for being load balanced for client calls.

More To Come

This remains a complex subject and there is more to come. Please stay tuned about auto scaling using the Horizontal Pod Scheduler (HPA), Resource (CPU / Memory) Requests and Limits and more...

Thanks,

Binit