Avoid These Kubernetes Anti-Patterns

DevOps has come a long way, and Kubernetes is quickly taking over the technology world. Kubernetes is an open-source container orchestrator system: it automates the deployment, scaling, and management of containerized applications. It is a powerful tool to manage distributed clusters of containers economically and reliably. While it is considered a sophisticated tool, it can throw challenges at you if it’s not configured correctly. If not taken care of, these Kubernetes mistakes will lead to failures in your production environments.

Understanding the basic architecture and how Kubernetes works is essential in order to avoid these pitfalls. This blog will explore some common mistakes in Kubernetes deployments, how they work, and how you can fix them – or avoid them altogether with some simple tips.

The more complicated your deployment, the more likely you will make one of these mistakes.

1. Ignoring Health Checks

While deploying services to Kubernetes, health checks are very important to help you keep services running as expected. To confirm whether everything is working fine, it is important to know the status of the pods and the overall health of the Kubernetes cluster. To do this, there are startup, liveness, and readiness probes that will help you know the status of your app and the services running inside them. The startup probe ensures the pod is initiated and created successfully. The liveness probe lets us test whether the application is alive. The readiness probe is employed to determine whether the application is ready to receive traffic.?

2. Mounting Host File Systems in Containers

Mounting host file systems in containers is a common anti-pattern that leads to many failures. First, it is essential to know that all files created or modified inside the container are not visible to the outside world.

The primary use case for mounting host file systems in containers is persisting data. The simplest way of doing this is by mounting the host’s local directory as one of the directories within the container’s file system. This way, anything written to that directory will be persisted on the host machine. However, mounting your host file system does come with consequences:

• You can’t share state across multiple containers (i.e., you can’t mount two different directories onto two different hosts).
• Any changes on your host’s file system will be hidden from other containers.
• You can’t manage files on any mounted directories without changing their ownership and permissions.

To avoid these consequences, don’t mount any file systems of your host inside a container unless you need them for data persistence purposes.

3. Using the ‘Latest’ Tag

Using the latest tag in production creates chaos.. As it isn’t clear enough about the version and other descriptions, it is not recommended to use it in production. Furthermore, it creates more confusion when things break, and you need to bring things back to the available state as you won’t know what version of the app is running. Therefore, it would be best to always use meaningful Docker tags. Many of us think that the tag ‘Latest’ always points to the newly-pushed version of an image, but that is not the case. By default, the image gets the tag ‘Latest,’ but it doesn’t mean anything.?

4. Deploying a Service to the Wrong Kubernetes Node

Kubernetes is a complex system, and one of the most common mistakes beginners make is deploying a service to the wrong node. In Kubernetes, nodes are either master nodes or worker nodes. Every job in Kubernetes has a controller and a scheduler. The controller runs on a master node, and the scheduler runs on a worker node. The master node’s primary function is synchronizing with its corresponding workers and managing cluster-level resources like volumes, network, and persistent data storage.?

Worker nodes only run tasks assigned by their masters. This means that if you deploy your service to the wrong node, it may not work correctly – or at all! Also, it will take longer than expected for your new containers to start up, because they will need to wait for an available scheduler to assign tasks before it starts anything else.

To avoid this, you should always know which type of node your services are running on – master or worker – before deploying them. You should also check whether the pod has access to other pods in the cluster that it needs to communicate with before launching any containers.

5. Not Employing Deployment Models

Application deployment is a challenging task for developers, and Kubernetes makes it easy with its numerous deployment techniques. To keep your application available and make sure users don’t get affected by possible downtimes while deploying new software, Kubernetes recommends using deployment strategies:?Blue-Green, Canary, and Rolling .?

6. Duplicated Deployments

One of the most common mistakes in Kubernetes is when a deployment strategy is duplicated. This happens when we create more than one replica of the same state, deployed in parallel to different clusters.?

What does this mean? Essentially, this means that if one cluster goes down, the other will continue to process requests for your deployment. However, when they come back up (or if you add them), both replicas will be processing requests and doubling your requests because there are two sets of replicas running. This can be bad news, as it may oversubscribe the CPU and memory on the underlying hosts. To fix this mistake, we would recommend using a service type such as Headless Service or Daemon Set so that only one version of the deployment is running at any given time.

7. Using Only One Kind of Container (i.e., Stateless) in Production Environments

Initially, containers and containerization were designed for stateless applications, but then there went a lot of effort to support stateful applications. With Kubernetes enabling containerization and supporting modern data-driven applications, it has become critical to employ stateful applications.?

One common mistake developers make is using only one kind of container – usually stateless – in production environments where they should be using both stateful and stateless containers. Many people mistakenly believe that all containers are the same, but they have significant differences. Stateful containers allow you to store data on persistent storage like disks, which means they’ll never lose data. In contrast, stateless containers will keep their data as long as they’re running, after which it’s lost forever (unless backed up). Hence, it is a good practice to make use of both stateful and stateless containers.?

8. Not Setting Resource Consumption Limits

If you see your resource utilization and your bills shooting up, then it’s time you take control and determine which services are needed and which ones are not. One way is to perform a stress test against your application.??

Then, you can set a limit on the CPU and the memory of containers. Kubernetes defines ‘requests’ and ‘limits’ in its resource utilization category. Requests represent the minimum resources an application needs to run, and limits define the maximum resources. No control over the resources also means we are not monitoring the application. We can specify the resource limits in the deployment YAML.

Harness Cloud Cost Management ?(CCM) gives suggestions for your Kubernetes clusters by showing resource optimization opportunities to reduce your monthly spending. The recommendations are computed by analyzing your workload’s past utilization of CPU and memory. The implementation uses a histogram method to compute the recommendations.

Kubernetes is great, but the learning curve can be daunting at times. We knew developers were facing issues running Kubernetes, hence, we listed these common mistakes and pitfalls with tips to avoid them so that you can work efficiently with your Kubernetes deployments. Additionally, these mistakes can be avoided by paying close attention to your interactions with Kubernetes and understanding the differences between how it interacts with your deployed services.

Make sure you’re doing your due diligence before your application goes live.

Also, take a look at my new article (link shared below) that talks about some important tricks to optimize your cloud costs.

Let me know what other anti-patterns or mistakes you have encountered or know while employing Kubernetes.

Thanks:)

Credits: My own article on DZone