Every Silver Lining Has A Cloud - Part 4

Scaling for business, scaling for cost

This fourth and penultimate installment of this set of posts concerns the setup and configuration to manage a scalable powerful solution to ensure a solid robust service for your consumers and manage costs. Ideally what we want is a linear scaling set of costs as our user base scales. Certainly cloud based providers give this ability, but what is also available is that hour by hour, minute by minute, the cloud provider can scale up or down systems to keep the costs low in periods of low utilization. For example, if your clients are predominantly based in the USA then there may be long periods of say 10 or so hours where the system can be scaled down.

In addition, we will conclude on a few items to complete the checklist from part 1 and check off each item that this is all covered by the setup and configurations given within these posts.

Containers

We have briefly mentioned about containers in previous posts but just to put some contours around it, containers are literally that, a container which houses an environment containing everything you need configured in that environment. It could be Linux, Windows or any other operating system.

This would either be a virtual machine or an image which, when uploaded to a system which understands the construct of the image can create instances of that image to provide your clients a service. A popular container is Docker which I will refer to over the course of this post.

So we have a docker container. Let’s say for the purposes of simplicity that this container has our application service running which provides a website for clients to connect to. Remember this is just a container at the moment. We need to ‘spin up’ or make at least one instance of this container, assign it appropriate networking and then it can provide meaningful work.

Container Storage

With our containers we need somewhere to store them and make them available to systems that can take the containers and spin them up. This is where container storage comes in. We also utilize container storage to provide versioning to the products. We simply ‘tell’ the controlling technology that we have a new version, the controlling technology will spin up a new instance running the new version and start routing traffic to this new instance. At the same time, it will slowly drain the traffic from the old version and when all drained off, it will shut down and terminate the old version and voila!, you have upgraded your environment.

Container Computation Engine – Serverless!

As previously mentioned, we have our containers containing our product, we have a storage area to save our containers and now we need to make them available to a ‘controlling technology’ which will perform the spinning up and downs of the containers. This is where we go serverless. Instead of managing the servers ourselves, we simply permission the computation engine to spin up instances of containers in some controlled manner.

Typically we use this kind of serverless environment to host containers containing web applications, micro services and web API’s. Remember in the past you may have deployed all these services on one system but with this configuration and flexibility, you can deploy onto separate containers giving immutability. If you configure in such a way, you have a good security framework and if the interconnectivity is setup correctly, there is very little that can be compromised. Using separate passwords maintained by key management services just adds to the security.

The container computation engine removes the need to provision, configure, or scale clusters of virtual machines to run containers. The knock on effect is that you no longer need to choose the size and power of the servers and you can optimize when to scale.

Time consuming system updates – Gone!

By using the container computation engines, you no longer need to patch, secure and manage servers again saving time and the need for qualified personnel. As mentioned previously, for valuable updates, i.e. updates to your software as you evolve the product, your DevOps environment can create the new container (e.g. in Docker), upload it to container storage and then register it with the container controller which will systematically drain all existing containers running and spin up your new version. You get a log of bang for buck utilizing these technologies.

Configuration of your containers

So you have a container for your application but you need to be able to configure the environment it runs in if you want to. Remember you almost certainly have monetary constraints on your environment. So, the computation engine can be configured for:

• How much CPU and memory to use for each container?
• The underlying infrastructure where you would like your containers hosted
• The networking modes for your containers
• The logging configuration for your containers
• The command that the container runs when it's started
• Any data volumes that are used with the containers?
• The specific access and security roles that your containers use
• What to do in the event of any failure?

?You can also logically group a set of containers into a group and spin those up in a logical set which means that your product which may have a number of moving parts, is scaled up accordingly.

So next we look at options as to how we dictate the number of container instances that are spun up to handle the load from your customers.

Application Load Balancing

An Application Load Balancer operates at the request level of the networking stack (layer 7). This routes traffic to targets based on the content of the request. This is ideal for advanced load balancing of HTTP and HTTPS traffic typically any browser based application. The Application Load Balancer provides advanced request routing targeted against delivery of modern application architectures, including microservices and container-based applications. The Application Load Balancer also simplifies and improves the security of your application, by ensuring that the latest SSL/TLS ciphers and protocols are used at all times.

?This last point is worth stressing as it opens up 2 important points. The load balancer will take your secure traffic from the public internet, decrypt it (in hardware) and pass it to internal containers and services within your private network in an unencrypted form. This means that the containers do not need to decrypt (which puts additional load on the server). Secondly, TLS/SSL certificates expire but the cloud provider will keep the certificates up to date for you therefore removing the need to renew and reapply certificates.

Why is an application load balancer used?

An application load balancer serves as the single point of contact for your customers. It abstracts the end point where customers connect to (typically a URL) and maps that to a container or system which will handle the request. The load balancer distributes incoming application traffic across multiple targets but also if configured, multiple discrete datacenters to provide redundancy. Your customer need never know a particular container or system is unavailable. Your availability of your application increases.

The Container Compute Engine Architecture

So putting all this together we use containers to house our application, store the containers and spin up the containers. Using application load balancers we can distribute our load to a number of application containers across a number of datacenters for redundancy. But if we have too many containers we have unnecessary costs and if we have too few containers, our customers may suffer with poor performance.

?To solve the problem we need to address 2 metrics. The first is that is the container system is up and running and the second is that under what load is the specific container. Fortunately under the load balancer each container reports this health metric (load + availability) back to the load balancer. This allows the load balancer to shape the traffic accordingly. Now, let’s assume that the load suddenly spikes up (as your customers start using your system), what is the next step for the load balancer?

Serverless Functions

The cloud provider has a set of services which run code or instructions (functions) without provisioning or having to manage any servers. These functions can be anything from removing old versions of your containers from the application pool, performing system maintenance, logging information or other custom functions. These functions can also take constant inputs or respond to events and then action accordingly. The best thing is that these functions can be a simple as a few clicks to a full piece of code with all major languages typically supported.

Taking Inputs To Manage Your Systems – Power Management

Within the cloud provider environment, we can define such a function that when the application load across all containers reaches 70% for 5 minutes or more then we spin up another instance and have the function continue to do this until 60% container load is achieved up a maximum number of 10 instances. All of this is completely customizable. We simply need to define a function which takes the health of all the containers and makes a call when certain thresholds are met.

Taking Inputs To Manage Your Systems – Cost Management

Within the cloud provider environment, we can define such a function that when the application load across all containers drops below 60% for 5 minutes or more then we spin down an instance. Ah yes, but which instance? Well, we don’t care, we just notify the load balancer to stop using this particular instance and when all requests have been processed, the container will spin down automatically.?Imagine how powerful this is, overnight, while you are asleep you must maintain a presence so your systems are up and respond to requests but because the load is so low, the number of application containers is 1. This keeps the system at the lowest possible price and manages costs accordingly. You can also instruct the containers to spin up lower numbers of virtual CPU’s or memory again to minimize costs. Think of the function as keeping the lights on but barely until an event comes in (usually increased traffic) which starts the automatic scaling process.

Ensuring Secure Containers

While a good solid infrastructure is always preferrable to facilitate a strong foundation to build your applications, containers, production and DevOps environments on, we should always ensure that our containers are hardened to make sure that we further reduce potential unauthorized access, denial of service attacks and other cyber threats by ensuring that our environments are as locked down as possible.

?Imagine you install Microsoft Windows Server. From the onset the server comes with no functionality installed. Then you add Microsoft Internet Information Services onto the server. How can you be assured that all the potential security holes or even configurations that may lead to a potential bad actor discovering something about your system are closed?

?This is where hardened machine images come into play.

Hardened Machine Images

Hardened machine images are operating system or other such images where a reputable 3rd party has closed off or otherwise restricted the operating system by making configuration changes to the system. The Center For Internet Security (CIS) images are a good start. There will be a number of these on the marketplace stores in the cloud provider.?

These normally come in 2 flavors. The first is a compliant system where there is no or little loss of functionality. For example let’s say you are running a web server and you harden it to remove the type of web server. So, on any request, the caller doesn’t know whether (for example) the web server is Apache or Microsoft. This is a simple change but highlights the point.

?The second flavor reduces functionality in specific areas and is used where that extra level may be required. In this case you will need to test that your application still works. However, as we shall see in the DevOps section in the final post, this should be of little problem if the same machine images are used throughout the DevOps lifecycle. If it fails in the development cycle there is plenty of time to fix the problem so that your staging, pre-prod and production environments are well prepared.

Closing the checklist

These are a few items which don’t really fit into specific areas as they are generally add-ons to the infrastructure to provide specific functionality. They are detailed here to close the checklist.

Secure Domain Name System (DNS)

As an additional item, sometimes bad actors can hijack traffic to internet endpoints (could be yours!) by intercepting DNS queries and returning their own IP addresses to potentially fake sites mimicking your application. Cloud providers make this easy to configure especially if you use their DNS service. The basic approach is to establish a chain of trust from the source to the destination. If the DNS resolver in the chain does not support secure DNS then the authenticity checks are not performed. ?

Via the cloud provider, it is very likely that the cloud provider’s DNS service supports secure DNS which means that the top level domain (i.e. your route domain) should be registered with the cloud provider and the creation of the private/public key system required can be also setup easily to establish the chain of trust. ?

Another advantage (one of many as you have seen in these posts), the cloud provider (because it uses a key management system) can rotate the keys on a regular basis completely transparently to keep a high level of security. Imagine doing this yourself. Yes it can be done but, would you ever get round to it?

Single Sign On

As part of your corporate policy, internal systems that you are now wanting to move to the cloud ideally should have a single sign on approach so that when an employee leaves or joins, the whole process is seamless. In addition, multi-factor authentication can be enforced since there is only one place where the credentials are validated.

?In the corporate environment, it is likely that you will have an Active Directory system or some other appropriate authentication mechanism. You can use LDAP or some other directory protocol to provide this functionality but, in the event of using cloud based directory services (like Azure Active Directory), cloud providers have written specific enterprise applications to facilitate the smooth flow of traffic for authentication purposes.?

?I am just touching on this here because each system and setup is different but, if you are a traditional enterprise (and I use this as an example), you may have on-premise Active Directory Controllers which will sync to Azure Active directory. From Azure Active directory you install an enterprise application from the cloud provider and configure. Then, inside the cloud provider environment, you can assign certain rights for users and/or groups. Then, to log onto instances, cloud consoles etc. are all done via a single sign on mechanism. When the employee leaves the company all rights are centrally controlled. Similarly, when a new employee joins, simply add to the respective group on Active Directory, the group syncs to Azure Active Directory and then onto the cloud provider.

Identity And Access Management (IAM)

Firstly, what is identity and access management is a framework of policies and technologies to ensure that the right users have access to the right resources (and only those). This extends across hardware and software. Such controls are needed to ensure that a framework is in place for ever stringent security and compliance requirements. ?

If you are thinking that this is all about usernames, passwords, multi-factor authentication and access rights on those you are right but in the world of cloud computing we can extend this to produce some exceptional use cases which really do hammer home the massive return on investment for moving to the cloud. Stop thinking people think objects instead. These objects could be people, machines, services. If we allocate a right to them then the object can have the necessary access to another system or service. This is where the power comes in.?

Take for example the following setup. I have a serverless database service with the cloud provider and a number of container instances which connect to the database service. As you will see not a single person in involved but many objects make up the infrastructure.

Breaking the chicken and egg

I need to establish a connection to the database server from the app server so I will need credentials. So as you can imagine from my previous posts and in this post, we store the credentials in a credentials manager on the cloud provider not on the application server. ?

Why we do this is because we need to store connectivity credentials securely and we need to disable quickly if there is a problem. Also remember me talking earlier on about cycling passwords and credentials? Well, if the credentials are stored in the credential manager, we can set this up as an automated process, generating new credentials every 30 days for example. We can also utilize the services provided by the cloud provider to automatically update the credentials in the database service.

?But then how to access these secure credentials? Do I store a username and password on the application server to access the credential manager? Of course not! So we need to break the dependency circle.

But this has 2 knock on questions?

Firstly, updating credentials is an elevated operation so needs to be tightly controlled, updating the database credentials in line needs to be an elevated operation also. Secondly, retrieving those credentials needs to be thought out.

?This is where IAM comes in. For the first point, I define a policy which allows the automatic cycling of credentials and updating the database every 30 days. It can only do this task and nothing more. I then assign this policy to a serverless function and the serverless function then has the access to perform the task. No human need ever know the credentials. The serverless function is the object in this case.

For the second point IAM comes in again with subtlety. I define a policy where the reading of the keys from the credential manager at run time is allowed I also define that this can only be done via an API, not by a user accessing a console. Then I assign the IAM policy to the container image and when this image is ‘spun up’ to make an instance, it can make a call to the credential manager because the instance can inherit or the container controller can inherit the IAM. The cloud infrastructure framework container controller is given the IAM policy and therefore no credentials need ever be stored on the servers, I simply ask for the credentials, the credential manager checks for access rights and then returns the information but only to that specific object or objects. Super secure, centrally managed.

IAM is a huge topic in itself but I hope you can agree that abstracting the secure storage of credentials from the systems/objects that use them and also cycling the passwords regularly increases security. The cost is the initial setup but then after that, no system downtime even when you cycle your credentials.

Monitoring

While I will go into more detail on this in the next post, it is worthwhile saying that all cloud providers have sophisticated monitoring systems in place. For example, monitoring of CPU utilization as a whole down to individual containers that make up the application cluster, to memory and networking. These are sent to a central dashboard system so that you can monitor the health of the system and its component parts. In addition, you can get alerted on potential attacks, changes to configurations which may be symptomatic of an intrusion, error messages within your application, billing a cost alerts.?

Checking In

So now we have come this far, let’s check off items in our checklist from part 1.

You can see that with monitoring in place which I will elaborate in part 5, we have a complete checklist and approach for setting up a secure environment in the cloud with greater security that on premise in most cases. This is because the items in an on-premise datacenter we may be limited by cost constraints, under a cloud model they are pay-as-you-go.

?Until the next post where I fill focus on monitoring and the integration of DevOps and close this set of posts.?