Automation in Scalable WordPress using Docker, CoreOS, etcd, SkyDNS, and confd

DevOps has come a long way in the short time span it has been around. The new ways in which we can now automate how software is delivered, and management of the infrastructure used to do so, are amazing. For Elastisys Scalable WordPress, we use Docker, CoreOS, etcd, SkyDNS, and confd as automation building blocks. In this post, I will discuss how they all fit together to create what I think is the easiest to use high-scalability deployment of WordPress ever. (My bias in having that opinion is obvious, as I am the principal software architect for that project -- request a demo and I'm sure you will agree, though!)

Elastisys Scalable WordPress includes load balancing, SSL termination, a networked file system, a clustered database, HTTP acceleration caching, and database offloading via caching. The Elastisys Cloud Platform ensures that there is just the right number of each component deployed by using our patented predictive auto-scaling engine. Performance is optimized for WordPress sites of all sizes, and administration is as simple as for a single-server deployment.

I recently posted about the software architecture in Scalable WordPress. This post follows up where the other left off: this time we dive in to the automation features that make the deployment dynamically resizable.

Scalable WordPress architecture overview

As a refresher before we go, here is the architecture of Scalable WordPress:

Read the previous post on Scalable WordPress' architecture if you want a more detailed description of each of these components. If you don't, that's fine -- here is a short recap, just for you.

When a visitor accesses the site, their request is first met by a load balancer. Depending on whether the visitor wishes to use HTTP or HTTPS, the traffic is either routed directly to a caching server (HTTP), or to a server that performs SSL termination (HTTPS). In the latter case, SSL encryption will be added back again before the response is ultimately sent back to the visitor.  The caching server offloads the rest of the deployed system by checking if it already has stored a suitable response to the requested page or file. If so, it just responds, and the components in the lower part of the diagram are spared from processing the request at all. This is great for performance. If there is no suitable response stored in cache, WordPress must generate a new one. By using the W3 Total Cache plugin and having Elastisys Scalable WordPress providing it with a replicated database, a replicated networked file system, and a high-performance in-memory object store, this process is accelerated in numerous ways.

The takeaway message regarding the architecture is simply that we somehow have managed to deploy these pieces of software in an isolated and replicated fashion and then hooked them all up to handle dynamically growing and shrinking the deployment. It is that mysterious somehow that is the topic of this post. Read on!

Automation in Elastisys Scalable WordPress

Software packaging and delivery via Docker

Docker makes it easy to package Linux software in self-contained images, where all software dependencies are bundled and deployed in a repeatable manner. An image will have exactly the same software installed when I run it on my laptop, on my server, now and forever. It is immutable. This is great, because fewer changes in dependencies makes software easier to reason about, and reduces the difference between what a developer is working on and what is actually deployed in production (cf. dev/prod parity).

Because images are immutable, there is a much clearer separation between deployed software and the data upon which it acts. Software can be updated by shutting down an old version and starting up a newer, but the data is kept around between updates.

From a user's point of view, the use of Docker means you can be certain that the software running your Scalable WordPress is exactly what we at Elastisys packaged, and use ourselves. Additionally, updates are safe and predictable, due to clear separation between software and data.

Cluster-wide deployment using CoreOS

CoreOS is a Linux-based OS tailored for deploying Docker containers on a cluster, rather than just a single host. It comes with clever tools to achieve this, such as fleet and flannel. Fleet bills itself as a distributed init-system: a mechanism for running systemd units across a cluster instead of a single machine. It includes a rudimentary resource scheduler, which supports basic primitives such as co-location of related units. Flannel creates an overlay network and attaches all Docker containers to it, giving them cluster-wide IP addresses. Used correctly, these tools make deploying software on a cluster as simple as on a single machine.

Elastisys Scalable WordPress benefits from CoreOS and its supporting software in that it has made it easy to deploy Docker containers across a dynamically sized cluster of machines. Our use of predictive auto-scaling ensures that the size of the cluster is always just right.

Configuration stored in etcd

etcd is a distributed consistent key-value store, usable for e.g. distributing configuration values across a cluster of machines. It comes with CoreOS, but can be deployed on other systems, too.

An etcd cluster consists of members and proxies. Informally, members differ from proxies in that they have a "vote" on what values the cluster agrees on. For technical reasons, etcd needs at least a certain number of members to ensure consistency. Elastisys Scalable WordPress, however, makes use of auto-scaling, which lets servers come and go over time. To avoid confusing the etcd cluster, all auto-scaled servers are proxies and rely on a group of ever-present members for configuration values.

Clever tools that interface with etcd have been built, and we use two of them: SkyDNS and confd.

Service discovery via SkyDNS

As shown in the architecture overview, Elastisys Scalable WordPress consists of many different components. Serving your content to visitors is team work between components. Team work requires at least two things: knowing who your team mates are, and working toward a common goal. Service discovery is the process that lets components figure out who their team mates are.

In Elastisys Scalable WordPress we use two related pieces of software for service discovery: registrator and SkyDNS. Registrator interfaces directly with the Docker system on each host, determines their network connectivity information, and pushes the information into etcd.

SkyDNS is a domain name system (DNS) server, which translates domain names such as wordpress.elastisys.local into an IP address. It integrates with etcd, and can respond to DNS queries using information stored therein. Registrator is configured to push its information in a format recognized by SkyDNS which effectively means that service discovery happens as follows:

1. Fleet is instructed to start a unit (script that starts a Docker container), determines which machine in the cluster will host it, and instructs the machine to do so.
2. A Docker container is started on the machine. It is given a cluster-local IP address by flannel.
3. Registrator determines that the Docker container is running, inspects it, and pushes information about its IP address and ports into etcd.
4. If any component makes a DNS query relating to the component's domain name, SkyDNS checks etcd and responds with information regarding the newly started component.

Neat, right? When the component is shut down it will eventually disappear, because registrator has been configured to let its entries expire after a certain amount of time. With them gone, SkyDNS will no longer respond with them to queries.

Dynamic reconfiguration via confd

Alright. Components can find each other via etcd and SkyDNS. Now, they just need a common goal. That is where confd comes in. It is a conceptually simple tool: read from etcd, generate a configuration file from a template, and then see if the result differs from the current configuration file. If it does, run a configurable validation tool (if available), place the validated configuration file where it can be activated, and instruct the component to reconfigure itself.

Because registrator puts all network connectivity information into etcd, it is possible to make a template for e.g. the load balancer's configuration file. confd just needs to be configured to look where SkyDNS is supposed to look for SSL terminators and caching component instances and render the template appropriately.

Closing remarks

Scalable WordPress is constantly evolving to better serve its purpose of providing the best high-performance platform for your content. Automation in terms of software delivery, data management, and service discovery/configuration makes even a complex setup easy to work, administer, and publish with.

Your content is what matters and makes your site unique. Your hosting solution should make that content accessible to as many visitors as possible. If your core business is not in optimizing WordPress hosting, let somebody else do it instead. I hope that we can earn that trust, because I know Elastisys Scalable WordPress is up to the task. Contact us today and see for yourself.