Microservices as a Metaphor for the Evolution of Work

I often find metaphors for human work patterns in software architecture. Bear with me for a moment...

You are probably knee-deep in re-architecting some monolithic legacy system into a collection of microservices. If you're slightly ahead of the curve, you've already done this, possibly multiple times. If you're at the leading edge of the curve, your thinking extends beyond microservices to serverless architecture and transactional compute frameworks. We've seen a movement towards a separation of concerns and an encapsulation of logic at an increasingly granular level, and this trend has driven innovation of open source and vendor-specific solutions to support transactional granularity with ephemeral state. As a result, or perhaps as a prerequisite, the way we design systems has changed.

The motivations behind microservices and serverless architecture are grounded both in design philosophy and economics. From a design standpoint, microservices embody the ultimate separation of concerns, isolating functional areas from one another and allowing you to assemble complex functionality from a collection of well-defined services that can be developed, deployed, invoked, and managed independently. It is essentially the same philosophy that drove information hiding, encapsulation, and object-oriented design in decades past, but taken to its logical extreme. Beyond maintainability, quality (from reduced complexity), and fault tolerance (one failing service doesn't bring down the entire monolith), microservices lend themselves to horizontal scaling and a more efficient response to load that is uneven across the application as a whole--you scale each microservice depending on its load profile. Serverless architecture in particular provides a very cost-effective way to manage resources, since the app developer is not responsible for the infrastructure that invokes the serverless functions, and generally speaking only pays for compute resources while they are being used.

These architectural and economic factors change the way you think about system design. In such a highly distributed world, you are more likely to make use of services developed by third parties, as you can now consider build vs. buy at the function level rather than the application level. You will make technology choices that allow you to spin up and wind down functions with low latency, which probably means minimal overhead of heavy infrastructure or resources. You will move session state out of these functions, necessarily, and think of data persistence as a separable service rather than an asset of the application. You might think of this as a natural outgrowth of a more general cloud strategy, building cloud-native applications in a way that leverages the infrastructure and services of cloud providers; certainly the economic considerations of a serverless approach are most germane to the public cloud ecosystem. Regardless of the primary driver for this shift, a more distributed, granular computational capability shapes how we think about system design and influences the way the industry is organized.

This trend can serve as a metaphor for the evolution of labor markets and how people work.

The decline in median job tenure is sometimes debated, but pretty obvious when you look beyond the last couple decades and correct for Simpson's Paradox. It's difficult to separate the cause from the effect on this trend--are millennials holding jobs for shorter periods of time because they are intrinsically more mobile than their predecessors, or has the nature of work changed such that the conditions under which long-term job commitments occur are less common? This is like asking whether the trend towards microservices and serverless architecture is being driven by technological change that enables these approaches, or is the technology being driven by the evolution of distributed system architecture?

The metaphor extends beyond job tenure. The Gig Economy parallels the trend of self-contained, ephemeral functions that are spun up and wound down based on demand. Job tenure and company affiliation is shrinking, and this has been enabled by mobility, digitization, and an infrastructure that supports this type of work (e.g. nonemployer firms), in much the same way serverless infrastructure allows app developers to build loosely coupled ephemeral functions. Gig opportunities afford a segment of the workforce to maintain employment while limiting their commitment to any one employer. This class of jobs must have low overhead in their startup and tear down--providing a ride, a room, or a well-defined service. So not only does this affordance create a new class of jobs, it changes the shape of the organizations who participate in this ecosystem.

One thing seems clear: technology is changing the type of jobs that will be in abundance. Robots in factories, driverless cars and trucks on the road, AI-driven automation in many domains--all these technological advances will impact the type of work humans do. It is creating a disparity in the job market that has become manifest in the current political dialog. A foundational element of our free-market society is that companies compete with each other to provide goods and services in a way that maximizes value to the consumer while minimizing cost to the company. They are compelled, through the natural forces of the market, to take advantage of technology when it reduces costs or increases value. Automation changes the type of work best done by humans, but humans can't adapt to new types of jobs as fast as automation is applied. There will always be legacy monolithic systems that defy the trend towards more granular computational entities, and there will always be social, economic, and political forces that resist the trends of automation and the gig economy. In that resistance lies tension, and we will struggle to find a balance for years to come.