A New Take on Microservices

In my previous post, I wrote about challenges about distributed architecture particularly in microservices and their evolving solutions. But sometimes we need to think from different perspective and the question to be asked is “Are we developing microservices all wrong?”

Let me start by recent quote from Kelsey Hightower, Google’s former engineer and Kubernetes advocate - “If people would just start with organized modular code, we can make the deployment architecture an implementation detail”.

Wait…wait...wait…this is not what microservices are all about. Isn’t it so?

What Went Wrong with Microservices?

A few months earlier, the engineering team at Amazon Prime Video posted a blog post explaining that, at least in the case of video monitoring, a monolithic architecture has produced superior performance than a?microservices and serverless-led approach.

In fact, Amazon?saved 90% in operational costs?by moving off a microservices architecture. For a generation of engineers and architects raised on the?superiority of microservices, the assertion is shocking indeed. “What makes this story unique is that Amazon was the original poster child for service-oriented architectures,” weighed in Ruby-on-Rails creator and Basecamp co-founder?David Heinemeier Hansson.

The task of Amazon engineers was to monitor the thousands of video streams that Prime delivered to customers. Originally this work was done by a set of distributed components orchestrated by?AWS Step Functions, a serverless orchestration service, AWS Lambda serverless service. Initially, the team tried to optimize individual components, but this did not bring about significant improvements. So,?the team moved all the components into a single process, hosting them on Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Elastic Container Service (Amazon ECS).

“Microservices and serverless components are tools that do work at high scale, but whether to use them over monolith has to be made on a case-by-case basis,” the Amazon team concluded.

New Take on Microservices

This was the main thesis of “Towards Modern Development of Cloud Applications” (PDF), a paper from a team of Googlers presented in June at?HOTOS ’23: Proceedings of the 19th Workshop on Hot Topics in Operating Systems.

The problem they pointed out, was that?microservices?largely have not been set up correctly, architecturally speaking. They conflate logical boundaries (how code is written) with physical boundaries (how code is deployed). And this is where the issues start.

Instead, the Google engineers suggested another approach. Build?the applications?as logical monoliths but hand them off to automated runtimes, which makes decisions on where to run workloads, based on what is needed by the applications and what is available. With this latency, they were able to lower latency systems by 15x and cost by up to 9x.

?In their paper, the Google engineers list a number of shortcomings with the microservices approach, including:

·???? Performance:??serializing and sending data across the network to remote services hurts performance, and, if the application becomes complicated enough, could even lead to bottlenecks.

·???? Comprehension: Bugs are notoriously difficult to track down in distributed systems, given the many interactions across microservices.

·???? Management issues: It is considered an advantage that different parts of an?application?can be updated on their own schedules. But this leads to developers having to manage a huge number of binaries each with their own release schedule. And good luck running end-to-end test with a locally-run service.

·???? APIs get brittle: The key to microservice interoperability is that once a microservice is established the APIs can not change, let them break any other microservice that relies on the API. So APIs can only be extended with more APIs, creating bloat.

Analysis

Interestingly, the architecture the Amazon video team used was not exactly a monolithic architecture either.

According to?Adrian Cockcroft, the former vice president of cloud architecture strategy at AWS, “This definitely isn’t a microservices-to-monolith story. It’s a Step Functions-to-microservices story”. He pointed out that in many applications, especially internal applications, the cost of development exceeds the runtime costs. In these cases, Step Functions make a lot of sense to save dev time, but can cost for heavy workloads.

Solution

The Google paper tackles this issue by making lives easier for the developer while letting the runtime infrastructure bets figure out the most cost-effective way to run these applications.

“By delegating all execution responsibilities to the runtime, our solution is able to provide the same benefits as microservices but with much higher performance and reduced costs,” the Google researchers wrote.

Their prototype implementation is available here with the Service Weaver docs.

Spring Modulith (short for modular monolith) is another option to build well-structured Spring Boot applications divided in?application modules?per domain.

Conclusion

In absence of a microservices chassis platform, starting with a modular monolith is recommended so that it becomes easier to refactor later and deploy the modules as microservices independently.

We may soon see new category emerging in ever growing list of CNCF landscape that breaks these modular monoliths and deploy them over in microservice style.

Hopefully that will reduce the friction and load on developer community.

Wish you all good luck in 2024!