Dependency Hell in Microservices and How to Avoid It

In my previous post I talked about independence being THE defining characteristic for a microservice. It is also the biggest worry of new adopters of the microservices architecture paradigm. Independence can be simple to provide (design is always important). Or it can be a nightmare of a dependency graph. By dependency I mean both development and deployment dependency.

If you google "microservices dependency hell" you will find some pretty entertaining reading in thousands of articles. The naysayers are very quick to point out that only trivial microservices can be truly independent. After all, when we design a relational database, we spend quite a bit of time analyzing "functional dependencies", and removing them in steps, called "normalization". We want to have at least 3NF (Third Normal Form), before we can use the resulting schema. Read up on normalizing a relational schema to refresh your mind about normalization and why it is necessary. The normalization process removes "update anomalies". These can be disastrous in a database leading to loss of data, or mysteriously erroneous duplicate data values. All the arguments of normalization to remove functional dependencies apply to microservices. Relevant, but a side topic. The normalization guidelines for microservices can be much simpler.

I don't want to make this conversation lengthy, or overly theoretical. There are many excellent articles about how to decompose a monolithic service into a set of cooperating microservices. In particular, I like everything that Chris Richardson has to say about everything microservices. Clear, and practical. Check out his website on microservices. He has an excellent article about decomposing applications into a set of microservices.  

Here, I want to only go through the "elevator pitch" version. 

I like to think of using microservices, and compositions of microservices, in a manner similar to writing a Unix shell script. Each Unix command is a single function that can be combined with other functions via pipes and filters, and can be called from a script that provides a higher granularity function. The Unix analogy points out that we can write many more shell script than Unix commands (which are crafted much more carefully). Fewer microservices, many more ways to compose them via "microservices scripts".

If you keep that in mind, you can develop microservices that are independent single functions, and not slide into dependency hell. Just observe two simple rules:

- A microservice cannot call another microservice directly. 
- A microservice can be invoked in only two ways

• Through an event 
• Called from a "microservice script"

 A "script" can be any software component that implements the script pattern. A script can be implemented by a composite service (higher granularity than a microservice) - SAP follows this architectural pattern by classifying services into core services and composite services. It could be a UI application, a service object, or an api gateway that manages cooperation between a set of microservices to deliver a higher order business function. Consciously avoid creating microservice dependency graphs. Follow good rules of Structured Design (from the 1979 classic by Yourdon and Constantine). That's where the terms "coupling" and "cohesion" came from.  You don't want a web page to make 150 microservice calls (Amazon actually tolerates 100-150 microservices per page). Restrict fan-out to about 10, and by, introducing no mare than two levels of composite services,  you can reach a 100 microservices per script, and still live with the rule that no microservice can call another. That's a bit simplistic, but follow design guidelines that carefully partition a large service into microservices.

Transitive dependencies are not possible. Dependency trees are not possible. An ordering of microservice calls will result from the pattern of event production and event consumption. There are a lot of details, but without adhering to the above two rules, dependency hell can easily result, and you are back into the dark days of failed SOA. You will quickly discard microservices and declare they are just a new fad, and merely hype, and "not for us".