Software Evolution: How to Keep Up with the Red Queen Using Microservices

In a constantly changing world, businesses must adapt to survive. As Lewis Carroll wrote in the book "Through the Looking-Glass, and What Alice Found There," -?"Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"?This quote perfectly encapsulates the Red Queen problem, which refers to businesses needing to evolve and change faster than their competition to remain relevant.

As companies rely more and more on software, the complexity of software grows, making it increasingly difficult to evolve and eventually paralyzing. One critical software architecture choice for businesses is modularization and standardization. Modularization helps to develop faster by breaking down complex systems into smaller, interchangeable, reusable components that can quickly and easily adapt to changing market conditions. In addition, each module can innovate independently and in parallel, reducing the time to market significantly. For example, modularizing of hardware components in the PC industry led to dramatic innovation, cost reduction, and reduced time to market. Breaking down business capabilities (domains) into microservices is the first step towards modularization in software. Like hardware modularization, software modularization leads to dramatic innovation, cost reduction, and reduced time, enabling the company to stay competitive.?

?

Conway's law states that software architecture mirrors the organizational structure. For example, if four teams design software, there will be four modules in the software. As the?mirroring hypothesis?is proven, aligning the modules in the complex software to the organization structure is imperative. How to modularize complex software is a topic in itself. In the subsequent article, I will discuss how to modularize complex software while balancing the overhead costs of modularization.?

Each module should abstract the complexity, technology, and inner details to the consumers of the modules. The abstraction can be a standardized interface such as REST API, Protobuf, etc. However, since the independent module is also subject to evolution, two aspects become essential: the plasticity and composability of the module. Composability refers to the ease with which changes can be made in a module without compromising its integration with the other modules and applications. Plasticity refers to the degree the module can deliver functionality it was not originally designed for. Architects should enforce these two aspects to ensure the software domains/modules can evolve to keep up with the Red Queen. For a detailed example of the benefits of abstractions, refer to an article I posted a few days ago on the?API Abstraction on Kafka.

Correctly designed modularization, with abstractions (API), composability, and plasticity, creates strategic optionality. Modularization gives strategic options for splitting, substituting, augmenting, and porting complex software.?

In summary, complex software is hard to evolve rapidly (Red Queen Problem), but evolvability can become your company's competitive edge with the correct software architecture.?

The article is based on my experience in large-large distributed platforms and microservices. The theoretical framework is inspired by books I've read, including "Platform Ecosystems" by Amrit Tiwana and "Design Rules - The Power of Modularity" by Carliss Baldwin and Kim Clark.

#innovation #microservices #organization #future #systemsthinking #systemdesign