Distributed Systems Design - my thoughts on building distributed and future ready systems

I have been doing some reading on distributed systems design lately. I came across some great articles explained in so much simple terms and some of them very sophisticated. There are tons of material available on WWW like articles, videos, pod casts, talk shows etc. on how systems can be highly scalable, reliable and fault tolerant and all other good features.

What I observed from various sources of knowledge is that, while the art of building distributed systems requires lot of thought and effort to make the systems future ready (to tackle at least some of the challenges). Designing and developing these distributed systems is perceived as some kind of complex area done by some elite group of wizards.

?As technology is changing every single day, hour and minute, what I truly believe will help developers, budding architects or architects who are trying to get a grip of designing complex systems or individual who are thinking to build these kind of systems, is to refer to 'design patterns or architectural patterns for building reliable, repeatable and efficient Distributed Systems.

Just like we have design patterns for writing Object oriented programs which has prevailed since ages, providing consistency while writing/designing programs for web applications, web services/Api's or may be even background services.

I know that there are design patterns and clearly documented use cases proven by industry experts. The biggest gap according to me right now is, usage of these design patterns is not wide spread and I have seen instances of building things from scratch among different projects leading to duplication, inconsistencies, difference in point of views and probably misused. In order to overcome the above problems, I believe thinking in terms of these design patterns as a first choice approach to leverage and re-use the components which are already tried and battle tested (not re-inventing the wheel) brings in reliability, consistency and maintainability. We can re-use the components that are already built by someone else, share components with someone else reducing inconsistencies and greatly improving dynamic coupling there by reducing the dependencies. There are so many other advantages as many of you may already know, when we have clear direction set and many examples provided.

If you are new to this area or trying to explore things, check out side-car design pattern, how can this be useful in your use case? Give it some thought!?

My recent obsession and love for containers has made develop some proof of concepts using Docker containers and Kubernetes while implementing these design patterns. Containers and Container orchestration has become an ubiquitous choice while designing these distribute systems. I plan to write about some design patterns in my next posts starting with side-car design pattern and some of its use cases.