How to deal with dependencies in large scale? part 2: clear intent

In one of my previous blog on "dependency management", I have talked about the reason behind why dependency exist and organisational challenges. In this article, let's dig a little bit deeper to the dependency management through improving understanding of architectural patterns.

Rule No.1: there's no technological silver bullet that solves dependencies if we don't understand the problem domain

Traditionally IT is considered as utility provider, a black hole cost centre rather business enabler and partner, overtime, things just get more and more complex and technical debt become waves of boomerangs?that keeps flying back and hit us, people come and go, project starts and finishes, over years, as a result, firefighting becomes business as usual, dependencies become harder to manage, sometimes it feels impossible.

Figure 1: Integration reality (illustrative example)

In 2003, Eric Evans has wrote an "epic" book called "Domain Driven Design - Tackling Complexity in the Heart of Software" (aka DDD)

"Every software program relates to some activity or interest of its user. The heart of software is its ability to solve domain-related problems for its user. All other features, vital though they may be, support this basic purpose. When the domain is complex, this is a difficult task, calling for the concentrated effort of talented and skilled people. Developers have to steep themselves in the domain to build up knowledge of the business."

Figure 2: Book cover of "Domain Driven Design"

A domain is the logical area that defines the problem you want to solve, such as customer services, sales or core banking application.

Bounded Contexts: the logical boundary around the code that represents the solution for that domain. The bounded context is the solution of the problem described by the domain.

Entities: domain objects that are uniquely defined by a unique identifier, and not by their attributes (e.g. User, Customer, Message, Notification)

Value Objects: an unchangeable object that has attributes, but no distinct identity. (Name, Address)

Aggregates: it is hard to keep the consistency of changes to objects in a model with complex associations, we cluster the entities and value objects into aggregates and define boundaries around each.

Factory: A mechanism to encapsulate and abstract away the details of creating a complex object. A factory ensures aggregates are initialized to a consistent state.

Repository: the repository pattern is a?collection of business entities?that simplifies the data infrastructure. It releases the domain model from infrastructure concerns.

Service: A stateless functionality that renders its service via an interface, typically used when a workflow doesn't fit the current model.

The?layering?concept enforces the separation of concerns.

Figure 3: Layers in a Domain-Driven Design Microservices

Domain Model Layer: Responsible for representing concepts of the business, information about the business situation, and business rules. This layer is the heart of business

Application Layer:?Defines the jobs the software is supposed to do and directs the expressive domain objects to work out problems. The tasks this layer is responsible for are meaningful to the business or necessary for interaction with the application layers of other systems. It does not contain business rules or knowledge, but only coordinates tasks and delegates work to collaborations of domain objects in the next layer down

Infrastructure layer: how data that is initially held in domain entities (in memory) is persisted in databases or another persistent store.

Figure 4: Dependencies between Layers in a DDD

Dependencies in a DDD Service, the Application layer depends on Domain and Infrastructure, and Infrastructure depends on Domain, but Domain doesn't depend on any layer. In accordance with the "Persistence Ignorance" and "Infrastructure Ignorance" principles, the infrastructure layer must not "contaminate" the domain model layer.

DDD is easy to understand, the book was not too easy to read, but the thoughts in this book are considered as foundation in terms of technical dependency management and architectural design. In real world, tech choices are made without clear rational reasoning, often formulated based on hypothesis that is not tested. So understanding the program domain is really critical before we pursuit any technical solutions that will hunt us in long term if not careful.

Figure 5: DDD?is in the Late Majority stage of architecture and design adoption curve

Besides, technical solutions?are circled around the business model?by connecting execution to the key business principles, for which we must understand product architecture, and the fundamental business logic.

Rule No.2: applying architecture pattern according to its context, going back to the basics, devil are in the details

Architecture design is a very complex activity. In addition to reflecting on the modular structure, architects need to consider required technologies and the dependencies, available resources, customer challenges, pressure from the market. Particularly if there are concerns spanning multiple layers or various components, the modularity of a software itself is not straightforward either.?Here're the 14 common architectural patterns according to redhat.

The?layered?pattern is commonly used in e-commerce that include groups of subtasks that execute in a specific order. The layered pattern makes it easy to write applications quickly, but a disadvantage is that it can be hard to split up the layers later

The?circuit breaker?pattern minimizes the effects of a hazard by rerouting traffic to another service. While it helps make systems more fault tolerant to prevent accidents, it also requires sophisticated testing

The?client-server?pattern is a peer-to-peer architecture that is comprised of a?client, which requests a service, and a?server, which provides the the service. Examples include banking, file sharing, email

The?command query responsibility segregation?(CQRS) pattern handles the situation where database queries happen more often than the data changes. It separates read and write activities to provide greater stability, scalability, and performance, but it requires more database technologies and therefore may increase costs.

The?controller-responder?pattern divides the architecture into two components: The controller handles the data and distributes workloads, and the responder replicates data from the controller and generates results. One advantage is that you can read data from the responder without affecting the data in the controller, but if the controller fails, you may lose data and need to restart the application.

The?event sourcing?pattern is good for applications that use real-time data. It sends a continuous stream of messages to a database, web server, log, or another target. It's very flexible but demands a highly efficient and reliable network infrastructure to minimize latency.

The?microservices?pattern combines design patterns to create multiple services that work interdependently to create a larger application. Because each application is small, it's easier to update them when needed, but the complexity means you need greater architectural expertise to make everything work correctly.

The?model-view-controller?(MVC) pattern divides an application into three components. The?model?contains the application's data and main functionality; the?view?displays data and interacts with the user; and the?controller?handles user input and acts as the mediator between the model and the view. This pattern enables the application to generate various views, but its layers of abstraction increase complexity.

The?pub-sub?pattern sends (publishes) relevant messages to places that have?subscribed?to a topic. It's easy to configure but more challenging to test because interactions between the publisher and the subscriber are asynchoronous.

The?saga?pattern is used for transactions with multiple steps, such as travel reservation services. A "saga" includes the various steps that must happen for the transaction to complete. This pattern enables transactions (ideally with five or fewer steps) to happen in loosely coupled, message-driven environments, but it requires a lot of programming and can be complex to manage.

The?sharding?pattern segments data in a database to speed commands or queries. It ensures storage is consumed equally across instances but demands a skilled and experienced database administrator to manage sharding effectively.

The?static content hosting?pattern is used to optimize webpage loading time. It stores static content (information that doesn't change often, like an author's bio or an MP3 file) separately from dynamic content (like stock prices). It's very efficient for delivering content and media that doesn't change often, but downsides include data consistency and higher storage costs.

The?strangler?pattern is used when you're making incremental changes to a system. It places the old system behind an intermediary to support incremental transformation, which reduces risk compared to making larger changes. However, you need to pay close attention to routing and network management and make sure you have a rollback plan in place in case things go wrong.

The?throttling?(or rate-limiting) pattern controls how fast data flows into a target. It's often used to prevent failure during a distributed denial of service attack or to manage cloud infrastructure costs. To use this pattern successfully, you need good redundancy mechanisms in place, and it's often used alongside the circuit breaker pattern to maintain service performance.

Improving architectural design is vital to continued success within any industry as companies move to large-scale digital transformation, reduce dependencies, eventually quality of execution is depending on people, so empower your employees to grow with your business, drive continuous learning culture and stick to the basics.

Design patterns give engineers a reusable way to solve these problems, allowing software engineers to achieve smiliar output structurally for a given problem domain, the architectural pattern captures the design structures of various systems and elements of software so that they can be reused, to reduce complexity and improve cost efficiency at build.

Here is the link to previous article in this serie:

How to deal with dependencies in large scale agile? part 1 | LinkedIn

Source: dev.to/microtica, learn.microsoft.com, makingsoawork.blogspot.com, InfoQ, redhat