How to get a good design and well-architected software. Our experience with DDD paradigm

One of the most relevant issues to be considered in software development, under agile principles, is the architecture and design. This is essential in order to attain good software. Six principles of twelve from the agile manifesto for software development, direct, or indirectly, are related to this subject. Synthetically, they talk about simplicity, continuous attention to technical excellence and good design, to be able to deliver valuable software frequently, and continuously, maintaining a constant pace over time, and one more refers to architectures, requirements, and designs that emerge from self-organizing teams.

 Following this kind of principles, the best software companies have set the bar quite high to others, because they are able to deploy new features for their software products at a daily pace, also, these systems scale optimally on huge ecosystems, with more than a billion of users in some cases, or even, like Amazon ecosystem, which besides over 300 millions of customers, has more than a million of companies integrated on their internet sales supply chain platform, and growing up. Behind this agility, scalability, and quality of software, there are some common traits, like freedom, “simple mindset”, or small multidisciplinary teams, self-organized teams, with enough skills to define, design, implement and deploy with total autonomy. One of the highlighted learned lessons from Amazon is maximum “2 pizza teams” (8 people). Spotify refers to its famous tribes as teams maximum 8 to 10 people. Teams defined focusing on responsiveness, and organized around business/customer outcomes, over teams focused on and organized around specialized tasks.

 Organizations structured around small teams, and with this capability, only it’s possible with good engineers, engineers with the ability to create really well-architected software and good designs. It’s Important in this context talk about to craft and not to do. If you want to get good software, the first big rule you should accept is that software it’s something creative, and not process intensive. Good design/architecture is closely related to creativity, and creativity it’s difficult to appear on heavy processes and documentation oriented software development models.

On this post I will try to expose, from our experience, five main drivers we focus on to get our goals for quality, most simple and best software design/architecture. These drivers emanate mainly from the paradigm we use on our way: “Domain Driven Design” (DDD). This is a paradigm that helps high performance teams to tackle complexity. It’s a model suitable when business needs to solve are complex and change often over time. DDD works only under agile principles, agile teams and agile models. We tried to use DDD with traditional development organizations, and we didn’t get it to work.

Driver 1: Good design, Simple design to get an agile system easy to maintain

When business is complex, as it’s our case, the challenge to get a good design it isn't a technical Issue, at least, it isn’t the key issue. Under these circumstances, more or less, 80% of the software development effort and complexity are on business logic.

 All the development models have in common that a good software design must be based on, and close related with business problem you are trying to solve. This seems something obvious, but it’s not. Some projects fail due to not consider properly and not put the focus on it. Different software development models have appeared over time. All of them have one of two possible approaches for design:

• Traditional Methods. These were the first one to appear. Include techniques like Data Flows Diagrams (DFD), Structure charts, and so on. In this kind of approaches we chose represent and make knowledge explicit based on business processes, their relationships and the information flowing through them. Afterwards, using techniques like “structure chart” we attain our application design as a reflection of the business processes structure. This has been commonly used with procedural languages like cobol.

• Object Oriented Methods. In this case we represent business reality focused on business concepts and their relationships. There are multiples variants here (RUP, use case driven projects,…). These are common approaches used with object oriented languages like java.

Here we have an important selection criterion to get maintainable systems. Business processes change often, however business concepts hardly ever change, or don’t change over time. Therefore, systems  which design is based on business concepts, don’t require structural changes and are easier and less expensive to maintain than systems which design is base on business processes that change often. The problem is that make explicit business from concepts is much more complex that make explicit business from processes, maybe, this is a reason why many object oriented projects obviate this fundamental step, resulting then on difficult to maintain systems.

 DDD is an object oriented approach that’s go one step further. With DDD explicit business isn’t close to software design, but it is the best design you can get.  DDD is an object oriented approach specifically designed to tackle software developments that solves complex business problems (complex domains). The main different from others object oriented approaches is that DDD focuses first on the problem without consider the solution. It focus first on the “What” and the “Why”, not the “How”. Under this premise, if you are able to make explicit, correctly, the business concepts, their relationships, and business rules properly embedded on these, then you already have the right input for the best design you can implement for your software. I’m not talking about heavy documents, I talk about good plans. A “good image/graphic says more than 1.000 words” a popular quation than in DDD is a must. DDD also propose specific design patterns to move from the business problem space to the solution space. Make this transition simple is also a creative work.

 For DDD is really important the “Shared Knowledge”, and so you must establish as a key rule a “Ubiquitous Language” that must be precise and unambiguous, a language that makes concepts and rules explicit in the same way. A common language used for all, business experts and engineers through all the development process, on any interaction between team members, and in all the deliverables, including source code. This is especially important, our source code also speaks business language, the same language our business experts and customers speak.

Driver 2: Teams configurations and skills required are important

Looking for productivity in software development, one of the most common trends has been tackling complexity through big teams, working under heavy processes, oriented to specialization, and with heavy management and high control. The more complex, the bigger are the teams. Belief under this paradigm is doing one task by one team member very well can be more productive than team members doing several tasks. Under this model you can find projects with analysts than only do requirements and analysis. Or designers that only do system design, architects architecture, programmers programming, testers test, and so on.

 However, DDD consider software like something highly creative. More complex the business is, more difficult is to attain, and productivity decrease significantly when you try to industrialize the development process on this situation.  More functions executed by different people you split on the chain, more time spent transferring knowledge and more difficult is keep up to date all the teams, and all the deliverables. When the system and complexity grows there is a moment in time where unsustainability appears, and starts a spiral of delays that tends to grow as you incorporate more people to the team trying to recover lost time.

 Nowadays, my team has 4 times less people than when I started. Every reduction of people we made has contributed to improve our productivity, and the resulting software we get has improved significantly. It sounds weird, but it’s real. Also we broke specialization. Every engineer is responsible to analyze close to business experts, design, implement, test and deploy quality code, frequently on an iterative way. 

Driver 3: Software Product Architecture evolves and is a consequence of business knowledge

Business must guide technical decisions and not vice versa. This can be a trivial issue but not. It’s very common follow market fashion trends, choosing technical frameworks or frequently used software pieces that, after a period of time make us captives, setting us a software design that is not what we need, or what we want, and increasing significantly development times for any new requirement. To choose a framework, a library, or a third party software product for your system without know anything about the business problem to solve it will be a mistake. A system that is not tightly linked to our users/customers domain will be less efficient, less relevant, and far much complicated to stick to every new needs of our users. Architecture evolves and improves as business is known by the engineers through the project, best architectures and design don’t appear on the inception stage. High cohesion and low coupling are principles to prioritize to get a good architecture and design for business logic.

 From a technical point of view, DDD pose a four layer architecture with business logic on one layer  decopled  from technical details. Another key rule for the team it’s that every line of code is also design, and every line of code less is also design. Clean code, and not a single line of code should have that it’s not been used on the system. Reuse yes, but knowing what copy/paste you are doing.

Driver 4: Test Driven Development to get quality of software

On this driver there aren’t specific DDD issues to apply, the agile ones are enough:

• Business cases real defined with accuracy, and end to end scenarios (User stories and acceptance criteria) are the main challenge for our business experts, besides knowledge transfer.
• TDD (Test Driven Development) and BDD (Behaviour Driven Development) are good approaches to get quality of software. In our experience, more or less, the time spent on development it’s 50% to implement unit tests, and 50% to write source code  lines
• A good battery of automated tests it’s what allows us the evolution of the system keeping good quality, even when the team tackle structural changes, or core refactorings.

Driver 5: Quickwins and frequent deliveries of working software as measure of progress and to get feedback

It’s not necessary to explain very much about this driver. It’s another key principle for agile teams. 3 aspects to highlights:

• Short cycles and visible results continuously help us to detect deviations, or improvements chances avoiding high impact failures.
• This iterative model with frequent deploys allow us measure with working software, and not through milestones and deliverables on a plan. Focus on goals and not on tasks. Goals only related to working software, never with any other kind of deliverable. Continuous Integration and Continuous Delivery techniques and tools, complement our model to get shorter cicles.
• In the case of DDD we can stand out that iterative process, enabling end to end results, using shorts cycles helps for a better shared knowledge, and to get more creative results. In DDD slang we begin witth “Knowledge crunching” in an iterative way, with laboratories between business and technical people, brainstorming sessions, using sketches or similar techniques, and implementing prototypes when needed.  The results are quite creative and simple designs.