A reflection on software engineering

In my view, a software engineer has to be a designer first, at least for architects or senior people; this stuff is very complicated intellectual activity, requires very unique mindset – thinking very abstractly and systematically, very much like a writer or artist; basically they would present their abstracted view of what they see and experience in the real world, and emphasize/de-emphasize/express what they think the most important things among many many other things using their choosing medium; in our case it is a computer programming language; in the art world, it might be a film, a painting, or a published book. It is all about presenting something. There is something quite unique though, the software is not a one-time thing, it has to be able to evolve and adapt; there is no end, will never finish. From another perspective, this is also a good thing – we always have a chance to correct what is not right; problems are just the price of progress. 

1. What is software engineering?

Fundamentally software engineering is a modeling process - domain modeling process. The final product, the executable code, is just a representation of that modeling process. Yes, programming language and library, and technology are important, but if we look at the whole process, we can clearly see the whole thing starts with the domain modeling at the very beginning; so programming language and library, and technology are still secondarily things coming later at least for complicated business domain, they are just tools/building blocks to implement the master plan - the domain model. Just imagine if we got the modeling wrong in the first place, how would programming language and library, and technology help us? We were wrong at the very beginning! 

Based on the reasoning above, the software engineering is really about the thinking process, it is a way to emulate the domain world in which things would be done in the most efficient ways - that is just the human nature; less is more, the most cost-efficient way would be perused by us. Business domain already presents a master design for us, we just need to present it in a way so that it would semantically match it; it is also a way to simplify the real domain world since it would totally overwhelm us if we try to literally emulate it; so we would employ the most important technique – abstraction, a process to abstract the domain world. So the conclusion here is that the most creative thing for software engineering is the thinking process, the design, the domain design, and particularly thinking abstractly. The coding is the least creative thing in this context since it is just following the master plan/design. 

2. What are the goals of software engineering?

The ease with which we can modify a system, and adapt it to the rapidly changing business landscape, is closely linked to its simplicity and its abstractions: simple and easy-to-understand system is usually easier to modify than complex ones; it is also relatively stable because it has a stable structure; if a software structure has to be changed, we cannot say “usually easier to modify”. 

Fundamentally software engineering is also about the separation of concerns since we can only handle small things(single concern/responsibility) most effectively because of the limited size of our brain. All of these boils down to the idea of small, single-purpose things that are easier to understand and implement. These small things have to come from the design of domain modeling – the domain boundaries, bounded context. It all goes back to domain modeling as we can see, the root of everything. Eventually and mostly, we do not write code for simply writing code, the code is replacing the business process which we would otherwise have to execute manually. 

3. Some principles of engineering.

Mostly software engineering is about maintainability(which implies extensibility, adaptability, and scalability as well), because of this, more efforts should be devoted to maintainability than delivering the new features. Ignoring maintainability is like accumulating debts. Small cumulative refactoring has a big impact in the long term. Refactoring does depend on you being able to write tests that baseline current behavior to ensure that changes can be made safely. So we need code testability (domain logic self-containment, separation of functional vs non-functional), which can only come from the good domain modeling. Stopping refactoring because no one knows what might be broken is very troubling; it is questioning the quality of tests, maintainability of the system. 

4. Anything new on the horizon from the engineering methodology perspective?

The software industry is just about 70 years old, still relatively young compared to other industries. The software engineers struggled from the very beginning trying to figure out ways to perfect the design methodologies after we had high-level languages beyond the assembly language. So we had structured design(the late ‘50s), then object-oriented(the ‘60s), functional programming(invented first in the '40s). There were no new methodologies after that except the domain-driven design introduced by Eric Evans in his blue book in 2004. 

To some extent, microservice is just the application of separation of concerns at the much larger context/scale – enterprise level, not at class or module level. It is the second layer design after domain modeling, it is more about implementation decision of domain modeling, or maybe about the delivery process - the size of the final product, a choice of large code product or small code product, which is nimble, small, and independent.

In my view, we have to pick up the old engineering textbook, studying those methodologies, which are almost 40 years old, to raise the abstraction level by doing domain modeling.  

5.  Imbalance of design vs implementation, low quality of design==low quality of implementation

Lots of software architects in the IT fields, in my view, devoted too much time to technologies, serving as the go-to person for solving tough tech implementation issues developers encounter. Design is happening at all levels, did architects participate in design sessions, producing design documents, domain modeling, code review, and write some code to demonstrate their visions for the future? In my view, writing code is a very important role of software/application architect, they have to eat their own dog food.

The industry has all kinds of processes to guard the delivery/implementation. Do we have any process to guard the design, the mother of everything? Do we emphasize what we try to achieve in terms of architecture/engineering? I think the mentality of everyone-knows-this is leading us not explicitly expressing the design goal. 

6. The core business competitiveness.

The latest technologies are all available to all competitors. So generally speaking, no one would gain much advantage in that front; the only thing that would eventually distinguish them apart is how well the core domain modeling is serving the business, providing them a stable/flexible structure for them to easily/quickly adapt to the new evolving demands, at the same time, without fundamentally changing the existing software structure; it is the most effective way to reduce the cost of business.  

7. Training on technologies vs engineering.

Without the right engineering skills, any training on any technologies is a waste of resources to some extent in my view. We would just write the same software using different syntax/framework. Old habits die hard, and would still drive the process. Remember that our ultimate goal is to produce maintainable code, which can easily serve different technologies/data persistence platforms/computing platforms, not chasing fancy stuff.