#0 What is Software Engineering?

With Arranged ROMs (Rational Opinions, Mostly) , I plan to write about software engineering, among other topics (system design, product design, programming languages, technical books, ...). Sharing my thoughts on what software engineering is, from an high-level perspective, consists in a good starting point. Hence this first article.

Engineering

Wikipedia defines engineering as "the use of scientific principles to design and build machines, structures, and other items". The operative words here are the "use of scientific principles", "design" and "build"; the rest of the sentence hints that engineering applies to a variety of fields. The "use of scientific principles" suggests engineering requires a structured approach (methodology), relies on established facts (data), and leverages on experimentation (research, test). The words "design" and "build" indicate engineering is about the what and the how. "build" is important as it infers the design is meant to be brought to existence. If it wasn't the ultimate goal of engineering, it would be called... essay, theory, or fiction.

Aside from Wikipedia's definition, I read in the past that "engineering is a function of constraints". This definition stuck to me, and I gave it a lot of thoughts. I later came with mine, as an evolution:

engineering is a function which, given an array of problems and constraints, returns an array of solutions

Or, as a formula:

engineering([problems], [constraints]) → [solutions]        

Writing formulas makes the content more serious, right?

An example

Let me share a small example of engineering. I had the idea of writing this article months ago when my wife asked me to drop her phone from our window. Some context is required, here. Our apartment is located on the second floor (counting ground floor as 0th) and the immediate surroundings of the building are concrete. On this day, I had already started working from home. My wife had just left and was about to drive to work when she realized she had forgotten her cell phone. She rang the doorbell from the main building entrance, and asked me to throw her cell phone from the window. This looked risky to me: the device would very likely break into pieces if ever I missed my shot... or if she missed catching it. My wife was in a hurry, she couldn't afford the time for one of us to walk the stairs. I understood the problem (handing the phone to my wife) and the constraints (shipping by air, safely, quickly). Being an engineer, I designed a solution:

I looked around me, put the phone between two blocks of foam, inside a plastic salad box, and wrapped it with a rubber band. It worked! And should it have hit the ground floor, I was confident the foam would have mitigated the impact. This was an example of engineering: taking a problem, constraints, and yielding a solution.

Software Engineering

Wikipedia defines software engineering as "a systematic engineering approach to software development", and states "a software engineer is a person who applies the principles of software engineering to design, develop, maintain, test, and evaluate computer software".

I would have ordered the keywords as: design, develop, test, maintain, and evaluate. Note this last word appears only twice on the page, and can be included into the meaning of the others.

The presence of "develop", as a subset of what a software engineer does, points at the difference between engineer and developer roles: if you only implement designs of others, then test, and maintain, you're a developer. And it's great! The world needs awesome developers who can translate designs into acute, well-executed implementations. Note you may have a developer title and still be an engineer. I held a Technical System Analyst position at some point in my career, and I was doing way more than just... analyzing technical systems. What's important is what you do, not what your title says. I'm digressing, let's return to our initial topic.

So, from the engineering definition written above, and this additional light shed on software engineering, what do software engineers do? My take on this is the following:

Software engineers seek problems and constraints to design, implement, test, and maintain solutions addressing them. When choosing from multiple solutions at each stage, they revisit the problems and the constraints. They reduce the scope, or extend it to account for more present or future problems and constraints, in order to limit the number of solutions to choose from.

Problems

Software engineers are problem solvers. They don't hide so others take the lead in solving problems. Actually, the more senior and experienced engineers are, the more they seek problems to solve on their own. These can be current or future problems, for current or future users, customers, or even engineers.

When designing, software engineers usually tackle more than one problem. Under the umbrella of "provide a better business engineering and reporting tool", lie the problems of sourcing, transforming, and storing the data, computing the business indicators, offering interfaces for superusers to configure the application — not to mention the problem of displaying metrics in an understandable fashion to the users.

Software engineers revisit existing solutions by challenging problems which lead to such design or implementation. What was a problem last year (generating reports twice a day, plugging in 10+ new data sources every week, serving 10,000+ users) may no longer be one today. To be honest, this is rare. The opposite (generating reports more frequently, plugging in more new data sources every week, serving more users) is more likely. Still, software engineers may have created a tactical application (let's say a badge scanning system) to fit a need, pending to a long-term system to take over the problem. Once this happens, the solution can be taken down, or reduced to edge cases the new system doesn't cover. Revisiting solutions is also the opportunity to consider new problems an existing system can evolve to address. The multi-channel component informing users of the availability of a report can now be upgraded to become a generic notification system. Software engineers work with product owners to evaluate how users actually achieve their goals with the system — and they can be creative, bending an application to fulfill unexpected needs.

Constraints

With no constraints, all problems can be solved. Finite amounts of time and money are common constraints. Availability, response time, number of parallel executions, mandatory locale support, permissions — all of these are constraints to factor in the design and the implementation of the solution. All functional and non-functional requirements are constraints. A team strong preference for JAVA over Python is also a (somehow softer) constraint to factor in.

Solutions are created by humans (engineers) for humans (users). Even an API shall be thought with users in mind, since there will always be a time a human reads its documentation, or deciphers the error message the service returns. Software engineers seek to understand who these users are, and what their own constraints in using the future solution are. Developing without knowing your users is like offering gifts to kids without knowing who they are, what they like, where they live. A bike is not a great present if they're not old enough to learn how to ride. A saxophone is not a great present if they live in a small apartment.

Software engineers revisit existing solutions by challenging constraints which effected their design or implementation. What was a constraint last year (CPU, memory, bandwidth, storage, money, number of users to serve concurrently) may no longer be similar today. The cost of technical resources usually trends down, while performance increases. Deploying to less hosts which are more powerful can improve the overall deployment time (at the expense of reduced redundancy). Another example: while migrating users from a service to another, software engineers can plan to downscale progressively the resources of the former until the migration is complete. The constraint of writing serverless functions in a specific language can be lifted when new runtimes are supported. The constraint of pre-processing a large data file can be removed when a new column in the source allows to filter out unnecessary records.

Solutions

In the process of designing, implementing, testing, and maintaining solutions, multiple options can arise. It's rare there's one single possible solution to the combination of problems and constraints explored. If there's only one suggestion, ask the opinion of a second engineer :-)

While designing a new feature, a common question is whether it shall be a new micro-service, or part of an existing service. When starting the implementation phase, a team can face the choice of writing the code in a programming language members have experience with, or one which the industry praises for such task. When considering the tests to write, the team needs to decide if it relies on unit and integration tests only, or if it goes the extra mile and write more difficult user interface tests. Last, during the period the team maintains the service, it must assess whether to bump version number and migrate code reactively or proactively. After years of existence, the question of replacing the service usually surfaces — when it's not simply the call to shut it down.

Software engineers evaluate these options, reaching out to other teams, senior advisors, or product owners if another perspective or more context is required. Ultimately, software engineers are left to exercise their good judgement, based on their knowledge and experience. Great software engineers are right, a lot; they're occasionally wrong, and they learn from failures. As much as possible, software engineers make 2-way door decisions to mitigate the impact of erroneous calls.

A word of caution

You know the saying: "if all you have is a hammer, everything looks like a nail ". I would also say "if what you use most is a hammer, everything looks like a nail". Software engineers have to both use their experience, and set it aside to consider problems and constraints from a fresh perspective. Never underestimate the benefit of a junior asking naive questions! Sometimes, the solution is not a new system or feature. It can be a better user guide, a new process, or an organizational change. The best solution, accounting for the constraints, can even be living with the problem, knowing it will fade away in months, or years.

Thanks for reading this far

These are my thoughts on what software engineering is. If this is not your newly preferred definition or illustration, I hope it still sheds some additional light on what software engineering is. In any case, please leave a comment whether you appreciated this article or not, sharing your own thoughts on software engineering.

Stay tuned for another article on Arranged ROMs next month. Subscribe to the newsletter to be notified when future articles are published.