How to think smarter about software development

Edge.org is a website where some of the most renowned intellectuals discuss ideas and express opinions. Every year, the Edge, asks a question which is answered by a large number of leaders in various scientific fields. The question in 2011 was ―What scientific [1] concept would improve everybody‘s cognitive toolkit? There were 165 responses from eminent thinkers. In this article I have listed the concepts which are most relevant to software development.

How to read the links

All the responses to the Edge question are available on the web. I would recommend you first read the links I have provided before reading my commentary. It is important to read each link — the description of each concept will change the way you think. I recommend you take printouts of the web pages and highlight relevant portions. Some of these articles are heavy reading. If you wish you can read one or more every day. When reading the links, try to put aside any preconceptions you might have about software development.

If you are a software developer, this will make you smarter!!

Wicked Problems

(Read the hyperlink before reading ahead)

Creating software can be challenging. Software development is definitely a wicked problem. Developers and testers are good at solving difficult problems. Wicked problems require a different mindset. I like to think that a smart developer has already tried to solve a ‘wicked problem’. This makes testing even more ‘wicked’.

The Name Game

(Read the hyperlink before reading ahead)

?The best example of the ‘Name Game’ is the word ‘test’ (related to software). There is no one in the world (including grandma) who would admit to not knowing the meaning of the word ‘test’. In reality the word ‘test’ is highly misunderstood (in the context of software). It is important to understand what ‘test’ really means. Another word which is a victim of the ‘Name Game’ is ‘agile’.

(See the related concept: The Dece(i)bo Effect)

Experimentation

(Read the hyperlink before reading ahead)

If what we do isn’t ‘testing’, what is it? It's experimentation. This isn’t being pompous. As in real life, when you are trying to determine what works (or may not work), you need to experiment. The agile movement has created practices which make software increments available in a few days. There is no reason not to start experimenting immediately and all the time. We would be better off if we realized that ‘user stories’ (as used in agile) are experiments. (Note that most non-scientific people use the word ‘experiment’ in a negative way, as if it is something frivolous as opposed to being deliberate. See the related concept ‘Science’ and the ‘Uselessness of certainty’ to remove that misconception.)

Gedankenexperiment

In the case of software, unlike physics, you can conduct physical experiments (on the software). However, the gedankenexperiment is a great concept to stimulate thinking and can be an alternative or precursor to physical experiments. It's another alternative to the heinous word ‘test’.

The Double-Blind Control Experiment

It‘s worth understanding double-blind control experiments from Wikipedia. While the previous concepts of experiments and Gedankenexperiment are powerful alternatives to ‘test’, a ‘control experiment’ is a much more specific, rigorous procedure. One of the concepts that refer to the software development process is Timo Hannay’s concept, ‘The Controlled Experiment’ (this is the only concept not available online). Hannay explains that online companies like Google, ―don’t anguish over how to design their websites.

“Instead they conduct controlled experiments by showing different groups of users until they have iterated to an optimal solution. (And with the amount of traffic those sites receive, individual tests[2] can be completed in seconds.) They are helped, of course, by the fact that the Web is particularly conducive to rapid data acquisition and product iteration. But they are helped even more by the fact that their leaders often have backgrounds in engineering or science and therefore adopt a scientific-which is to say, experimental-mind-set.”

Although this is a great example of a controlled experiment, the more powerful concept is that the entire process of software development can be viewed as a series of experiments (see Roger Schank’s concept ‘Experimentation’). Why do we only selectively perform controlled experiments? Can‘t we conduct such experiments all the time? When we don‘t have access to customers, can we use proxies such as in-house staff, or even developers and testers?

Cumulative Error

(Before you forget, read the hyperlink before reading ahead)

While Jaron Lanier is referring to the flow of information in high end financial systems, software development itself is an elaborate game of ‘postman’. The agile development community has acknowledged this problem by declaring the importance of verbal communication over ‘messages’ inscribed on paper. Unfortunately, some traditional teams[3] haven‘t caught on and continue to play ‘telephone’, without realizing it.

I have often seen software developers and testers struggle with this cognitive fallacy in vain. Mr. Lanier explains this beautifully, ‘The illusion of Platonic information is confounding because it can easily defeat our natural skeptical impulses.’

The virtues of negative results

Related to the concept of ‘experimentation’ is the concept that ‘Failure liberates success’. In the world of software development, the idea of testing (evaluating/passing judgment), and the idea of logging defects (someone creating something ‘defective’/‘faulty’) has rubbed people the wrong way. In the testing community many have written about how to log defects so as not to offend. The software development community could benefit if engineers realized that failure is a cornerstone of progress/science. The agile movement has at least partially recognized this scientific concept — short of equating failure with progress. The idea more common in agile is to limit the scope of failures. While agile retrospectives result in learning from failure, I don’t think failure is actively used as a tool for learning, i.e., engineers don’t intentionally make software fail to understand flaws in design[4]. Good software testers have been doing this for a long time.

You can show something is definitely dangerous, but not definitely safe

This concept should put an end to the angst surrounding defects in the software development world (defects found during development/before release to the market). Since you cannot prove that the software being developed will cause no harm (to users), you continue to find ways in which it can cause harm (defects). Tester redeemed!

The uselessness of Certainty

We don’t need ‘scientifically proven’. We need enough information which will allow us to make decisions and act. The agile movement has successfully thrown off the shackles of Tayloresque management and adopted a personal, direct approach based on valuing people and their skills and personal needs. However, in traditional testing and management, the word ‘scientific’ often shows up as the preferred approach. ‘Scientific’ implies well documented and detailed, and engineers who follow instructions. While in most teams there is no compulsion on rigorously following instructions, i.e., engineers are allowed to question what was documented, the bigger point is lost, viz., there is no value in certainty!! Will the software crash when it has to connect to eight servers? Even if you ‘test’ it with eight servers, you can‘t be certain. If it doesn’t crash, you could be proven wrong, when the environment changes. The key challenge in software development is to correctly qualify your level of uncertainty.

Science’s Methods Aren’t Just For Science

Science is a way of thinking, of making better approximations of how things are. For software developers, software development is a hard technical skill with the discipline of math or ‘science’. Agile has greatly broadened that outlook to include communication, especially with customers, teamwork and an overall understanding of how people work. Despite those gains, ‘testing’ is probably still seen as ‘fuzzy’ by software developers, while most traditional testers would balk at equating testing with ‘science’. Software engineers (testers) would greatly benefit by using scientific methods to make sure customers perceive value in their products (or they should quantify the uncertainty about risks to that value). Scientific methods don‘t need to include conducting experiments. They could be activities like a ‘journal club’ or applying different ways of thinking such as vertical, horizontal, or learning to take meaningful notes when observing software behavior.

(See the related concept Science)

Q. E. D. Moments

You need to experience the divine moment of proving a logical or mathematical problem. This may not be straight forward if you don‘t have a math/science background. When developing and testing software, i.e., determining value and what might go wrong, you can conduct investigations to validate your hypothesis. In the long term, it‘s still worth pursuing the goal of experiencing a QED moment. I have listed only a small number of the concepts discussed in the book and website. Most of the concepts are worth reading and will make you think smarter. Many of the concepts can be applied in everyday life, not only to work.

Notes

1. From the website: ―The term ‘scientific’ is to be understood in a broad sense as the most reliable way of gaining knowledge about anything, whether it be the human spirit, the role of great people in history, or the structure of DNA. A “scientific concept” may come from philosophy, logic, economics, jurisprudence, or other analytic enterprises, as long as it is a rigorous conceptual tool that may be summed up succinctly (or “in a phrase”) but has broad application to understanding the world.
2. Although the use of the word ‘test’ in this context is fine, do cross-reference the concept ‘Nominal fallacy’.
3. This is not a blanket judgment against teams not using ‘agile’. Teams not using agile have effectively promoted similar structures to improve communication.
4. While TDD is a great concept, starting with failed tests is different from the concept explained here.