But is your software agile?

By now, almost all software organizations have at least heard of Agile software development, and mostly adopted at least some practices recommended by the Agile movement. whether it's Scrum, XP, Disciplined Agile or any of the other inhabitants of the Agile zoo - management of software development has largely become Agile. But what about the software itself?

My experience is that even in an Agile organization, it is not that hard to produce software that is itself monolithic, inflexible and in general, contrary to the goals of agile development: deliver stakeholder value, respond well to changes, and build quality products by building quality into your process.

I have observed, broadly, two reasons for why that happens. The first is lack of design. First-generation Agile methods like Scrum have bred teams that look down of thinking ahead. The result is early commitment to architectures which are too limited, but cannot easily be changed by plain ongoing refactoring. This kind of malpractice is addressed by second-generation Agile frameworks.

The second reason is the wrong kind of design. Or rather, since "wrong" is a matter of context most often, let's just call it non-agile design. Agile teams still contain a lot of traditionally-trained programmers, who tend to think in terms of command-and-control structures, hierarchies, and grand top-down designs. I, on the other hand, have been influenced a lot in my development as a software designer by different thinking sources: the free-software community, in which reuse and modularity are prevalent and even essential; classical-liberal political circles, in which the value of distributed action is well understood; and the Israeli Defence Force, which surprisingly understands that too, perhaps better than most other military organizations.

Recently, while preparing to unleash another design proposal on my poor team, I have tried to collect and organize my design principles. I would like to present some of the items here, as what I have grown to think of as Liberal Programming.

Trust the civil society

The general principle is to treat your pieces of code not as an army, dedicated to fulfilling orders carefully thought of in order to achieve an overarching goal. Rather, imagine your code as the civil society: a broad collection of agents, each with its unique qualities and intents, collaborating freely through some coordination mechanism. In a free society, this mechanism could be the free market, or a host of voluntary non-market association forms.

A main observation of classical Liberal thought is that allowing free agents to choose their collaborations freely among availabe unique opportunities gives a much higher productivity compared to top-down control. But one might wonder: free agents of the civil society also have their own goals, they don't serve a single goal, which a software product supposedly does.

This is where the IDF comes in - it has won all its wars using a command philosophy that not only trusts the bottom ranks to take initiative and apply its local knowledge (which is often better than the higher levels' view of the battlefield). It actively encourages it, by sharing with each level the goals and designs of the level above it. It actively distributs knowledge that would allow the lower levels to act on their own for the common goal. This sort of thinking has meanwhile spread also to business. The main realization is that even when striving for a single common goal, one still gains a lot from autonomy and free collaboration.

Prefer communication protocols to compile-time checks.

This is one way to realize this trust of the civil society. Languages such as C++ allow you to be very strict in what you allow a piece of code to do. Although this first arose from a simple technical need to describe to the compiler what its data types are, some saw the opportunity to use types and other restrictions to guard against common coding mistakes. While this is generally a good thing, it can be taken too far.

To preserve flexibility, sometimes you have to replace static checks with things like "design by contract", where you describe the nature of the inputs and outputs of a unit in documentation rather than in code. You thus put the responsibility on the caller to adhere to the contract. Although you allow some errors in development, you are then less bound up when trying to be creative. This principle is applied when using "duck typing", for example. You judge an input not by its type but by what it can do.

Another form is to use dynamic structures such as hash tables, expandable lists or JSON objects as communication objects. While it means you will have to do your own type checking, it allows you both future change, and giving responsibility to the component who eventually has to read the data and interpret it. Intermediate layers that route the data are kept ignorant and therefore more general and open to creative usage.

That is not to say, of course, that all units should be that dynamic. Separation of concerns and isolation of code-paths to prevent spaghetti code is still valuable. Who communicates what should be made clear, and some components should remain as pure servants. Context is king, as Captain Lorca would say, but the context should be that of trusting the civil society.

Minimize red tape - Provide libraries instead of superclasses

You sometimes find that a large hierarchy exists just for the purpose of providing a service to other code. however, the use of this hierarchy requires and enforces many limitations, because the subsystem was developed with some use case in mind, and all its data structures and methods are geared toward it. To reuse it for something else, you must jump through a lot burning hoops.

A library of utilities, designed from the start to be used by more open use cases, is less limited. Think of the difference as that between an Ikea cabinet set and a Lego set. Both provide components, but only one provides freedom to change and react. This is the attitude I took when I converted 3DPTV, an aging package for particle tracking applications, into liboptv , a useful library for the same source. It allowed both simpler, more focused UIs, and use in scripts and so on, combining the basic functions in ways not allowed by the monolithic hierarchical code that preceded it.

Of course, there is a balance to strike here. Some programmers live in a cloud and try to make everything as general as possible. Real applications require somene to specify things somewhere. Again, the question of where to specify and where to generalize should be decided based upon the overarching idea - trust the civil society.

Enforce modularity by variety

This principle is both a thought guide and a management principle. If you try to write a library or use a communication protocol, but only two components are involved, in a master-slave relationship, then what you will get is vertical integration, not modularity. It is almost inevitable from the human aspect. Programmers who need to achieve a goal would sometimes take a shortcut, or short-circuit some data paths in order to "get it done". They would often be justified by the business reality.

Only having a variety of active users for a module will, in the long term, preserve its unique intent and broad applicability. That is simply because any short-circuiting or other monkey-business designed to advance one use, would break other users, and force the programmer to think it through, try to understand better the overall design, and be more careful.

Conclusion

I'm sure that some would think of what I wrote as trivial. But when I write, it's because I experienced a need for it. I have encountered several cases in my experience, where illiberal programming resulted in code that is hard to handle, in lost flexibility, and even in quality problems.

How are things in your company? do you practice Liberal Programming, or are you a top-down dictator? Is it working for you? Have you met overly liberal code? Let's hear about it!