Agile Project Management and "Normative" Paradigms

In software management conferences around the world, the topic of “agile” is now commonplace. Reform of the traditional approaches to managing software development projects is driven by several factors, not the least of which is some spectacular failures of software projects. Ranging from the IRS to the FAA, to large e-commerce systems, we all have some “war story” of a major failure that can be traced to non-technical causes.

One place to look for the source of failure is the methods used to manage the software project. Traditional project management is based on stepwise refinement methods. These methods make use of linear execution of activities whose plans were developed early in the project life cycle. This “style” of project management has as its source many of the “bodies of knowledge” used to train and assess project managers.

Although some would argue, that these bodies of knowledge don’t recommend a specific method of executing the project. In principle, something like PMBOK describes the various “components” of a project and how they’re related. In practice, the examples in PMBOK are taken as “recommendations” for execution. This of course was not the intent of the author. “These are just examples” is the response, but they are linear, heavy-weight examples all the same.

In the past, this style has been called “waterfall.” Although the pure waterfall methods are recognized as inappropriate for many domains (ERP, CRM, EAI, e-commerce), the legacy of the waterfall framework is still in our thoughts and in our actions. The waterfall method, or its linear phase–centric cousins (spiral for example) suffers from three erroneous?assumptions:

1. Planning – it is possible to produce a plan so that its implementation is merely a matter of executing a defined set of tasks in a predefined order. In fact, planning is a continuous process whose changes are driven by the delivery of software into the hands of the users.
2. Change – changes can be stabilized early in the process. The concept that change must be avoided, somehow “controlled” through management processes, ignores the source of most creative solutions in the software development domains. Business and external market forces usually drive late changes and are a natural part of the business cycle.
3. Stability – management can be given a plan to which it can commit. In fact, by making this commitment, they give up the ability to take advantage of fortuitous developments in the business and technology environment.

The source of these erroneous assumptions can be traced to the “normal–science” paradigm of the current “bodies of knowledge.” [4] By “normal–science” I mean as it is defined by Kuhn in Structure of Scientific Revolutions, Chicago University Press, 1962. This is an oft misquoted defines “Normal–science” as textbook science, which is primarily puzzling solving. It is predicated on the assumption that the community of practitioners knows what the world looks like. All that is needed is a set of tools to gain control of the process. These tools could be actual tools or intellectual tools.

“Normal–science” is based primarily on “normative” and “rational” methods that make use of processes that are known to work. These methods can be conveyed through standards and bodies of knowledge. They are independent of any specific application of this knowledge – that they are domain independent. Finally, they assume the underlying processes are stable and not impacted by the very efforts they are trying to manage. One final aspect of the “normal–science” project management method is the overwhelming emphasis on the “planning–as–management” paradigm. This paradigm creates several “myths,” including: [1]

1. Clear–cut investment opportunities exist with an explicit purpose, the beginning, duration, and end that can be identified early in the project.
2. Low opportunity costs for each business or technical decision exist, in most instances, with a reversible decision process.
3. Feasible, suitable, and acceptable project attributes can be identified early in its lifecycle.
4. Accurate predictions of project duration and resource demands are possible once the requirements have been defined.
5. Worst–case consequences can be determined well in advance and clear–cut mitigations can be created.
6. The failure of the project was due to a lack of technical and managerial skills rather than inappropriate feasibility, suitability, or acceptability of the solution.

Moving to the Next Phase of PM Methods

So what’s a project manager to do? Toss out the old methods and bring in the new? Hardly a low-risk approach to dealing with the current software project management problems.

One approach is to broaden the set of project management methods in some higher–level context. One place to start is to acknowledge that the normative knowledge in the various bodies of knowledge has value, but by itself is not sufficient in the software development domain. This kind of knowledge can be classified as transformational. It describes how to transform inputs into outputs. Requirements into requirements specifications, test plans into testing, progress to plan data into planning adjustments, and so on. This view has its origins in economics as popularized by Michael Porter’s Competitive Advantage , The Free Press, 1985.

There are problems with this transformational approach to project management, not the least of which is the fact that it is not the transformation itself that makes it valuable, but its conformance to the stakeholder’s requirements. Defining value-creating activities is not provided by normative methods. The normative approach provides very little direction in defining what NOT to do during the work process, preventing the minimization of time and resources.

Another approach is to see project management as a flow-based process. In this paradigm the goal is to eliminate waste, lead time reduction, make versus buy, simplification, and variability reduction are all activities found in this paradigm. Just-in-time manufacturing is one example of a project management-based process. In this paradigm the goal is to eliminate waste, lead time reduction, make versus buy, simplification, and variability reduction are all activities found in this paradigm. Just-in-time manufacturing is one example of project management.

A third approach is the value generation paradigm. Here, delivering the best value to the customer is the focus. In software development, value is defined by the customers rather than the designers of the software. Full participation of the customer in this “value defining” process is critical to the success of many agile development processes. [2] This Transformation, Flow Value model is based on the work of Koskela in the domain of lean construction [3].

A Software Project Manager's Toolbox

In order to successfully address many of the issues found in software development project management, we must somehow combine these three paradigms: transformational, flow, and value. We must also recognize the strengths and weaknesses of each paradigm, its applicable domains, and the external forces driving the project that are unique to software development. Along with these “methods”, the myths of software development must also be recognized.

Summary

It is conjectured in many project management realms that a well–functioning bureaucracy aided by scientific planning tools can efficiently deal with a software project through “normal–science” methods. This approach assumes projects are carried out under conditions of complete rationality. It also assumes that projects are repetitive, with their requirements and stakeholder needs built on existing business and technical knowledge.

Software development projects are not conducted under conditions of rationality. The deployment of the system creates new and possibly different requirements. This non-linear feedback is the source of emergent behavior as well as value generation. Software projects are not repetitive, stable, or linear. They are unique, driven by unstable requirements, technology, and market forces, and contain many non-linear activities. Software development is complex, and the exact business and technical outcome is difficult to plan. The processes used to manage the outcome may be chaotic. Software projects are often subjected to forces outside the control of the project manager, developers, and stakeholders.

Discovering and applying PM methods that work in these (emergent) environments should be the goal, rather than trying to change the environment to fit the current normative and transformational (linear, rational, predictive planning and execution) PM paradigm.

References

[1] Austin, Robert D. and Richard L. Nolan, “How to Manage ERP Initiatives,” Working Paper 99–024, 1998.

[2] One sidebar discussion among the normative body of knowledge adherents is that software development methods like RUP, DSDM, SCRUM, and XP are not PM methods, but development methods. To the software development manager, this appears to be an artificial and unnecessary distinction. Getting software out the door that meets the needs of customers is the “management” goal. The “purity” of what is a method and what is a practice is irrelevant at that level.

[3] Koskela, Lauri (2000). “An exploration towards a production theory and its application to construction,” Espoo, VTT Building Technology. 296 p. VTT Publications; 408. https://www.inf.vtt.fi/pdf/publications/2000/P408.pdf .

[4] There are currently 4 distinct project management bodies of knowledge.