Balancing Agile & Documentation Needs

For companies that have been following the traditional or waterfall system of development, one of the biggest concerns they have is how to handle documentation in an Agile world. Coming from this linear approach to software development, the adage for companies using this model was almost “Document the world before starting on a project”. It is hence not very surprising to see organizations and management raising such a grave concerns about how this phase will be handled if they adopt an Agile model.

One of the key reasons for this apprehension is this line from the Agile Manifesto - “Working software over comprehensive documentation”.

The focus of Agile is on getting things done and adding value to the stakeholders. It is built in such a way that the model intentionally avoids getting to work on the peripherals and on activities that do not directly and immediately add value to business. If a customer is able to start seeing something working from the first couple of Sprints, the expression on the customer’s face is anyone’s guess. Not only does this make the client happy but also helps in building confidence on the development team and the vendor. (or the seller in Agile parlance).

If the approach was traditional, most of the initial phases before start of development would have gone into someone working on creating elaborate documents trying to capture end to end requirements and the needs of the customer. While it is okay to document, in today’s world, the pace of change is so dynamic, that the needs of an organization would have completely changed twice over by the time, we complete the documentation phase rendering the entire activity useless. Maybe this process does work when the requirements are completely clear and we know with surety that nothing will change from what has been captured and base-lined.

However we know that this is not true, and that, in itself is the reason for the popularity of Agile. Agile clearly accepts change and believes change is a given and we need to be ready to respond to it. This is a far cry from the predictive approach where as much as we knew that requirements would change, in some way no one wanted to accept it and the whole team went ahead as if things were written in stone.

Having said that, Agile is all about adding value even if that means documentation. Agile would deal with documentation based on the type expected :

• Regulatory Compliance related - Agile is bound to get a buy-in from its teams to allocate effort and time to get compliance items done. Which means Agile, will deal with this item because Agile focuses on what is critical for the success of the project and not being compliant could put the entire project at risk. 
• User related artifacts - If there is value to the user in terms of how to use the software and if user manuals are required, Agile will consider this and help develop artifacts although as per the model this will be taken up as a last minutes activity after completion of the core project related activities. Agile will only implement basic, to the point documents, nothing flowery
• Elaborate Requirements capture - Agile believes in change and in adding value to stakeholders. It does not believe in wasting time creating huge documents as there is no direct value added by doing so. Agile teams would rather spend time working on the minimal requirements it has and get out a piece of code that works and demonstrates the captured requirements before moving to the next set of requirements without having to wait for elaborate documents. 
• More documents - If your needs do or dot not fit into any of the above and you still have documentation needs, sorry Agile can't help out.

Given the above, based on organization’s requirement, it really is a call the management will have to take on how to progress without rupturing the Agile model.

One typical example could be, if my concern was, let’s say attrition. What if my key team members leave before the project ends or even after. If new resources join, how will they know what’s been implemented and where what exists. Although Agile supports a model where all resources are expected to know the entire code base, it still would make sense to have some level of documentation that would give a new comer some insights into what has been implemented.

If the management finds such needs critical, then they can take a decision that someone will be assigned the task of documentation for this specific purpose. However this resource need not be from the core Agile team so as to not impact the ongoing Agile flow. This could be one approach. Likewise, management can tweak the processes by introducing parallel streams of resources or tasks that can be done while the Agile model is in progress. Basically find out ways to use a Hybrid Agile model which helps balancing out Agile practices with organizational needs.

However on a broad scale, for companies looking to get into Agile, the suggestion would be to move away from the traditional approach of having to have detailed overloaded documents to having what is really needed. If some level of retrospection is done on completed projects in the traditional flow, it will become obvious that most of the documentation done for those projects have hardly ever been used and is just lying there on some hard drive.

So, although companies can take an alternate approaches using hybrid models to ensure documentation is done based on the gravity of the need, it is more important that they move away from the traditional mindset and embrace a more meaningful - value driven, incremental delivery oriented, open approach which focuses on having just enough documents and not documenting for the sake of documentation.