Agile vs agility (Enterprise Data Analytics Product Development context)

?At first glance, the topic raises a basic question - “Isn’t Agile supposed to provide agility and if so, why Agile Vs agility”? If you had that question, you are not alone as the goal of Agile is to provide agility for organizations so that they can adapt to the fast-changing external and internal environments. Agile is believed to be different from traditional approaches in that it aims to return value much quicker. If taken at face value, Agile should solve many challenges, provide better value, and make everyone efficient and happy – it does not happen always.

This article tries to recognize some of the common challenges that are faced by organizations that are trying to build their data and analytics capabilities using Agile methodologies and explores potential solutions to these challenges.

#1 Customer misconception around the value of a “Working Software”

One of the four core values, as stated in the Agile Manifesto, is “Working Software over Comprehensive Documentation”. This simply means that Agile aims to prioritize customer satisfaction through early delivery of valuable software, as opposed to traditional approaches, where developers typically compile the needs and requirements of the users and then build the software all at once. While the former may sound like an efficient approach, the adoption of it depends on the stage of development.

Amongst the returns that can be realized from a high caliber “working software”, scalability and performance determine its strategic value to the organization. How can you measure the scalability and performance of the software from a strategic standpoint? Ask yourself these:

1.     Can the product deliver insights consistently and in an automated manner?

2.     Can the product be scaled across different data sources, domains, time periods, etc. without too much rework?

If the answer to either of the above questions is a NO, what you have is not a “working software”, but at best a Proof of Concept/Demo.

Suggestion: For Enterprise Data Analytics Products, setting up a strong base* system is critical to achieving a sustainable “working software”. While this may take a few sprints to set up, this investment is important for making it a strategic product that is scalable and automated.

 *“Strong base” would ideally be a scalable platform with the critical mass of data being available in it to be able to deliver insights to the business.

#2 Importance of the “Big Picture”

Closely following the ideology from the previous point is the importance of keeping alive the “Big Picture”. The “Big Picture” can be articulated through the efficiencies that are expected to be derived from the product.

Let us not forget that Agile could be hectic. There is an expectation for incremental value to be delivered every 2 or 3 weeks, so the focus is always on delivering the current sprint as no one wants to look bad in front of the stakeholders (and the stakeholders have been made to believe that Agile is the magic wand which will solve all their problems). Ceremonies such as retrospective meetings become a mere formality and the focus is only on the “sprints”. The collateral damage to this approach is the “big picture”,

When the big picture is lost, every sprint becomes a tactical piece of development. At the end of many sprints, there are many small tactical systems built instead of one strategic solution. The team hits a dead end where no further tactical solutions are possible, and significant rework is needed to move forward towards the end state of the product.

Suggestion: It is recommended that the team takes time out between “n” number of sprints to think about the big picture – “what has been done so far and what is needed to achieve the end state?”. This approach might result in rework or adjustment activities that are needed. These activities can then be put into a sprint of their own. However, it is imperative to take stock of the situation this way in order to make the product resilient and increase the velocity of future Sprints.

#3 Responding to change

Agile is all about responding to change quickly and effectively. Most customers I have spoken to say that they do not get the required level of agility from their “Agile” teams and are frustrated. Further analysis would say that the team configuration could make a big difference in this area.

When the team has narrow (vertical) skills (say one member who can deliver only on a particular technology, a member who can only test, a member who is only functional etc.), the team is pushed to follow a particular pattern of work even when there is a need for the work pattern to change. When this happens, agility is lost and “Agile” becomes “mini-waterfall”.

Suggestion: Build a team with members who can contribute to multiple areas of the product. (It is worth noting that the large and very successful IT companies are set up in a factory model which mandates people to be less versatile and thus increasing the bureaucracy and works against agility – this barrier needs to be broken)

 #4 Agile training & its application:

A good aspect of the Agile training is that anyone, from any background, with any level of experience can take part in it. In addition, they can also be Agile certified in a matter of a few days. The training skills people on the principles of the Agile methodology. However, this does not necessarily mean that they can readily contribute to a specialized area like Enterprise Data solutions.

While planning the first sprint of a global data initiative, a Scrum Master asked me “why can’t you give 1 dimension and a few measures at the end of the first sprint so that users can start using the product?” – I was stunned, explained why it does not make sense.

Suggestion: Agile is not a substitute for data analytics expertise and having “Agile only” members cannot help the team. While Agile principles are valuable to the team, the team should consist of individuals who understand the principles of data analytics product development.

In summary, a methodology like Agile lays out guidelines to achieve agility, but adjustments are needed depending on the situation. For example, Scrum is a concept that is meant to aid the implementation of Agile and should not be treated synonymously to a point where it is enforced at the cost of common sense. As an organization, while it is beneficial to use any such methodology to increase agility, it is worthwhile to pause and think if that goal of achieving agility has been realized. If the answer is No, intervention is required to correct the course.

(Thanks to Shibhu Kumaar for inputs and feedback on the article)

Pic credit: https://www.godparticles.in/