Be more agile about agile

Summary

• Agile methodology is amazing.
• When used in an agile way, it optimizes product / software development.
• When used in a dogmatic way it loses most of its benefits.
• Organization and project specifics should drive the details of the process implementation.
• Being more agile about your agile process helps.

History

Agile Manifesto introduced in 2001 with its 4 values (see quote below https://agilemanifesto.org/principles.html), along with its 12 principles (can be accessed here) is short and sweet.

“Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more”

It was created by the group of bright software developers who are, frankly, capable of creating great products under any methodology, but have chosen to share their wisdom for optimizing the software development.

The foundation of the Agile methodology could be traced to Iterative Development introduced as an alternative to Waterfall Development in 1960s and Extreme Programming introduced by Kent Beck in 1996.

By now Agile is a mainstream with many cubicles broken along the way and a whole new industry created to sell tools, process design and training to make sure that you are not missing out on the state of the art, best of the breed, etc. agile implementation. And that is largely a good thing.

Great things about agile methodology

1. It cuts the bureaucracy and extra layers around the software development by enabling people who do the architecture / design / development to communicate directly to people who know exactly what they want (or capable of developing that understanding along the way).
2. It shortens the product development cycle because it recognizes that you cannot know everything at the beginning of the project. That enables more work to be done in parallel. For example, you can agree on high level architecture and requirements (user stories) and design some details later, in parallel with developing and testing the fundamental building blocks.
3. It improves the productivity by having development team to share their progress daily. It is a psychological trick that increases sense of ownership and responsibilities and it allows to address issues as soon as they arise.
4. It minimizes rework because it forces business product owners to be more disciplined in defining their needs (by developing and communicating user stories). It also supports much quicker feedback when adjustments are needed.
5. It provides more realistic schedule for each incremental milestone with the help of the comparative story estimation. There you assign a value to the task you understand well and size up remaining tasks relative to that one using a Fibonacci number scale (1,2,3,5,8,13,21 and so on). It also leverages the wisdom of the team to do the estimation making results potentially more accurate.
6. It boosts morale because short iterations with delivered functionality (not necessarily production-ready functionality) provide a sense of accomplishment that might be hard to see otherwise for large scale projects. 
7. It often has other perks like cool office space, toys to play with during the breaks and free food.

Few things to be more agile about

It often happens that consulting company is hired to define an agile process for an organization. Once defined it is enforced across organization and sometimes it works really well but sometimes it does not. Based on my experience with agile projects here are things that are worth paying attention to.

• Sprint Duration. Most of the time it is two weeks, but it does not have to be. It might be more practical to adjust it for the project or even within the project based on the team’s schedule and nature of the tasks. Significant research or challenging development / integration tasks that go well together warrant longer iterations. The changing it sometimes is objected because of the team velocity metrics, but practicality should rule (see more on team velocity).
• Production deployments. Some flavors of agile mandate to deploy and show to users the production feature at the end of each spring, that is usually two weeks. It works great for the textbook or for simple maintenance, trouble ticket or small enhancement tasks. It usually does not work for complex products where deploying something early just for the sake of deployment is not the best use of everybody’s time. It is worth to be flexible about it. Often enough initial sprints should be focused of design, technology research and user interface mock-ups. None of that results in production deployments, yet the results of that work could be effectively communicated to the business owners.
• Documentation. Agile process occasionally ignores the system design and other documentation. The excuse is that stories for the project are written well and they are in JIRA (or TFS or any other tool of the day). That might be OK or might be not. Complex systems are impossible to understand by just looking at 1000s of stories that spread across multiple releases. It might be super helpful in such cases to have a solid high-level design / business rules / architecture documentation that allows development team to understand the functionality. If it is written and organized well it would help business users as well.
• Stand ups. Stand up meetings are usually daily, but there are cases where it might not be practical. In theory agile team is always small, all tasks are relevant to everybody on the team and all relevant people are always available. In reality, that is not always the case. Also, the nature of tasks varies and sometimes they require deeper focus. Based on that there are projects or phases of the projects where stand up two or three times a week is more practical.
• Pair programming. This could be very useful or other way around. When mandated all the time, it usually does not go well. The truth is that a) not every piece of code requires two people to write and review it at the same time, and b) personalities of the people on a team might not go well with pair programming. Pair programming is a great item in the agile medicine cabinet. It just should be used as needed. There are other options to assure desired quality of the code and to promote collaboration even when pair programming is not consistently used. That could include mentoring and code reviews / approvals strategies.
• Testing. In the ideal world complete testing happens along with the development as each user story defines perfectly well all entry and exit scenarios for both happy path and exceptions. Also, test cases are added to the test case repository and become a part of automated regression testing. If that is where you are, following the textbook approach on agile testing is great. However, many organizations never evolve to that level. That is because if that testing culture is not established from the very beginning, that effort itself is a very large project and often cannot be categorized as capital work and often is not prioritized high enough. The worst thing to do is to pretend that you are there because then the results would be faked, and quality would go down. The best thing to do is to realistically assess your testing state and then adjust your testing approach accordingly (that could be a subject for another, even longer, article). 
• Story estimation. Most of the time the textbook approach of estimating each story using the Fibonacci sequence works really well. This is because people can compare numbers in that sequence easily. For example, you might not be able to tell if building in front of you has 23 or 25 stories, but you usually tell of that is closer to 21 or 34 stories. The idea is that you take the task you know well; assign it (with the consensus of the team) value in the middle of the list, let’s say 21; and then measure other tasks relative to this one (one task might be 55 and another 3). But sometimes it does not work well. The examples are: tasks are not easily comparable; only one person is working in each area let’s say front end, back end, middle tier, data set up. In that case the estimate numbers become very subjective and not consistent across areas. It might be OK to still use them in hope that they will get better over the time, or just ignore the story estimation and trust that team does its best accepting stories to the sprint and adding more if needed.
• Team velocity. That is usually how many story points are done during the sprint (or per week if sprints have different durations). Because story points are subjective and established by each team, they only offer potential consistency within the team. It is very tempting to reward team with maximum velocity or look for continuous increase of velocity, but since metrics drive behavior, it might trigger people to add more points. Velocity is a good tool to use inside the team to help the development team to track its progress… if team finds it meaningful and helpful.
• Open space. The idea is that open space enables collaboration and it often does. It also increases the level of distraction. When distraction level is high people wear headsets, put music or noise reduction on so they could concentrate. Then it is harder to reach to them than just knocking on the door of the cubicle. Sometimes people who do not share any tasks are forced to share the work-space for the sake of being agile and be in an open space. That increases frustration on the team and reduces the productivity. Also, personality plays role in how useful open space is for the individual productivity. People who are more introverted could be brilliant developers and yet perform worse in the open space. 

The list could go on… The idea is that you want to continuously observe what works and what does not and be open to adjust, because process is not the purpose but just a tool to get better results.

Establishing an agile process is an agile project

Each organization is unique. That includes

• organizational culture
• type of development (new applications, legacy systems, off-the-shelf products, coordination of the vendor’s work)
• nature of the business (level of complexity, regulatory environment, cost of errors)

Within each organization there could be different types of projects that require different approaches.

One size usually does not fit all, and right development process is just one of the key ingredients to the success of the organization.

Furthermore, agile methodology is not an exact science. It is not even a science. When not applied carefully it has a risk to become a cargo cult science, a term that physicist Richard Feynman introduced in his 1974 commencement address at the CalTech https://en.wikipedia.org/wiki/Cargo_cult_science 

Agile methodology is a set of practices that could be utilized for the benefits of the organization. To define what would work for you the best is the project in itself. It should follow the same steps as any software development project: figure out requirements / user stories; design the process steps, tools, templates; implement some steps and test them, and so on. Treating process implementation as a technology project could be a subject for another post, but this is a gist of it.

Once process is defined, it should continue to evolve. This is where getting the open and honest feedback from development teams and business partners is critical. The only thing that is more critical is to act effectively on that feedback. This is what makes the process truly agile.

Conclusion

Agile is great!

It is an exciting but complex and involved project to design (or adjust) agile implementation that fit your company and your projects the best. It will take few iterations to make it right. As time goes on, you might have to adjust it again, but that is what agile development is all about.

The end-goal is never the process itself, but great product you create, team having great time creating that great product and ultimately - happy customers. Agile methodology could be a powerful tool to get there. Be more agile about agile!

And that is the story.