Finding the Balance: Agile User Stories

As part of a team of business analysts that is now about a year and a half into a transformation from waterfall to a more agile-aligned methodology, I am keenly aware of a tension that exists between two important and often competing notions: Detail and leanness.

In the old days of waterfall, we didn’t have this tension. We wrote our requirements in the form of traditional “shall” statements, and we were expected to capture every possible detail of the proposed implementation. Our goal was to minimize, if not eliminate, any questions or clarifications that might be raised by developers or testers after the requirements were finalized and handed over for implementation. Any such discussion, especially if it led to changes in the requirements, could introduce delays, extra work, and even missed delivery commitments. We needed to “get it right the first time,” and to do that, we needed to anticipate every possible question that might arise during implementation and answer it in the form of clear, detailed requirements. Never mind that some of our more complex change requests might add up to requirements in the quadruple digits!

As our team transitioned to agile, we learned all about keeping things lean, starting with one of the key agile values:? Working software over comprehensive documentation. We learned how to write user stories with concise statements focusing on what the user needs to accomplish and why. We learned to craft acceptance criteria that spell out in plain language how the implemented solution would allow the user to achieve their goals. We learned to keep our stories as brief, simple, and lean as possible. Short, simple user stories are easy to work with for many reasons: The small size makes it easy for developers, testers, and of course the customers, to digest and understand. We write our stories in JIRA, which allows us to keep the user story, acceptance criteria, any needed attachments such as mockups, and discussions and clarifications all in one place.

It’s also important for sprint planning to have units of functionality that are broken down into the smallest units practicable. A simple analogy illustrates this quite well: Imagine a gallon-sized jar. That jar represents your implementation team’s capacity. Your job is to fill that jar as efficiently as possible, minimizing the empty space – in other words, to keep your team fully engaged in sprint activities. The items you are given to fill that jar vary in size: some are jellybeans, some are golf balls, and some are tennis balls. These items represent the user stories. If you fill your jar with tennis balls you will have a lot of empty space in your jar – your implementation team will only be able to handle a few items in each sprint. Even though they may have capacity to do more work, there aren’t enough smaller stories they can fit in around the larger ones. But if you have more jellybeans and golf balls, you can fill the jar far more efficiently and make the most effective use of your implementation team.

But is it possible to be too lean? Keeping user stories short and focused comes with some downsides. When a large, complex piece of functionality is broken down into a large number of small user stories, it can become difficult for the team to see the big picture and design holistically. Dependencies between components of the solution can be lost, leading to an implementation that feels disjointed and inconsistent. In extreme cases, important functionality can be missed because different stories cover different aspects of what should be an integrated workflow.

Leaving out too much detail in the stories can also lead to omitting key information needed by developers and testers. This leads to delays during implementation as developers and testers wait for clarifications from business analysts, and even rework as business analysts revise and rewrite stories to incorporate the missing information.

With all of that in mind, here are some strategies and techniques that have helped our team find success in striking an appropriate balance in our user stories.

Story Maps. A story map is a visualization method that arranges user stories in a grid. The left to right flow represents the user’s path through the functionality in each story, while the top to bottom flow represents the order of implementation based on priority and value to the users. Traditionally, story maps are used to visually lay out the complete product backlog across time to aid in prioritization and facilitate a release strategy across an entire product. However, story maps can also be a powerful tool when used at the level of a particular epic. In agile software development, epics are often used to describe a complex piece of functionality, which is then decomposed into any number of user stories. This keeps the units of development manageable but makes it easy to lose the “big picture” view of the workflow, module, or major feature to be implemented. A story map at the epic level can be an invaluable aid in visualizing how the stories comprising the epic relate to each other, and in what order they should be implemented to maximize value to the user.

Linking and Cross-Referencing. Agile orthodoxy will tell you that user stories must stand on their own. In our experience developing complex enhancements to a mature software system, writing user stories that truly stand on their own while at the same time keeping them short and simple is next to impossible. This is where linking and cross-referencing come in. When one story depends on another, link them together to make those dependencies explicit. In JIRA, we use issue links to define the specific ways stories are related, such as “is dependent on” or “deploy before.” This allows us to break down complex functionality into small, digestible chunks (in true agile fashion) but still identify how each chunk interrelates and fits into that big picture. Making these interrelationships explicit is particularly useful during sprint planning, ensuring that stories are pulled into sprints in a sequence that maximizes efficiency and aligns with the overall design strategy for the epic.

Workflow Diagrams. When your epic introduces a new piece of functionality, or has impacts throughout an existing workflow, create a workflow diagram. This is a great way to visualize the user’s flow through the system and identify all the pages impacted by the functionality. Workflow diagrams also help flesh out navigation-related questions, such as where to direct the user when they cancel an action or an error occurs.

Tables and Matrices. Sometimes the functionality of a story requires the analyst to flesh out multiple “if-then” scenarios. Written out individually as acceptance criteria, these can become very lengthy and intricate. Instead, consider developing a matrix to represent the scenarios in a compact, easy-to-visualize format. In the example below, a matrix is used to define how a system should behave when merging two duplicate contact records. In this scenario, one record is “kept” and the other record is “merged” – that is, relevant information is copied from the “merged” to the “kept” record and then the “merged” record is deleted. The matrix specifies the expected result for all possible combinations of institutional affiliations in the “merged” and “kept” records:

For example, let’s say the user has a “Primary” affiliation to an institution in the “merged” record. The user has a “Not Active” affiliation to the same institution in the “Kept” record. Consult the matrix to see that a combination of “Primary” (across the top) and “Not Active” (down the left) results in a final result of “Primary.”

This technique can be used in any similar situation that requires the comparison of multiple “if-then” scenarios, and allows for a wealth of information to be communicated in a small amount of space.

Roll your own! Don’t limit yourself to industry-standard tools and techniques. Adapt them to suit your team’s needs. In the example below, our team needed a way to visualize the user’s navigation through a workflow. We started with a typical workflow style diagram, with each rounded rectangle representing a page in the system. We then added white, square-corner rectangles to each page to represent the various navigation options the user has on each page such as context menu links or page action buttons. The arrows show the flow through the system for each navigation option.

Conclusion: Our team’s journey from traditional waterfall methodologies to agile practices has highlighted the need for balance between detail and leanness in crafting effective user stories. While the old approach required exhaustive documentation to reduce the risk of misunderstanding, agile principles prioritize concise, actionable narratives that promote collaboration and adaptability.

Yet, as we've experienced, this pursuit of brevity isn't without its challenges. Breaking down complex functionalities into smaller stories can make it difficult to maintain focus on the overarching vision, leading to disjointed implementations and missed opportunities. Moreover, insufficient detail in user stories can impede developers and testers, causing delays and rework.

To strike this balance, our team has embraced various strategies. Story maps offer a holistic view of epics, fostering clarity and alignment across a complex set of features. Linking and cross-referencing user stories expose dependencies, ensuring a cohesive implementation. Workflow diagrams and matrices provide visual aids for understanding complex scenarios, while the flexibility to innovate and customize methodologies ensures adaptability to unique project requirements.

In essence, achieving balance in user stories is not merely about finding a midpoint between verbosity and conciseness but rather about dynamically calibrating our approach to fit the needs of each project. And this, in many ways, is the essence of agile!