Epics, user stories, architectural work items and the backlog.

There are three levels of analysis and breakdown for a software engineering project. It might apply to all engineering work for that matter. The three levels are very natural and logically consistent. You do not need more than these three levels when explaining, keeping track and delivering your work. They are also naturally aligned with the well known ways of agile working, where you have one person owning the business requirements, one person owning the process excellence, and the team delivering work.

Features - Epics

The first level of analysis is the level of business requirements expressed as features of the system, features of the solution. When talking to the customer to understand what they need, we must translate requirements into features we can deliver. We want to deliver features in an iterative way, so they should be nicely separated, with well understood dependencies between them. Let's use an example of a "notepad" style application. Such application might have many features, but let's start small.

The most basic functionality we want is the ability to take notes and save them to a file. We will call this feature "Create and save notes". Each epic corresponds to a single feature, and keeping track of the development is on the highest level represented with multiple epics.

If there is a need for the feature to evolve over time, it is also recommended to version the features e.g. we can have one epic represent the "Create and save notes version 1" and another representing "version 2". On the lower level they will be groupings of use cases or user stories, which will summarise the differences between versions.

Use cases - User stories - Sequence flows

The second level of analysis is the use case level. Each feature will itself be composed of multiple use cases. Each use case is a sequential flow of steps, operations we want to perform on some user data. This can be data representing any part of the solution, like text on the page, the page itself, the file etc. It is quite generic, and the idea that features are compositions of use cases is universal. Each part of the system, zooming into the details and zooming out to view the big picture, is a composition of basic operations inputting data into a function which manipulates it, and outputting a new form of data.

All programs are exercises in data manipulation. It is always true, that you can produce an exhaustive representation of a feature by breaking it down into these flows, which are frequently also called use cases. Taking the above mentioned example of a notepad app, how would we break the feature down? We can specify one positive flow - "the user will be able to create a text note and save it to a file by clicking ..." - which will encompass the core idea behind the app. We can then take a couple of negative flows - "if the specified drive doesn't have any storage space available, an error will be presented..." - which will capture the idea of the application failing, while attempting to fulfill the positive flow.

This way we can create a specification of multiple use cases for each feature, say 10 each. The benefit of doing things this way is that it allows the testing scenarios be tied to the individual flows of data within our application. These flows are very nicely represented with data flow diagrams, or sequence diagrams.

I always recommend my teams produce such sequence diagrams for each use case, all positive and negative flows representing the totality of functionality required for certain features. Each use case can have tests associated with it, proving the flow is executing as intended.

It is also guidance for my teams to always keep track of the versions of each use case. New use cases might be modification of an existing use case, which means implementing the new one might be much easier than doing it from scratch. Tools like PlantUML which allow you to represent sequence diagrams in plain text format, and adding them to your version control systems next to your code, makes this exercise of versioning use cases a pleasure.

It takes a certain amount of discipline to maintain this representation in the backlog, of features composed of multiple use cases. This is because it is very easy to fall into a trap of adding into your work backlog items which are not themselves representing something valuable to the customer, but are just things you want to remember e.g. you want to remember that this or that feature needs to go through a review process, so you add to your backlog an item called "review". An item like that does not provide any value to the customer and is not tied to any feature we would like to deliver. It is just something the development team wants to remember.

Architectural items

Here we reach the third and final level of analysis and breakdown, which is the architectural work items. These items concern the structure of our application. They are not the things the customer is paying for directly. They are the things the customer doesn't and shouldn't need to know about. They are all the steps we need to perform to deliver a working use case, which passes our tests.

This is the level at which lives the definition of done, the architectural considerations, the SOLID principles, the tech stack, technical excellence, modularity and design of your application. And most importantly, these items SHOULD NOT be visible in the backlog, in the same way as the items above. Because they are not tied to business value, it is very difficult to understand and measure when they are "done", when they fulfil requirements. The design can evolve and change while we are working on the application.

Each of these items constitutes at most a percentage completion of the use case, but until the use case is fulfilled in its entirety and tested, the progress is always 0%. Only when the use case is tested, implying all the necessary architectural work for it is completed, can we say it is done. This is a binary question.

Many project owners and agile coaches view these architectural items should be excluded from the backlog entirely. My recommendation is they become a set of tasks tied to the use cases, invisible from the top, but visible from the bottom.

Once again, the levels are:

1. feature level - a level from which a PO would look at the project,
2. use case / sequence level - here both the team and the PO meet in the middle,
3. and architectural / structural level - the perspective of the team, software architect, tech lead.

You can use only 3 items types in your backlog: epics which correspond to the features of the software, user stories which correspond to positive and negative flows represented with sequence diagrams, and tasks which correspond to architectural items, tests, elements of the DoD, and all the work the teams need to do to deliver the user stories.

Some tasks might not be testable or measurable, and that is OK. All user stories must be testable though, because only through the testing do we know when they are done.

Epics compose the product, user stories compose epics and tasks compose user stories. There are no bug items, there are no sub-tasks, there are no issues etc.?

If you uncover a bug, well it better be a bug of one of the sequential flows, otherwise how can you say what is not working? And if it is a bug of a positive or negative flow of some user story, it becomes quite simply a modification of that user story. You can then capture it like a task under the story, or as a new version of the story. The same heuristic applies to any other type of work item you might think of.

The first two levels provide everything you need to successfully manage a backlog for an agile project from a business perspective. The feature level is frequently represented with epics, the use case level is represented with user stories. This is what epics and user stories were always designed to represent in the first place, the exact ideas I shared in this article.

The benefits of such representation are numerous, besides enabling agile and iterative delivery.

• Your progress charts, when tied to epics and user stories, are closely tied to customer value.
• Iterative progress is just the integer count of how many epics, and how many use cases per epic were delivered. Since each was confirmed done by its own tests, we can be sure they are done.
• Epics can be used to manage iterative delivery, because their relationships are clear.
• New features requested by the customer can be easily accommodated and added to the backlog as new epics.
• Additional work discovered through learning is always conforming to being either a new feature, a new sequential flow, or a new architectural item. This makes trivial the handling of unforeseen work.
• Bugs are tied to the specific feature as a modification of an existing use case, or a creation of a new use case. Which means bugs are just use cases.
• Work becomes easily measurable and predictable.

This is not a complete list of benefits of the mental model expressed in this article, but it gives a flavour of how easy agile management can become.

Give it a try today.

Thanks for reading! If you have questions about any of the ideas I shared and if you would like to receive more content, please ask in the comments, subscribe to the newsletter and follow me on LinkedIn.