When is a user story not a user story? Acceptance Criteria and the Definition of Done

While sitting through a refinement an unusual user story popped up in the planning poker window. As the PO continued to describe the new product behavior, the team stated this story was developed 2 sprints ago, but the specific scenario wasn’t tested.  

My agile radars were on high - if we could have tested something earlier why didn’t we? What level of testing should we include in the definition of done? Is there a user story out there marked done but not shippable done “done.” 

Will this behavior of “testing only” stories decrease focus on making user stories shippable, dis-empower the team to test only what is documented and will it decrease throughput and reduce quality?

Let's look at some of the reasons why this matters.

Acceptance Criteria & Team Empowerment

I have a hard time arguing against verifying we did not break existing functionality as part of acceptance criteria. The more we can test now the better we are at improving feedback loops and empowering the team to look beyond the requirements. 

Let’s look at a sample user story that includes verifying existing features did not break:

User story: I need a way to download latest content.

Acceptance Criteria

• Make sure this works with a copy file.
• Make sure this works with an archived file.
• Verify content is accurate. 

Calling out what needs to be verified to make this story shippable done will surely increase the size of the story. The gain is early detection of bugs and delivering “shippable” user stories. However, it is important to note, identifying areas of impact for a user story isn’t a PO responsibility alone. The team shouldn’t be limited by acceptance criteria, instead needs to be guided by it to formulate test strategies and start the conversation. Everyone including the PO and SCRUM team is responsible to identify gaps, have the conversation and modify acceptance criteria to improve delivery of the overall product. That is the part of what makes SCRUM agile and in doing so will lead to better working software over process and better human interaction over being limited by documentation.

Team focus and the Definition of Done

There is importance to defining what done will mean for your team. If done includes testing, its important to include what level of testing will be necessary for your sprints completion. How you define user story doneness will impact how early the team will have design discussions, how deeply to define test strategies, and how focused on completion they should be before moving on to new features. Being careful to craft a definition of done to include not breaking existing functionality will encourage the team to include this as part of their early design decisions, a win for the team, the PO, and the end customer.