What Should We Be Using for Sprint Planning; A Statistical Look, Sort Of

I was going to call this "An Early Statistical Analysis of Velocity and Story Count in Relation to The Amount of Unfinished Work At The End of a Sprint" to almost sound like I am writing some sort of academic paper to be written in a journal somewhere but it is so not that so I came down to earth and rewrote the title and I am starting the article mocking myself about it.

I have been a Scrum Master for over 15 years now. I have a whole host of certification initials I could put after my name if I wanted to. But I am one who is a firm believer that Agile, Scum, Kanban, and all the rest are just frameworks and mindsets to work and live by not a bunch of rules to live by. With that I have been the Scrum Master for a variety of teams, mature teams, new team, development teams, CI CD teams. Teams that were standalone teams, teams that are apart of larger trains, scrum teams, scrumban teams, kanban team etc. etc. In short, my experience is long and varied, good and bad.

I really like the general idea of a Sprint “commitment”. Have a plan, have a goal, let’s see how close we can get to hitting it, what worked, what didn’t, next. Not the level of commitment where the team does not complete two Stories and we need to make a huge thing about it. Fill out reports, report up to the train what happened and what we are planning to do about it. That is not Agile. Sure, let’s talk about it on the retro, what happened, but I know scrum masters where this becomes a whole retro just talking about this and nothing else. Which is way too much. But with that one of the hardest parts of deciding on a commitment is how much work can the team get done. Can we reach that goal, or can we only get to this goal?

I have never used capacity for this, ever. By capacity I mean you task out the work and then put estimated hours into each task and you then work out how many hours the team has available and there you have capacity. The problem with this is the whole reason we went away from estimating hours on Stories is because humans stink at estimating hours. With that we then we went from estimating hours on Stories to estimating hours on Task like we would stink at that less. What kind of logic is that? Capacity is just the definition of waste and can add up to a chunk of wasted time and effort that is lost value delivery time.

I have almost always used velocity. Using a points scale based upon the finocchi sequence. I do think this is an effective system where there are clear steps in size and demarcations between sizes, but they are not too broad either. I know some people use t-shirt sizes for Stories, but I think they are too broad and vague. They work for Feature sizing but not so much for Story sizing. Velocity works well for me. But it too can take a decent amount of time, voting, okay not everyone voted the same so now we need to talk about it and work on consensus. I do not think this is waste because there is value and generally good conversation out of it, but it is still potential lost time delivering value by the team.

I was working with a mature team for a few years, and I was noticing a trend. With that maturity they were sizing almost all their stories the same size (just happened to be a 3). If the Story was bigger than that size, they had established a natural tendency to break down the story down before they even discussed sizing. Meaning they did not size a Story a 5 and then said OKAY we need to split it, they split it before they even got to the point of sizing it. ?

Now I am always looking for ways to make my teams more efficient, every little bit counts and that is especially true for time in meetings like grooming. Grooming is vital and very important and should never be eliminated. But if you can groom for forty-five minutes instead of an hour then that is more value the team can deliver. The thought had crossed my mind with this trend I was seeing, should we even be pointing? But it was hard for me to break away from the pointing as it was so intrenched in my mind. But then I heard that a fellow scrum master who was planning just based upon Story count. Instead of “the team is averaging x points so plan x points”, it is just “the team is averaging getting x Stories completed so plan x Stories”.

I did it. Just with this one team (of my three). I switched to just Story count which greatly ruffled the feather of our very old school Release Train Engineer at the time as well as an Agile coach we have. But I promised if the team started to carry over more work we would immediately switch back and that I think the team would get more work done this way. They did, not leaps and bounds, but they stopped pointing we just worked of Story count. The level of Stories getting carried over really did not change at all and the team appeared to be getting more work done and was certainly spending less time grooming but still had well-groomed Stories.

This lasted for about year when it was decided by management to reshuffle the whole release train. All the teams were blown up and reorganized. Essentially, I was going to be starting with three new teams with a mix of people with varying knowledge and experience working together. I was not comfortable starting straight with Story count and not pointing, so I went back to velocity with all three new teams.

At this point I need to do a little aside here. In my long work history, I spent a chunk of time working in process improvement projects where I learned a lot of data analysis skills and it stuck. I really enjoy working with data and trying to glean insights from it.?To which I am working on teaching myself Python, Pandas, and more statistics with zero expectation of ever getting paid for it. I am doing it just because I think it is fun (yes, I know I am weird). Taking those two pieces of my life and putting them together I have been collecting data for my three new teams to analyze the relationship between, Story Points in a Sprint, Story Count in a Sprint, and the number of Stories the Team does not finish in the Sprint.

Some data set qualifiers:

·????????I am far from a statistics expert and a true data analyst (see my conclusion).

·????????The data set currently is very small; 3 teams, 7 Sprints each.

·????????Those 7 Sprints do not include the team’s first Sprint as well as the one IP Sprint they have gone through as they would be large outliners in the data set. Note it is 7 data points with these two points excluded, not now down to 5 data points each.

·????????As these teams mature, I expect there to be less variation sprint to sprint like there is now (I will talk more about this a little later).

·????????At this time since they are new teams, they are working a similar process as I am their scrum master which will probably diverge more over time as they make the flow better for each individual team.

With all that said, with the limited data I have right now the findings are, well, interesting. At this moment the data is not showing a strong correlation between either velocity or story count with the number of unfinished stories in a Sprint. Which honestly makes NO SENSE to me, okay maybe it would not be a strong correlation but right now I am seeing almost none. But I am sharing what I am seeing and will continue to share as the data grows and any new patterns emerge and as my knowledge and education grows (and I discover I am doing something wrong.

Let’s look at numbers but let me explain them a little bit first, if you know what a correlation number is please skip down past this paragraph. This initial analysis uses a statistical number called correlation. 0.000000 correlation means there is absolutely no correlation between the two data points. You will never see this number be 0.000000. Statistics is never exact, it is not math, it is statistics, it is about what the chances are of a number being right or wrong, nothing is ever explicitly right or wrong. ?Even if you took something like the temperature of your living room over time and compare it to the number of gazelles on the African plains over time, the correlation may come out as something like 0.000001 but it will never be all zeros. A correlation of 1.000000 means there is a perfect correlation between the two data points. Again, you will never see this, well you will when you do something like comparing the temperature of your living room to the temperature of your living room, but even if you did the temperature of your living room to the temperature of your bedroom it will not be 1.000000. This number can be positive or negative, so a 1.000000 means there is perfect positive correlation between the two points, as x goes up, y goes up at an exactly predicable amount. A -1.000000 means there is a perfectly negative correlation, so as x goes up then the y goes down at a perfectly predicable amount. Again, you will never get perfect positive or negative ones, this is just an easy example.

What does my data look like?

You can see that for team B the Velocity and Team C the Story Count correlations to unfinished stories are a little up there but not as strong as I think anyone would really assume they would be if you asked.

One thing I touched on earlier is that being new teams there is a lot more variation than a mature team would have. If you look at this data in a scatter plot some of it looks like it is a bit stronger than the correlation number implies. ?

Below is the chart of the velocity to unfinished stories for Team A. The closer the points are to the line the more predicable the data is, and you see that one significant outlier where the points were down, but the unfinished stories are higher than average. But if I remove that data point the correlation number barely moves.?

At this point in time my only conclusion is “I have to make sure I am not messing this all up because this makes no sense”. Which is partly why I am writing this, and I am posting it because someone may be able to point out a huge flaw in what I am doing here. Which is okay, if I messed up, please tell me I messed up. Because this is not at all what I expected. My second conclusion is I need a lot more data points to really see what is going on and how this holds up.

Please feel free to give feedback positive or negative, ask questions, whatever you got, I will collect some more data and maybe in a few more months I will post where this goes.?

#agile #scrum #scrummaster #dataanalytics #statistics