How keeping track of your development bugs is killing your Project and your Team
It’s becoming more frequent having organisations demanding from their development teams a detailed track record of the bugs discovered…during development! And by development, I’m referring to the cycle code-test-code-retest. Measuring team’s progress, developers’ quality, etc., are some of the reasons supporting this kind of request. Unfortunately my experience has shown me in 100% of the situations that:
- This approach brought a significant overhead to the team;
- Data collected doesn’t tell a thing about real team performance;
- Atmosphere and team spirit suffered…a lot!
Maybe this sounds a bit strange, but take a closer look to what kind of problems this approach can lead you into and in which way they can seriously jeopardise your project:
1. You cannot measure developers skills based on the number of bugs during development – There are innumerous factors that can lead a developer introducing more bugs than usual during development. For instance, trying new things, refactoring, coding more complex features, just to name a few. This is because it’s precisely during coding that developers can create bugs. A developer who doesn’t introduce bugs is probably because is not trying new things nor the best way to develop things. Remember that making things simple is actually one of the hardest skills.
The REAL important thing here is that the team implements mechanisms that allow identifying the bugs BEFORE being integrated in the delivery. Unit testing, functional testing, automated testing, just to name a few.
2. The use of an Issue Tracker to manage bugs - An Agile project should have a team that works as a whole. Constant face to face communication, built-in trust, cross-functional profiles are some of the key principles that allow teams to perform at a high level.
If a team member finds a bug inside a functionality that is being developed but it doesn’t yet comply with the definition of DONE (for that user story), he/she should not register that error in an issue tracker tool! This is because it will bring an overhead to both of them who will have to manage that issue in the tool (we all know the several workflow steps that are usually implemented in order to “treat” an issue inside these tools!!). If the issue is a simple one and it can be solved immediately, you don’t even need to register it. It’s just all about solve it! On the other hand, if it’s something that can take a few hours (or a bit more), then it can be added as a task under that specific user story and then communicated to the team in the next daily stand-up meeting.
3. Afraid to make mistakes - One of the principles engraved in the agile manifesto foresees motivated individuals and that you should support them and trust that they will get the job done. Period.
If you create a mechanism where the goal is to produce statistics on the mistakes made by developers, even BEFORE the functionality is delivered to the costumer, you’re basically saying that they have no margin for errors! This will obviously lead to fear. Fear of the consequences for failing, fear of the exposure and fear that no one will no longer trust the person. With this kind of atmosphere, developers will not unleash their creativity and will not try to find the best options to solve the problems. They will always go for the path they know they can do, even if it’s not the best one. Now imagine this applied to all team developers. Soon you will no longer have a team. But wait! You might be thinking that you still can live with this and still successfully complete the project without any major issues. Well, just keep reading…
4. Afraid of not reporting errors - If with a system like this, a developer is afraid of making mistakes, for sure the tester will have his legs shacking just for thinking about the possibility of not finding and reporting them! You saw that coming, right? This will mean that the tester, instead of trying to discuss and solve the bugs directly with the programmer, he will report EVERYTHING that he finds, no matter how small and simple it may be! This obviously will lead to a significant workload increase! But even worse than that is the natural clash that is going to happen between programmers and testers, since it’s just a matter of time until they enter in “guerrilla”, playing mouse and cat game. Programmers arguing that it’s not a bug to be registered and the testers mentioning the opposite, since everything is to be registered! I don’t think I need to mention what’s going to happen to your team’s cooperative and “we’re all in this together” spirit.
So, the next time you think about evaluating team’s quality and performance using data from bugs created and found during development, please make sure you take this into account! If you really what to understand team’s quality and performance levels, pay attention to things like a proper implementation of unit, system and acceptance tests platform, including automating mechanisms. You can also check if the team is following a proper continuous integration strategy. There are dozens of other examples that can tell you a lot about your team’s quality, including each team member. But whatever you do, just don’t chase them because they are doing their job.
QA Test Analyst, Quality Controller, EU citizen
7 年I stumbled upon this interesting little article, thank you for it. Good points, I agree on each one on the surface but the real focus i think is timing: exactly what you mention at the end of point3. Also a good tester will use their experience and interpersonal skills to not allow the negative spiral of point4 to get too far...simple defect Severity/Priority classifications are important and there for a reason. Great final paragraph, the ideal world eh!
Program Management, Project management & QA Roles Software delivery using either Waterfall, V-Model or Agile methodologies
8 年I have long used a risk assessments to predict where defects are likely to be. But unit tests and acceptance tests are design to capture different defects so I can't see how you can correlate the two.
Chief Product Officer
8 年It is interesting how emotional this debate gets when you have the development and testing disciplines represented in a meeting when you are trying to agree ways to demonstrate that you are trying to demonstrate the shifting of quality to the left in the cycle and using defect removal rates to prove it. The testing fraternity believe they can use defect rates in development as a predictor for the defects they are going to find in Acceptance testing which of course is complete nonsense (as this article demonstrates). However, the impasse I am currently at with our methodology is how do you determine how good the work packages that are coming from development are so that you can predict how many cycles will be needed in UAT. Equally, how do you measure defect removal rates in development? The only answer I have managed to come up with is to introduce independent unit testing. Basically have a test between the point when a developer says a use story is done and the entry into QA. Any defects found in that test should then be tracked. Without this, it is arguable that no defects are removed in CUT, just introduced.