A good team likely is trained to consistently report defects as accurately and promptly as possible. This means that over time the bug backlog builds up, and looking for what bugs to fix starts to seem like searching for a needle in a haystack. The best way to keep your tracker under control is to improve the quality curve earlier.
The United States Postal Service has long been the carrier for the majority of US mail. Last year it handled more than 155 billion pieces of mail. A staggering number of valuable papers and packages fail to reach their intended recipients due to improper addressing, violation of postal shipping regulations, or some other reason. In 2011, the most recent year for which figures were available, 53.4 million articles were sent to the Mail Recovery Center in Atlanta, Georgia. The facility eventually was able to deliver 43 percent of that mail, but the rest had to be recycled or auctioned off.
To put it simply: The USPS feels an obligation to its customers to spend a lot of time and energy looking through undeliverable mail, and that time is often invested without return.
For some, this sounds an awful lot like a defect tracking system on a large, mature product. After all, what is a defect tracker but a storehouse of defect or enhancement information that can’t be used at this moment so is archived for future value that we hope will come later? (Notice the word hope.)
What’s Logged in a Defect Item?
A documented defect—what I’ll called a “defect item”—usually contains several pieces of information about the failure. That can include steps to reproduce the defect, screen captures, and recordings. It may include debugging information; some testers track the problem down using the source code library or even lines of code. Severity, which is a human’s appraisal of the bug’s seriousness, and priority—the best-guess appraisal of the relative importance of the defect compared to other bugs—are common things to add. Most bug trackers include a unique identifier and a summary, description, or title that gives a brief synopsis of the issue. (I am using the terms bug and defect interchangeably here.)
Let’s spend a moment talking about what happens to these reports.
A Common Pattern
A good team likely will be trained to consistently report defects as accurately and promptly as possible. This means the list of bugs begins to increase as the project grows under the heat of change. In fact, it is possible that hundreds, perhaps thousands of bugs could be reported in a short amount of time.
On more agile teams, some bugs may be fixed in a matter of minutes, whereas on a more traditional, “structured” team, the testing that leads to the bug reports may lag weeks or even months behind when the defects were introduced—and that’s not counting the time a developer has to wait to begin an investigation. This understandably has an impact on the speed at which teams can deliver, and it increases the amount of time between a defect being created and introduced into a branch of code, its eventual discovery and reporting by a stakeholder, and the duration of time needed to implement a fix and then verify it each time a bug is introduced and discovered. Thus, this process has a compounding effect on the timeline of fixing a product, and for many teams, time is of the essence.
Over time the bug backlog builds up; looking for what bugs to fix starts to seem like searching for a needle in a haystack. The team stops caring about new bugs, because what is five more added to a pile of five thousand?
Then, one day, you realize: Unresolved bugs are like the undeliverable mail of our day—a one-way communication without a recipient.
While the best teams will collaborate and try to fix bugs as soon as they are discovered, even their best efforts may not always keep up with the growing defect count. Features get notice; bugs that were deferred last week are likely to get deferred again. When push comes to shove and the team has to decide on deferring new bugs, what’s ten more added to a pile of ten thousand, anyway? When the technical staff realizes that product management cares about features only, consistently asking if the bug is deferrable, they lose pride in work, wondering, “Why bother to build something good when shoddy makes people happier?”
Then you realize something else: A large defect backlog is demotivating for the whole team, whether they created it or are just hired into it.
Developers and testers alike want to see bugs identified and fixed as quickly as possible, but it should come as no surprise that even the most agile teams may have periods where their focus is on new, billable features, not fixing bugs.
So, what can be done? How can you keep your defect tracker from becoming like the old “dead mail” bin at your local post office? While there are strategies to deal with your bug backlog when it grows to sufficient size, the best method would be to never let the unmitigated bug count grow to such an ungainly mass in the first place.
Work to Improve the Quality Curve Early
You could start by working to improve the quality curve early by introducing techniques like code reviews and pairing programmers with other programmers or testers as part of your definition of done. By working together and discovering defects earlier, you decrease the time between the bug’s introduction and when it is ultimately verified as present in the system. Testing early and often will also allow many of the defects to be fixed before the feature work is done and while the feature is still clearly remembered by the developers, therefore reducing the risk of introducing that bug into production.
On the testing side, it might be prudent to consider what sorts of defects really need to be logged long-term. If a simple conversation with a developer can get a defect solved in a matter of moments, then does it really need to be committed to a database for all to see? Probably not. Now you can begin to rapidly provide information to your teams when it seems to be most important, which will enable them to pivot, act, and increase the understanding of the application. Not all bugs necessarily need to be stored for a long time on the bug bookshelf.
Ultimately, the real goal of testing should not be to build great works of artfully reported bugs and enhancements, or to build towers of many defects found—but rather to communicate, collaborate, and work alongside the development team to provide the best and highest quality experience possible to your users and customers.
What techniques do you know to keep the artifacts in the bug tracker small, tight, and valuable?
