In his April 2005 column, "After the Bug Report," Danny R. Faught suggested that when you're testing a bug fix, you should also look for additional bugs. This week, he expands on that idea, showing you how one bug report can multiply into many more bugs.
As a tester, you know you're doing your job when you see a bug fixed. You can do an even better job when you use that bug as a guide to show you where more bugs are lurking in your product. My personal goal is to open at least one more bug as a result of checking the fix for a bug that was reported earlier.
Tips
Here are some specific ways you can use one fixed bug as inspiration for finding more bugs in a product:
- Look around the same area where the first bug was found. You already know to check that the bug fix works and didn't introduce new problems. Also look for latent bugs that may have been masked by the first bug.
- Look for the same bug on another platform. Your product might be ported to more than one operating system, or there might be overlapping features in a natively compiled application and a Web application. See if you can find a similar bug in places where the same feature is implemented.
- Look for a similar feature in other areas of the product, to see if a similar bug afflicts these areas. You may find that a user interface change that fixed one bug should be propagated to other areas in order to maintain consistency.
- If the original bug involves a corner case, push farther into the corner. If you make your test data more stressful or unusual than before, you may find additional bugs.
- Check the documentation. The bug fix might have changed user-visible behavior, and the user documentation might need to be updated to reflect this. If you've shed light on a limitation or subtle detail that wasn't documented before, consider whether it would help the user to mention this in the documentation.
- Look for bugs that have nothing to do with the bug fix you're testing. If you watch carefully, you will notice other bugs along the way that you weren't even trying to find.
I find it helpful to keep the fixed bug open and in my queue while I'm still brainstorming for additional bugs to file. If I become distracted by another task, lunch, etc., I have this reminder still in the bug database, which is especially important when I'm trying to juggle many tasks at once. If your organization requires you to close bug reports as quickly as possible, you may need to keep a separate log of notes on test ideas instead.
How It Fits In
When I use a fixed bug to generate ideas for finding additional bugs, I often find a flurry of newly discovered problems that I need to report. If I don't find any additional bugs, and I actually allow the count of open bugs to drop, I feel like I'm missing something. If you get that feeling too, then you've successfully adopted the habit of making bugs multiply.
I usually end up on projects where I'm testing a product with many bugs. If you're testing a mature product, you may not find as many additional bugs. But you won't know until you look.
If you're required to follow a highly structured process, you might have difficulty finding sufficient time to explore the additional areas that I'm suggesting in addition to your scheduled duties. In that case, I recommend taking a minute or two to see if you find any new symptoms. If you do, tell your manager your concern about the problems you find, and ask to schedule some time to explore those areas. You may have uncovered an area of the application that's under tested. If you've wandered into an area that someone else is responsible for testing, share your findings with him, and take advantage of his specialized knowledge about the best way to address problems in that area.
You may be wondering why I didn't suggest using this checklist right after you've filed a bug report rather than waiting until the bug is fixed. Some of these tips may help you further isolate a bug you've found or to find more bugs based on a bug you just reported. However, I suggest using the checklist after a bug is fixed because a design decision has been made at that point. Once you know how one bug has been fixed, you have valuable information about the scope of the fix that is hard to predict when you first report a bug. Once you know the boundaries of where the fix has been applied, you're ready to explore those boundaries with additional testing ideas.
These ideas have helped me to submit a relentless onslaught of bug reports. The developers' best hope is that the severity of the bugs is decreasing on average as the number of open bugs continues to rise. I hope you can use some of these ideas to make your own bug reports multiply.
