When it comes to testing, there are so many areas that it's difficult to feel sure you covered everything. It can be helpful to use a mnemonic or other memory aid to remind you of your main points to hit. Terry Wiegmann recommends adding the acronym NEBS—for Normal, Error, Boundary, and Special test conditions—to your toolkit.
Years ago I attended a presentation by my colleague Al Cline of Carolla Development about unit test coverage, based on his paper “The NEBS Transform.” He discussed how he assured test coverage by using the mnemonic NEBS to remember to consider Normal, Error, Boundary, and Special test conditions.
At the time, I didn’t know much about testing, but his explanations of Normal conditions (what should happen), Error conditions (what could go wrong), Boundary tests (maximum and minimum values), and Special tests (situations that have not been tested before) made sense to me, even if his code samples were beyond my reach. I knew software testing was an area I wanted to explore someday, so I mentally added NEBS to my list of methods that might come in handy in the future.
Test Design Patterns
Over time, as I did analysis work in both plan-driven and agile environments, I noticed two patterns happening in both. One is the tendency to focus on the “happy path” of usage: the explanations that begin with “Usually …” or “Ninety-five percent of the time …”
The other is the opposite: a fascination with the edge cases, exception conditions, and what happens during that remaining 5 percent of the time. While these can be intriguing puzzles to solve, if the guess is right, they really do represent only 5 percent of the actual uses. Worse, they are often the invalid and inappropriate uses. Don’t get me wrong, defining error messages is important, but too much time spent on “abuse cases” can slow the team down and detract from progress. It’s all too easy to spend an inordinate amount of effort designing a solution to a scenario that is very unlikely to occur, represents low risk, or would have a low impact.
However, there usually are a few typical alternate paths or usage scenarios that fall somewhere between the one happy path and the once-in-a-blue-moon occurrence that warranted exploring.
On one team, as we talked with the product owners or champions about various ways to use the software, we realized that we tended to focus on understanding what the product owner wanted to happen and sometimes neglected to ask about other cases, such as what was not supposed to happen or what should happen at the boundaries.
Finally, a Chance to Use NEBS
I thought this might be a great opportunity to revisit NEBS and use it to ensure we didn’t focus only on the normal or “happy path.”
We weren’t likely to forget to ask about Normal conditions, so we decided to repurpose the N in NEBS to remind us to ask about negative conditions while we were discussing a story and thinking about tests and test data that would be needed. Echoing titles of The Big Bang Theory episodes, we called this the “negation alteration”!
The product we were developing was a set of educational materials that would be used by teachers and students. One scenario in which the negative case became apparent was that when a teacher graded an assignment submitted by a student, no other student should be able to see the submitting student’s grade.
In some lessons, students were asked to answer a question using ten to twenty-five words. The E in NEBS reminded us to ask what should happen when the student didn’t answer the question and attempted to proceed through the lesson. Could they progress to other questions and return to this one? Could they exit the lesson and return later? If they did, what data would be saved?
Similarly, the B in NEBS reminded us to ask what the client wanted to happen at the boundaries—when the student entered nine or twenty-six words.
Not all stories or use cases will have Special conditions, but we found one that the S in NEBS applied to: While products typically spanned a school year, we learned that one course was semester-based. This different time span had an impact on course design and licensing, so we added the topic of “semesters” to our list of questions to ask the product owner when discussing other stories. Another example is substitute teachers. Using NEBS helped us understand special conditions that existed when the user was not a permanent teacher.
Remembering a Helpful Testing Mnemonic
I have used NEBS on subsequent projects and find it very handy for exploring the domain and understanding feature requests and desired behaviors. It is a useful complement to other analysis and testing heuristics and mnemonics available, whether you are investigating usage scenarios, generating test ideas, or planning a testing effort.
In the example above, our team was composed of members working together for the first time; all of us were new to the client and domain, as well. To aid us in ramping up quickly while absorbing as much as we could from our very experienced subject matter experts, we pooled our continuous learning and created a list of domain-specific questions to ask our SMEs as we considered users, features, and stories.
Some time later, I was sharing the NEBS model with a colleague on another team, and he immediately said, “Oh, NEBS—the Northeast Barbecue Society!” So now I have a double mnemonic: one to remember the conditions to make sure we cover, and a memorable way to recall the mnemonic!
And that is why the Northeast Barbecue Society helps me ensure I consider Negative, Error, Boundary, and, when applicable, Special conditions as I am planning my tests. If you worry about having adequate test coverage, use whatever convenient tricks you can to help yourself remember some pointers.
Now, if only I could help the Northeast Barbecue Society test their product!
Do you have any useful mnemonics or tricks you employ when you test? Please share them in the comments below.
