Software Testing in Twos

Since I first learned about pair programming in 2000, I’ve had good experiences pairing with both programmers and other testers to do all types and levels of testing. In spite of knowing firsthand the benefits of pairing, I work by myself too much. Our team is small, and it’s easy to get into that frantic “I can get this done faster by myself” mode. In the past several months, though, I’ve had a good reason to pair, which refreshed me on the power of pair testing.

Last fall, we hired a new tester, Michael. Everyone on our team, including developers, system administrators, and DBAs, paired with Michael to help him get up to speed on the application. Pairing is a good way to help a new team member learn the team’s infrastructure around continuous integration, source code control, documentation, data model, and testing frameworks and drivers.

Michael has many years of experience in development and testing but had not worked on an agile team before, and our financial services domain was new to him. Undaunted, he dove into the most difficult areas right away, picking up testing tasks for tough user stories. Michael and I had worked together at a different company in the late ‘90s, so I already knew his strengths. Where I’m sometimes short on diligence, Michael is always meticulous. I often do a lot of generalized hand waving when I explain things, and he insists on lots of detail. This is good for me.

Keeping Each Other Honest
Here’s an example. Michael and I paired to test some enhancements to software that automatically fills in government forms to report details about 401(k) retirement plans. One of the metrics is the number of participants in the plan. The algorithm to calculate that number is surprisingly complex and difficult to understand. If I got close to it in my testing, I called it a win. But when Michael and I came up with a number that was one off what the application produced, he wanted to dig deeper.

After more detective work, we discovered that a bug had indeed been introduced. In the process, I learned a lot more about that particular algorithm. Similarly, on my own, I would have blown off a small difference on a figure for administrative expenses for the plan. Michael wanted to understand why the numbers were different. We dug some more, consulted with one of the developers, and again, we verified that there was a subtle defect in the code. That logic had scared me (as well as the programmers!) for years, and now I’m relieved to understand it fully. We can now write better regression tests and documentation for that functionality.

Pairing helped my discipline in other ways. For years, I’d saved SQL queries that I used when testing different parts of the application but stored them rather haphazardly on a shared drive. Michael had trouble finding the queries he needed and insisted that I put specific queries for specific areas of functionality on the appropriate wiki page. Having done so, I can see that this will also save me time in the future.

New Perspectives
I’ve been testing our application since 2003. Sometimes, when you look at the same stuff for that long, you stop really seeing it. While we paired, Michael noticed subtle issues that my familiarity with the UI caused me to miss. He was also less tolerant of system behaviors that the rest of us had long ignored, such as misleading error messages in parts of the application that were used only by a few internal employees. These turned out to be easily rectified, and why not improve usability where you can?

As Michael learns our product, we’ll lose the advantage of his fresh set of eyes. But by pairing, our differing viewpoints will help us retain a fresh perspective. In my experience, you get the most value when you pair with someone whose testing approach and personality traits differ from your own. Each of you will see a different side to the software and push in different directions.

When I pair with another tester, programmer, or business expert, I’m less likely to be distracted. This focus extends to updating the wiki immediately or writing a new automated test case to document the discovery we just made. With two of us bouncing ideas off each other and analyzing results, we’re more likely to pursue that funny code “smell,” building on each other’s observations and ideas.Shared Understanding, Better ROI
My team has found that when a tester and programmer pair to write business-facing tests that drive development, we usually find differences in the way each of us has interpreted the customer’s desires for the user story. This leads us to use the “power of three” and get the customer involved in the conversation.

In addition, combining the programmer’s code design expertise with the tester’s knowledge of the right tests to write ensures our automated tests are more efficient, maintainable, and valuable. My fellow testers and I have programming experience, but we aren’t designing production code every day. Over the years, my programmer teammates have grown good testing skills, but they still don’t think of all the good questions to ask. Working together to turn examples into automated tests and overcome obstacles to automation ensures a good long-term return on our significant automation investment.

Courage and Confidence
Testing alone is the path of least resistance. Pairing requires that you get out of your comfort zone. Make the effort. On my own, I can talk myself into saying, “Being $17 off is no big deal; I probably just made a math mistake.” When I paired with Michael, he wanted to know what caused that $17 difference. Pair testing gives me the confidence to explore further and really make sure our application behaves correctly. Pairing on test automation builds my courage to take on difficult automation challenges.

There are so many pluses to pairing, but it’s a more intense experience than working on your own and can be tiring. I still find myself slipping into that lazy or sometimes panicked approach of doing testing activities on my own. Like any habit, pairing takes time and practice. At our daily standup meeting, I look for opportunities to pair. In pair testing, we learn faster and find more issues in a shorter amount of time, so even short time boxes of pairing work well. If you’re not pairing now or you’ve fallen out of the habit, experiment by testing with a teammate for an hour or two. I’d love to hear about your own experiences with pair testing.