How Do You Think?

When I am staffing a test team, I look for testers with genuine skills—both testing skills and soft skills. Neither of these are straightforward or easy to find or evaluate, especially within the time constraints of an interview.

Beyond skills, I look for people with the right qualities of mind. For me, these include:

• Real enthusiasm for testing
• Judgment
• Analytical ability
• Creativity and problem-solving skills
• Critical intelligence: curiosity and skepticism

Paper qualifications—whether tester certification, post-secondary education, or testing courses—don't tell me anything about a candidate's possession of these qualities. I could learn something by asking the person to solve a puzzle, or to demonstrate how she would test a given piece of software, but I'm wary of a "one-size-fits-all" approach to evaluating people. We all need to be aware that different kinds of intelligence manifest in different ways. So I try to engage candidates in conversations that make them think. I work to move people away from their prepared answers to a conversational territory where they will reveal not only how they think, but also the qualities they truly value in doing the tester's job.

Gauge a Candidate's Feelings
I start with a couple of simple questions: Do you like testing? What do you like about it? Every candidate will claim to like testing, but the answers tell me whether or not it's worth going on with an interview. Some people light up when I ask those questions. Others don't show any spark at all. There is no one right answer. A passionate concern for users is valuable, even more so when it is coupled with glee in finding bugs and pursuing difficult clues. Good candidates display enthusiasm, searching curiosity, and a sense of joy Some people don't even seem to like software, let alone testing.

I also look for judgment in evaluating risk. I want testers who care about more than whether the software matches the written requirements. A tester needs to know that some requirements might matter more than others, and that half of the real requirements might never have been written down. If a candidate talks about risk-based testing, then she'd better go beyond the buzzwords and demonstrate a real understanding of what it is and how to accomplish it.

Delve into Work and School Experiences
Evaluating analytical skills or problem-solving abilities is not easy. I try to do this by asking testers how they approach problems. I pose a hypothetical problem and ask the candidate to suggest ways to solve it. I also ask for a specific example of a difficult testing problem the candidate has solved and the method she used. Again, there are no right answers. I'm looking for a sense of method, but also for creativity and the flexibility to adapt an approach to different circumstances. I'm especially interested in people who know what a model is, and why it might be a useful tool in analyzing and solving problems. I'm even more interested in candidates who understand that models have limitations.

Candidates must also possess critical intelligence. One indicator of critical intelligence is a candidate's description and assessment of the various sources of project information. I ask interviewees to give examples of the sources they've dealt with and others they know of. What issues did they encounter in getting and evaluating information? Were all sources equally credible? If not, what did they do about that? How might they cross-check different sources? Finally, knowing what a candidate has studied or done in a former life can help guide interview questions that show her qualities of mind.Take formal education. Academic studies at any advanced level require problem solving, at least in principle. I expect people with academic credentials to be able to talk about their studies, not only in terms of how the work might have prepared them to test software but also in terms that show me how—or if—they think and how they generally approach problems. Their answers will also tell me the qualities and skills they believe to be valuable for a tester. A good tester should be able to provide an interesting analysis, even if it has never occurred to her to think this way before.

If someone were to ask me how my university studies in English language and literature helped equip me to be a tester, I would say that it's the passionate study of literature that's of interest, and the development of skills to pursue that study. A poem or a work of fiction is both a system and a model, just like a software system. Natural language is a symbolic system, used within various formal constraints to construct literature. Studying both language and literature require analytical thinking and conceptual modeling. You need the ability to dig out sources of information and think critically about their reliability, and to construct and test hypotheses. You must have the imaginative reach to get inside someone else's view of reality and also to understand the historical context of a different period.

I expect candidates to be able to talk about the testing they've done in a similar way. Whatever we talk about, I want to see a zest for thinking and problem solving. I want candidates to display the ability to know what a metaphor is and why it might be useful and important to a tester. I want to see tenacity and healthy skepticism in that person as well. The question that drives the entire interview is "How do you think?" It's the conversation that really makes the difference. Because I work to get candidates to think and to talk enthusiastically, it's hard for them not to reveal what I want to know. All of this makes for an enjoyable interview process. And when I do finally meet the successful candidate, the interview helps establish a foundation of trust and understanding for the future working relationship.