Executor or Engineer

Test executors and engineers have distinct places in the IT organization, but test executors who present themselves as engineers have posed a problem for true test engineers for years because they perpetuate the myth that "anyone can be a tester." This weakens the power testers have in an organization with regard to decisions that affect the quality of the IT systems; in addition, the quality of life for test engineers in an IT organization is significantly diminished by this confusion. This has been a private struggle for years, but it must become public because it transcends the boundaries of testing.

This conflict is a problem for IT organizations as a whole, because all too often managers and project leads believe they are hiring test engineers when they are actually acquiring test executors. Relying on an executor to do an engineer's job is detrimental to a project's budget and goals, because the project will experience bloated costs and/or inefficient practices that will lead to a low-quality product. The best approach for ending the duel begins with clearly identifying the points of divergence that cause the dichotomy, then explaining how to discern on which side of the dichotomy a potential testing resource belongs.

The Points of Divergence
"A test engineer minus the requirements analysis equals a test executor."

I recently wrote an article for Better Software magazine called "Your Job - Requirements = Less Value." That article introduced the concept of test executors versus test engineers by disputing the claim that "anybody can be a tester" while simultaneously declaring that "anybody can be a test executor." There are four points of divergence that separate an engineer from an executor:

• Analysis
• Knowledge of testing theory
• Attitude
• Technical understanding

Let's begin our look at these points of divergence by discussing the term "engineering." Engineering is about—the procedure(s) used to make a system or design as effective or functional as possible. A software project has limited time, money, and resources; someone must analyze the time, money, and resources available and figure out how to use them most effectively. Analysis is one point of divergence between engineers and executors. This includes analysis of requirements, processes, systems, and staff.

This analysis combined with a general knowledge of testing theory is used to engineer the most efficient testing approach possible in the existing environment. This is why knowledge of general testing theory is another point of divergence. Have you ever seen a tester take use-case scenarios or high-level testing scenarios provided by the requirements/business analysts and use those solely as the basis of their testing? That exemplifies how a test executor performs his job. On the contrary, test engineers analyze requirements to determine the completeness, correctness, consistency, and clarity of requirements. They work to get requirements improved as much as possible, and then they use knowledge of testing techniques (boundary analysis, exploratory testing, etc.) to go beyond what is stated in the requirements to further exercise the system.

Another point of divergence is attitude. Test executors are experts on what cannot be done, while test engineers are experts on what it will take to get something done. When faced with challenges, such as poor organizational processes or low resource availability, test executors excel in explaining why goals can't be accomplished. Test engineers understand, however, that conditions are never perfect, so they look at such challenges as opportunities to earn their paychecks. They begin to craft a set of scenarios that outline how various levels of a goal can be accomplished under the current circumstances.

Being technical does not mean being an expert programmer; it means understanding systems beyond what is seen on the surface so you can better interact with them. It also means having the ability to produce quality solutions with computer science concepts. Test executors shy away from these concepts and argue that they are outside of the scope of a tester's job function. A test engineer is excited over the prospect of streamlining processes with technical concepts, such as scripting language automation and test tool customization.

Differentiating Engineers and Executors
When evaluating resources, whether during an interview or on the job, there are certain things to look for that indicate on which side of the dichotomy the resource falls. The first is whether or not the resource is capable of relaying a basic understanding of testing concepts and terminology. You can always look at someone's resume and find strategically placed testing buzzwords, but be wary of people who can't speak concerning the information in their own resumes. I can't tell you how many interviews I've conducted where interviewees have drawn a blank when asked to describe what's included in a typical test case.

In addition to being able to speak on basic testing concepts, the applicant must be able to speak about these concepts using previous experiences as support rather than as definitions. Many people mistake being a subject matter expert for being a test engineer. When asked about the basics of testing, they talk endlessly about functionality of applications they've worked on without addressing testing's body of knowledge at all. Without an understanding of testing's basic concepts, it doesn't matter how important he was in his last position because the skills won't be transferable. True test engineers can effectively discuss basic testing terms and concepts using personal experiences as support. Key concepts include:

• Test cases
• Test planning
• Defect analysis
• Requirements analysis
• Requirements coverage
• Test reporting
• Test process improvement

The ability to develop innovative ideas is another way to differentiate between executors and engineers. This doesn't have to be anything major, and it doesn't even have to be specific to testing. All it has to be is something that provides a glimmer of hope that the individual may be able to engineer solutions. In an interview, I like to ask, "Can you tell me about an accomplishment that you're proud of that required you to be creative?" His answer tells more about him than other conventional questions about testing theory. This allows the true test engineer to shine, particularly if he doesn't have a lot of testing experience. A good response describes a situation in which he noticed the existence of a tedious process and took the initiative to create something that made the process more efficient. A test executor will almost always sink on this type of question.

Finally, if you already have a testing professional who consistently comes up with excuses instead of solutions, then he may be a test executor.

Conclusion
Exposing the testing dichotomy in this column is not meant to suggest that there is only room for one of the two groups. There is room for both test engineers and test executors in software development projects, but the two can no longer be thought of synonymously. For the health of IT organizations and the growth of the testing profession, it must be made clear that test engineers offer a level of expertise that can never be expected from test executors.