What's In Your Toolbox?

Summary:

There are a lot of tools available to testers. Some good, some not so good. As we use and gain experience with the various test tools, the set of tools in our tool box becomes larger. Having a large set of tools in your toolbox allows you to better adapt to any test situation.

Tools!

When I was in the Air Force fixing airplanes, there was one tool that was sure to be found in everybody's tool box–safety wire pliers! When repairing aircraft, every nut and bolt, every fastener, every connection, had to be wired to a fixed surface to prevent it from vibrating loose during flight. Did we need safety wire pliers? No, not really. But when you’re hanging upside down in the cockpit of an F-4E, from an armed ejection seat, trying to safety wire a component in place, they were nice to have. There were two reasons we all carried safety wire pliers. First, we all had to safety wire stuff and they made the job easy. Second, they were a wonder tool! As close to a single, universal tool that you could find. You could use them to cut and strip wires, break off striped or broken fasteners, use them as rudimentary clamps, etc. Since I’ve been retired from the Air Force, and I am now a dedicated software tester, I've been in search of a similar wonder tool. So far, no luck. Instead, I've found it important to have a well stocked toolbox with a lot of different tools.

Any good craftsman has a lot of tools. If you consider software testing a craft (and I do) then it is important to have a well stocked tool box. I like to think of myself as the Norm Abram of software testin–without the flannel shirt. Tropical shirt? Guilty.

For software testers, our experience with various testing methodologies may be considered tools. Test process methodologies, test development methodologies, defect management methodologies, etc., are all tools. If you are doing any kind of consulting work, it helps to have a pretty big, well stocked tool box. I;m a tool collector.

Will we use every tool on every project? I doubt it. Some tools are better than others. Over time we tend to develop a comfort level, a relationship even, with our favorite ones. Some tools aren’t so good. Since all tools have their good and bad points, a combination of tools usually works best. I have a favorite Test Management tool, Defect Management tool, etc. Some tools work great in very well defined situations, others, not so much. Some tools will work great doing tasks that they were not originally designed to do–like safety wire pliers. Is there one perfect tool? No, I don't think so. The important thing is to have a lot of tools, know the advantages and disadvantages of each, and know when to pull them out. The bigger, and more well stocked your toolbox, the more valuable you become.

One of the most valuable tools you can stock in your toolbox is knowledge of various test processes or methodologies, such as Waterfall, Iterative, Agile, etc., and how to apply each. I have found that one tool alone, is rarely enough to do the job! As a consultant, one of my favorite parts of the consulting process was the initial client meeting. The client would typically espouse how they had implemented the latest test methodology, yada, yada, yada–typically Agile. That was usually the first bubble I burst–sometimes tactfully, sometimes not.

I've seen and used most of the current and some not-so-current test methodologies. In fact, one of my responsibilities when working as a staff consultant was to evaluate and recommend tools, such as test methodologies, for client companies. As a result, I have used or evaluated a wide variety of popular test methodologies. What I’ve found consistently is:

No one uses just one tool or methodology. Most use a combination of tools.
Most formal methodologies are adopted in name only, rarely in practice.

A little investigation typically revealed that most clients really didn't even use their chosen methodology. They tended to use parts of them. Just because you have a daily stand-up meeting, or do frequent, small releases does not necessarily make you an "Agile" shop. No, what I have found is the best methodology is more like a well stocked tool box. You use the tool that works best for the task at hand and adapt as needed. I like to consider them "best practice>'

Unfortunately, this is where I will lose the Agile and Context-driven folks. So let me state right up front–Agile and the Context-driven School are great ideas and I'm a fan of both. They're both in my tool box and I bring them out when appropriate. But just because I tend to rely on them in many situations, doesn't mean I'm throwing away the rest of my tools!

At the risk of offending the Context-Driven school–let's call these tools "best practices." Best-practices, as far as I'm concerned are really nothing more than tools–and they work well in very well-defined situations. I find there is no one "best" practice, but rather a number of them. Each has parts that work great, while others are not so great, depending on the situation. To be successful I typically take a little of this, and a little of that, until I find the combination that works best for the client. So Agile as a whole–may not be the best solution given the current situation–but there are parts of it that are wonderful. I'm going to use them. Waterfall? Maybe not the best solution either, but there are parts that will work really well. Iterative? Also some good parts, we’ll use some of that!

The important thing is that you have a large, well-stocked tool box. Throwing out a name is not good enough. You need to know what tool, or combination of tools, works best in a given situation and know how to use them. How does each tool work? What are the strengths and weaknesses of each? Can I customize the tool, or combine tools to fit the current situation?

Lastly, you need to care for your tools, keep them sharp, keep them current, and always be on the lookout for new and better tools. How do you do that?