Does your team need an application lifecycle management (ALM) tool? What is an ALM tool, anyway? Since this term was coined a few years ago, Danny Faught hasn't quite gotten comfortable with it. ALM covers a broad set of possible tools. In this week's column, Danny maps out some of the various things that tools actually can do to help you. Once you understand which components you'd like to use in your work, then you can start to evaluate which bundles of features are a good fit for you.
There is a plethora of tools available to the testing professional, but before you buy a tool or a bundle of tools, consider what the tool is designed to do and whether you actually need a tool to do that.
Popular Tools
Probably the most common tool used by test teams is the bug tracking tool. This is also one of the tool categories in which you have the most choices, both commercial and open source. A few organizations still get by with a simpler solution like a spreadsheet or box of note cards.
The tool that most organizations seem to want is the test driver. A test driver is able to manipulate a user interface (like a GUI, Web browser, or command line) or a programming interface in order to automate your test execution. This is the crucial part of both functional and load test tools. Test drivers target the end user interface, a subsystem, or individual units of code. They can be used for straightforward functional tests or for load tests that involve simulating many users in parallel, though few tools do both functional and load testing well. You can also think of a test driver as the replay portion of a capture/replay tool, which is often paired with a capture tool to assist in creating tests by recording an interaction with the system under test.
For functional testing, a test driver goes hand in hand with a test case management tool, which helps to organize a repository of test cases and records the results when you run the tests. A test case management tool may handle manual test cases, which is useful if you have a large number of scripted manual tests. Some only manage automated tests, while others manage automated and manual tests. You also may run across test frameworks that provide some sort of infrastructure that makes it easier to maintain test cases and support more than one product or platform. Many test frameworks focus on providing a keyword language to simplify the process of writing test cases.
A few specialized tool bundles help with exploratory testing, which is quite a different activity than running scripted manual tests.
Beyond Automation
Beyond this, there are many different kinds of tools that you can use while running tests. A screen capture tool captures images or video of the details of your test results, which can be useful evidence. Monitors tell you about the health of the system and the utilization of resources, which is an important part of a load test, and can also point out functional problems that would otherwise be invisible. A comparator compares actual results to expected results to help determine whether a test passes. A few specialized tools compare the state of the system before and after you install the software, so you can scrutinize all the changes that the installation process makes to the system, which are usually surprising.
Some tools closely watch the operation of the software you're testing. A memory leak tool can help spot memory leaks in action. A test coverage analyzer reveals what part of your software hasn't been exercised during a test. And a profiler lists which parts of your code are used most often, so you can tune its performance.
Simulation tools give you powerful capabilities ranging from simulating an entire hardware platform that's not yet available for testing to simulating some component that's difficult to control otherwise (like with a mock object class for unit testing). A fault injection tool can simulate error conditions that are otherwise difficult to produce, frequently finding serious bugs in rarely executed error-handling code.
Test design tools help you create test cases, or at least the test data to feed into a test. They can be quite elaborate, requiring a detailed model of the software's behavior and producing thousands of tests, or they can be as simple as generating an all-pairs test set.
Uncontrolled variations in test environments cause big problems in a tester's productivity. Tools can help you set up a consistent test environment. Tools in this category include disk imaging tools that set up disk partitions and give you an operating system and software that are ready to run exactly the same way every time. Virtual machines allow you to run the software in a virtual hardware environment. Other tools, often home-grown, can set up various necessary parts of the test environment and also allocate lab resources from a common resource pool that's shared by the team.
Some tools help with a static analysis of your software or some representation of it. A static analysis tool examines source code and finds potential problems (once you tune it to filter out many things that aren't problems). Other tools help manage a software inspection process where we rely on humans to root out issues.
Beyond Testing
Some software applications not considered test tools are useful, like text editors, outlining and mind-mapping tools, scripting languages, and, of course, email and instant messaging tools. Even my cell phone has served as a useful tool for rounding up people who are needed in a meeting and taking screen shots. Other tools you may encounter, depending on your role, include revision control and configuration management tools, compilers, debuggers, requirements management tools, and system administration tools.
There are still other categories of tools that testers use beyond the ones listed in this article, but this is a start. As you try to evaluate what tools you need and whether you can get much of what you need from one vendor, focus first on the individual types of tools you need. Then you'll see how they can be put together. Sometimes you'll benefit from one-stop shopping, but you also might find a better fit for your needs by acquiring specialized tools from more than one source.
Further Reading
An explanation of application lifecycle management (ALM) tools.
