|
A Balanced Scorecard Approach for Assessing Test Value and Success Internal test metrics--test progress, defect density, and TPI/TMM measures on process improvement-do not reveal the complete picture of test value and success. By comparing common test metrics with those found in the Balanced Business Scorecard--financial, customer, internal, and learning/innovation metrics-we see the need to also report financial and customer measures. Some of these measures are quantitative (such as profits), and others are more qualitative (for example, customer satisfaction). Learn to measure the financial impact of testing through productivity metrics and measures of how testing affects the total cost of quality. Include in your reporting qualitative assessments such as the customers' perception of the usefulness of testing, the visibility of testing on projects, acceptability measures, and estimation accuracy.
- Set measures for all viewpoints of testing's value and success
|
Isabel Evans, Testing Solutions Group Ltd
|
|
Tester Skills for Moving Your Automation to the Next Level Job interviews for test automation engineers are often limited to, "How proficient are you with the tool vendor XYZ's scripting language?" This approach does little to help the hiring manager choose those individuals who are or will become highly skilled automation professionals. As a test engineer, you will need to acquire specialized knowledge and tool independent capabilities to become a test automation expert. Join Dion Johnson as he identifies the core set of tool-independent competencies required of a successful automated software test engineer: automation framework design, programming and debugging skills, object model concepts, and automation methods based on the required quality attributes. Learn how you, as a hiring manager, can identify these skills, or find out how you personally can improve your skills to become a true test automation expert.
|
Dion Johnson, DiJohn Innovative Consulting, Inc.
|
|
Build Rules: A Management System for Complex Test Environments Due to the interaction of many software components, there is increased complexity in testing today's software solutions. The problem becomes especially difficult when the solution includes combinations of hardware, software, and multiple operating systems. To automate this process, Steven Hagerott's company developed "Build Rules," a Web-based application with inputs from their build management and test execution systems. Using logical rules about the builds, test engineers define the characteristics of the build solution points. To deliver the latest and greatest builds that meet the characteristics defined for each solution point, the system dynamically translates these rules into server side nested SQL queries. Learn how their efficiency and accuracy has improved significantly, allowing test engineers to stay on track with many different build combinations and to communicate results to outside departments and customers.
|
Steve Hagerott, Engenio Storage Group, LSI Logic Corporation
|
|
STAREAST 2006: Testing Dialogues - Technical Issues Is there an important technical test issue bothering you? Or, as a test engineer, are you looking for some career advice? If so, join experienced facilitators Esther Derby and Johanna Rothman for "Testing Dialogues-Technical Issues." Practice the power of group problem solving and develop novel approaches to solving your big problem. This double-track session takes on technical issues, such as automation challenges, model-based testing, testing immature technologies, open source test tools, testing Web services, and career development. You name it! Share your expertise and experiences, learn from the challenges and successes of others, and generate new topics in real-time. Discussions are structured in a framework so that participants receive a summary of their work product after the conference.
|
Facilitated by Esther Derby and Johanna Rothman
|
|
Behavior Specification for Testing Embedded Systems A behavior specification is a valuable engineering artifact for the design, review, and testing of embedded software. It is a black-box model defining all interactions between system and environment and the conditional state-based causal relationships among them. Based on work by IEEE working group P1175, Dwayne Knirk describes a new reference model for specifying the behavior of computing systems. An embedded software control application is used to illustrate the application of this model. Dwayne provides an overview of the application and shows specification examples from the requirements document. He focuses on how the specification was initially developed, analyzed for completeness and consistency, used for generating test cases, and valued by the entire development and test team throughout the project.
|
Dwayne Knirk, Sandia National Laboratories
|
|
Open SourceTest Automation Frameworks Open source software has come a long way in the past few years. However, for automated testing there still are not many ready-made solutions. Testers often must spend their time working on test cases rather than working on a test automation framework. Allen Hutchison describes the elements of an automated test framework and demonstrates a framework that you can quickly assemble from several open source software tools. He then explains how to put the pieces together with a scripting language such as Perl. Once you build the framework, you can improve and reuse it in future test projects. At the end of the presentation, Google will release the described framework as a new open source project that you can begin using immediately.
|
Allen Hutchison, Google
|
|
STARWEST 2005: Planning for Successful Test Automation You have the automation tool. You have the right technical skills. You have the application experts at your disposal. It's time to jump in and start coding! Or is it? Many well-intentioned test automation efforts fail due to a lack of planning. Steve Walters describes his practical approach for developing an overall test automation strategy. Learn how to plan for automation success, select the right tests to automate, and prioritize them for a faster return on investment. By quickly eliminating poor automation candidates and using Steve's scorecard to assess the value of automating a test, you will be on the right track to achieving your automation goals. Take away a quantitative approach for deciding what to automate-and what not to automate-and the steps to develop a realistic plan and timeline for getting the job done.
|
Steve Walters, Dell
|
|
Aligning Testing Strategies wwith Corporate Goals When developing a testing strategy, test managers normally review the business case for the project, study the new requirements, and consider what they know about the system under test. By also including a review of your organization's mission, values, and corporate goals, you will immediately stand out among your peers and at the same time improve the business value of testing. Stewart Noakes has worked with test managers at both large and small companies to help them align test strategies with corporate goals. Using case examples, Stewart describes how they used this process to guide their testing approach and demonstrates how this approach significantly increases the tangible and intangible ROI on testing. Learn to use your company's corporate goals to help you make the right decisions about what to test, how much to test, and, importantly, when to stop testing.
|
Stewart Noakes, Transition Consulting Ltd
|
|
The Next Stop in Test Automation: Test Environment Setup To achieve the most effective test automation, you need to go beyond automated test case creation and implement automated environment setup. In tracking the testing time spent over several projects, the Clustering group Amit Mathur worked with found that more staff time was being spent on setup than on actual testing. He discusses and demonstrates the Environment Configuration System (ECS) his group developed to automatically set up test environments and trigger automated test execution. Find out how ECS has dramatically improved the ability to automate many more manual tests. ECS employs a scripting language (Perl) and libraries for environment setup and test cases. Based on interface specifications and dependency rules, ECS bundles environment setups and test cases for complex test scenarios. Learn how you can adapt the ECS approach for your applications and environment.
|
Amit Mathur, VERITAS Software Corporation
|
|
Journey to Test Automation Maturity Organizations that want to automate their testing generally go through a number of stages before they reach maturity. Whether you are about to begin your journey or are well under way, it is important to know where you are going and where you could go. In automating test execution, many organizations stop short of achieving their maximum benefits. This presentation looks at six levels of maturity in test automation and includes a self-assessment test to see where you are. It is important to have good objectives and realistic plans to achieve them. But in automating testing, these often seem very plausible at first but are not well expressed or are unrealistic. This presentation covers typical problems and examples of unrealistic automation plans and objectives. Leave with advice to help you have a successful journey to test automation.
|
Dorothy Graham, Grove Consultants
|