|
|
STAREAST 2006: Testing Outside the Bachs Simply put, exploratory testing means designing your tests as you perform them. When it's done well, it's a fantastically productive and rewarding approach to testing. However, to do it well requires training, practice, and discipline. Lecture presentations about exploratory testing are a poor substitute for seeing it and doing it. So . . . plan to bring your laptop to this session and test along with James Bach and Jon Bach as they demonstrate exploratory testing in a live testing workshop. Participate or just observe as exploratory testing is performed in real time with play-by-play and color commentary. Learn how to bring structure to this apparently unstructured testing method. See if you can find bugs that they do not find as you test "outside the Bachs"!
|
James Bach, Satisfice, Inc. and Jon Bach, Quardev Laboratories
|
|
|
Code Coverage: Where Does it Fit? Many organizations use code coverage almost religiously in their testing. Just as many or more organizations do not use code coverage or have tried it and stopped. If you want to begin using code coverage for the first time or improve its value and usage within your team, come hear what Dale Brenneman has to share. Using real-life examples, Dale explains the value of code coverage analysis as part of a comprehensive test plan and the potential side effects when you do not use code coverage. Find out about the many levels of code coverage and ways to enhance the value of code coverage analysis with other analysis techniques. Take away a step-by-step approach for integrating code coverage analysis into your organization's test process and fitting it into your functional test automation program.
- The levels of module code coverage: entry, line, statement, branch, Boolean, cyclomatic path, all paths
|
Dale Brenneman, McCabe Software
|
|
|
A Balanced Scorecard Approach for Assessing Test Value and Success Internal test metrics--test progress, defect density, and TPI/TMM measures on process improvement-do not reveal the complete picture of test value and success. By comparing common test metrics with those found in the Balanced Business Scorecard--financial, customer, internal, and learning/innovation metrics-we see the need to also report financial and customer measures. Some of these measures are quantitative (such as profits), and others are more qualitative (for example, customer satisfaction). Learn to measure the financial impact of testing through productivity metrics and measures of how testing affects the total cost of quality. Include in your reporting qualitative assessments such as the customers' perception of the usefulness of testing, the visibility of testing on projects, acceptability measures, and estimation accuracy.
- Set measures for all viewpoints of testing's value and success
|
Isabel Evans, Testing Solutions Group Ltd
|
|
|
Test Metrics in a CMMI Level 5 Organization As a CMMI® Level 5 company, Motorola Global Software Group is heavily involved in software verification and validation activities. Shalini Aiyaroo, senior software engineer at Motorola, shows how tracking specific testing metrics can serve as key indicators of the health of testing and how these metrics can be used to improve your testing practices. Find out how to track and measure phase screening effectiveness, fault density, and test execution productivity. Shalini describes the use of Software Reliability Engineering (SRE) and fault prediction models to measure test effectiveness and take corrective actions. By performing orthogonal defect classification (ODC) and escaped defect analysis, the group has found ways to improve test coverage.
CMMI® is a registered trademark of Carnegie Mellon University.
- Structured approach to outsource testing
|
Shalini Aiyaroo, Motorola Malaysia Sdn. Bhd
|
|
|
Inside The Masters' Mind: Describing the Tester's Art Exploratory testing is both a craft and a science. It requires intuition and critical thinking. Traditional scripted test cases usually require much less practice and thinking, which is perhaps why, in comparison, exploratory testing is often seen as "sloppy," "random," and "unstructured." How, then, do so many software projects routinely rely on it as an approach for finding some of its most severe bugs? If one reason is because it lets testers use their intuition and skill, then we should not only study how that intuition and skill is executed, but also how it can be cultivated and taught to others as a martial art. Indeed, that's what has been happening for many years, but only recently have there been major discoveries about how an exploratory tester works and a new effort by exploratory testing practitioners and enthusiasts to create a vocabulary.
|
Jon Bach, Quardev Laboratories
|
|
|
Your Development and Testing Processes Are Defective Verification at the end of a software development cycle is a very good thing. However, if verification routinely finds important defects, then something is wrong with your process. A process that allows defects to build up-only to be found and corrected later-is a process filled with waste. Processes which create long list of defects are . . . defective processes. A quality process builds quality into the software at every step of development, so that defect tracking systems become obsolete and verification becomes a formality. Impossible? Not at all. Lean companies have learned how wasteful defects and queues can be and attack them with a zero tolerance policy that creates outstanding levels of quality, speed, and low cost-all at the same time. Join Mary Poppendieck to learn how your organization can become leaner.
|
Mary Poppendieck, Poppendieck LLC
|
|
|
Let's End the Defect Report-Fix-Check-Rework-Cycle Find out how teams transitioning to Agile practices must re-think their workflows and project metrics originally designed to handle many hundreds of defect reports that occur in typical testlast development cycles. Richard Leavitt discusses how a real-world implementation of key practices like early testing and continual integration-though not without bumps and bruises-lowered the number of open defect reports by an order of magnitude. These practices also can improve how the team communicates, reduce delays, and provide more direct measures of project status, feature progress, and release readiness.
|
Richard Leavitt, Rally Software Development
|
|
|
The Next Stop in Test Automation: Test Environment Setup To achieve the most effective test automation, you need to go beyond automated test case creation and implement automated environment setup. In tracking the testing time spent over several projects, the Clustering group Amit Mathur worked with found that more staff time was being spent on setup than on actual testing. He discusses and demonstrates the Environment Configuration System (ECS) his group developed to automatically set up test environments and trigger automated test execution. Find out how ECS has dramatically improved the ability to automate many more manual tests. ECS employs a scripting language (Perl) and libraries for environment setup and test cases. Based on interface specifications and dependency rules, ECS bundles environment setups and test cases for complex test scenarios. Learn how you can adapt the ECS approach for your applications and environment.
|
Amit Mathur, VERITAS Software Corporation
|
|
|
Systematic Techniques for Fault Detection and Isolation Selecting the appropriate testing techniques and test cases improves test efficiency, reduces time to market, and gives you confidence that the system is ready to ship. Using real-world case studies as examples, Madhav Phadke explains the fundamentals of robust test case selection and how code coverage can improve your test results. He discusses ways for testers to support debugging and faster repairs by isolating defects to a specific part of the software. Learn to select test outputs based on "total function evaluation" rather than end customer outputs and ways to use orthogonal arrays for testing combinations of parameters. Take away a list of free or inexpensive tools that can speed up your testing process.
|
Madhav Phadke, Phadke Associates
|
|
|
Risk: The Testers Favorite Four Letter Word Identifying risk is important-but managing risk is vital. Good project managers speak the language of risk, and their understanding of risk guides important decisions. Testers can contribute to an organization's decision making ability by speaking that same language. Learn from Julie Gardiner how to evaluate risk in both quantitative and qualitative ways. Julie will discuss how to deal with some of the misconceptions managers have about risk-based testing including: Testing is always risk-based. Risk-based testing is nothing more than prioritizing tests. Risk-based testing is a one-time-only activity. Risk-based testing is a waste of time. And risk-based testing will delay the project.
|
Julie Gardiner, QST Consultants Ltd.
|
