Conference Presentations

I Wouldn't Have Seen It If I Hadn't Believed It: Confirmation Bias in Testing

"It ain't what we don't know that gives us trouble; it's what we know that ain't so." Will Rogers was talking about confirmation bias-the tendency to feel secure in our beliefs rather than to seek evidence that might challenge them. In testing, confirmation bias prompts us to stop a test too early, to choose tests that conform too closely to the happy path, or to ignore results that confound our expectations. As a result, defects have a chance to hide in our self-induced blind spots. We can't eliminate confirmation bias, but we can manage and control it by diversifying our models, our techniques, and our test teams. In this hands-on and eyes-on session, Michael Bolton presents a set of exercises, videos, and conversations that show testing biases in action. Discover some new tricks that can help you defend yourself and your testing clients from being too sure, too soon, and later ... sorry.

Michael Bolton, DevelopSense
Meet "Ellen": Improving Software Quality through Personas

Users are the ultimate judge of the software we deliver because it is critical to their success and the success of their business. However, as a tester, do you really understand their tasks, skills, motivation, and work style? Are you delivering software that matches their needs and capabilities-or yours? Personas are a way to define user roles-imaginary characters-that represent common sets of characteristics of different users. David shares how his team at Microsoft defined and used one persona named “Ellen” to help them design, develop, and test the first version of a new product. David shares before Ellen and after Ellen examples of the product, showing how the product changed when Ellen joined the team. See examples of the robust test cases and acceptance scenarios they defined from unique insights that Ellen provided.

David Elizondo, Microsoft Corporation
Chartering the Course: Guiding Exploratory Testing

Charters help you guide and focus exploratory testing. Well-formed charters help testers find defects that matter and provide vital information to stakeholders about the quality and state of the software under test. Rob Sabourin shares his experiences defining different exploratory testing charters for a diverse group of test projects. For example, reconnaissance charters focus on discovering application features, functions, and capabilities; failure mode charters explore what happens to applications when something goes wrong. In addition, you can base charters on what systems do for users, what users do with systems, or simply the requirements, design, or code. Rob reviews key elements of a well-formed testing charter-its mission, purpose, focus, understanding, and scope. Learn how to evolve a test idea into an exploratory charter using examples from systems testing, Scrum story testing, and developer unit testing.

Robert Sabourin, AmiBug.com
Large-scale Exploratory Testing at Microsoft: Let's Take a Tour

Manual testing is the best way to find the bugs most likely to bite users badly after a product ships. However, manual testing remains a very ad hoc, aimless process. At a number of companies across the globe, groups of test innovators gathered in think tank settings to create a better way to do manual testing—a way that is more prescriptive, repeatable, and capable of finding the highest quality bugs. The result is a new methodology for exploratory testing based on the concept of tours through the application under test. In short, tours represent a more purposeful way to plan and execute exploratory tests. James Whittaker describes the tourist metaphor for this novel approach and demonstrates tours taken by test teams from various companies including Microsoft and Google. He presents results from numerous projects where the tours were used in critical-path production environments.

James Whittaker, Google
STARWEST 2008: Branch Out Using Classification Trees

Classification trees are a structured, visual approach to identify and categorize equivalence class partitions for test objects. They enable testers to create better test cases faster. Classification trees visually document test requirements to make them easy to create and comprehend. Julie Gardiner explains this powerful technique and how it helps all stakeholders understand exactly what is involved in testing and offers an easier way to validate test designs. Using examples, Julie shows you how to create classification trees, how to construct test cases from them, and how they complement other testing techniques in every stage of testing. Julie demonstrates a free classification tree editing tool that helps you build, maintain, display, and use classification trees.

Julie Gardiner, Grove Consultants
STARWEST 2008: Telling Your Exploratory Story

What do you say when your manager asks, "How did it go today?" As a test manager, you might say, "I'll check to see how many test cases the team executed today." As a tester with a pile of test cases on your desk, you could say, "I ran 40 percent of these tests today," or "At the rate I'm going I'll be finished with these test cases in 40 days." However, if you're using exploration as part of your testing approach, it might be terrifying to try to give a status report--especially if some project stakeholders think exploratory testing is irresponsible and reckless compared to test cases. So how can you retain the power and freedom of exploration and still give a report that earns your team credibility, respect, and perhaps more autonomy? Jon Bach offers ways for you to explain the critical and creative thinking that makes exploratory testing so powerful.

Jon Bach, LexisNexis
Practical Pairwise Testing with PICT

Fault analysis reveals that interaction between the variables of dependent parameters is a common source of failure in complex systems. Imagine you are assigned to test a feature with twenty independent parameters and five possible states for each parameter. The total number of possible combinations is greater than five-hundred billion. At one test executed per millisecond, it would take more than 3,000 years to test all possible combinations. So, which combinations do we test? Pairwise testing is a systematic procedure to reduce the total number of tests by selecting a set of tests that evaluates every pair, rather than every combination. BJ Rollison compares the orthogonal arrays method to pairwise analysis and provides a detailed example of how to use PICT, a powerful, highly configurable combinatorial analysis tool that is freely available.

Bj Rollison, Microsoft Corporation
Pairwise Testing Comes of Age

You've heard of orthogonal arrays and pairwise testing. Perhaps you've used a pairwise test case generator tool. Have you ever wondered where these popular and powerful techniques originated? Actually they have been around for almost twenty years. During this time, important test design principles have emerged and choices for test generation tools have improved. George Sherwood, inventor of CATS, one of the first pairwise test tools, reviews what we have learned and how it applies to testing today. He shows the benefits of using pairwise test techniques for selecting configurations and for generating test data. George also outlines important considerations for successful pairwise test designs, including the problems to anticipate and avoid. George dispels the mystery of pairwise test generators with a simplified view of how they work.

George Sherwood, Testcover.com
Testing Lessons Learned from Extreme Programmers

One of the things testers often notice about Extreme Programming (XP) is that there is no defined role for testers on the team. Yet XP teams describe themselves as "test infected." They practice Test-Driven Development (TDD), writing executable unit tests before writing the code to be tested. Many teams practice Acceptance Test-Driven Development (ATDD), writing executable acceptance tests before implementing a feature. They use continuous integration to give them rapid feedback about the effects of changes. They practice pair programming, a technique that results in all code being peer reviewed before it's checked in. In short, XP teams test continuously from the very first moment of any given project. You could even call them "test obsessed." That obsession helps explain why Elisabeth Hendrickson, author of Test Obsessed, likes XP teams so much.

Elisabeth Hendrickson, Quality Tree Software, Inc.
Transforming Your Test Culture: One Step at a Time

Whether we develop software-based systems to create invoices, solve difficult physics problems, diagnose heart disease, or launch rockets, we've learned that nothing stays the same very long and software defects are inevitable. However, one thing has remained constant—the role and value of testing has been misunderstood by many in senior management. A Lockheed Martin Fellow since 2005, Tom Wissink describes steps undertaken at Lockheed Martin to change this culture of misunderstanding into a culture of appreciation, satisfaction, and excitement. Tom's experience has convinced him that this change is not just theoretical but both possible and rewarding. In a few organizations, both large and small, this has resulted in dramatic changes including greater tester satisfaction, increased company profits, and improved software quality often delivered on time and within budget.

Thomas Wissink, LM IS&S

Pages

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.