Vu Lam is CEO of QASymphony. He has over eighteen years of software industry experience, including founding two outsourcing companies and building a nationwide infrastructure connecting Vietnam’s banks, stock brokerages, and commercial entities. In this Sticky ToolLook interview, he discusses testing in the agile world.
Sticky ToolLook: What are some ways in which bug reporting and test documentation differ for agile organizations compared to “traditional” organizations?
Vu Lam: With respect to test documentation, one significant difference is that in traditional organizations, testers usually design detailed test cases based on available requirement specifications before actually executing any of them, usually weeks or months later. In an agile setting, testers tend to adopt exploratory testing, where they combine test design, execution, and emerging learning. Test cases are still designed, but they are created in the minds of testers when they test instead of being written down beforehand. In a sense, this is similar to the way agile developers design their software. That is, they rely more on emerging design happening during coding and refactoring rather than following the instructions of design documents.
Why does this make sense for agile teams? Changes. Teams adopt agile processes for that exact reason: Changes are inevitable in their project. Think about it. There is no point in rigidly executing a test case if it is already out of date by the time you test. Besides the overhead of documenting it and keeping it up to date, it simply does not make sense.
Regarding bug reporting, there is no fundamental change to the process. Most testers in an agile team still resort to bug-tracking systems to manage bugs, and they still need to submit well documented bug reports that help developers reproduce and fix the bugs. However, testers on an agile team have to rely on multiple communication mediums with developers more often, rather than collaborating via a non-verbal or non-visual medium constrained by a complicated defect-resolution workflow. On some teams I have been a part of, we even used the same tracking system to manage both user stories and bugs. On yet other teams, we treated outstanding bugs of a sprint just like user stories. Simplification, however, requires the team to approach bug fixing just like any other task in an agile project: It must be driven by the value it brings to the business.
Sticky ToolLook: Agile organizations focus on communication, often aiming for regular, face-to-face feedback. What are some of the communication challenges specific to a testing team at an agile organization, and how can teams overcome those challenges?
Vu Lam: I know many testers are more comfortable dealing with use case specifications, test cases and emails, etc. than doing face-to-face communications. Sometimes, this is simply their working style. Unfortunately, more often than not, this is due to a lack of understanding about agile processes and a tendency to stick with old habits. To get over these, testers have to educate themselves as much as they can about the values and principles of agile development and understand that they, too, have to embrace these values and principles for agile development to work. Regular, face-to-face communication is just one of the principles of agile development that testers have to adopt.
Yet, with global teams, face-to-face communication is not always possible. While local teams can and need to practice it, distributed teams have to learn how to maximize effectiveness and rely on other communication venues such as phone, IM, collaboration tools, visual correspondence, and documentation of ideas and test results.
Sticky ToolLook: How should a global organization handle agile? That is, if teams (or maybe even individual members of a team) are not in the same physical location, what can they do to improve their communication and documentation practices?
Vu Lam: Agile development clearly works best when the team and even their customers are co-located. Most of the literature on agile development does not deal with the challenge of distributed teams, so teams are generally left to solve this on their own. That said, I think there are a few practices that agile teams may use to alleviate the problem.
First and foremost, they can deploy a task collaboration system that shows clearly, in real time, who is doing what and their status. This is to empower project members to take ownership of their work and report progress, making it less of a coordination burden for the project manager. At the same time, it ensures that team members at different sites, while not seeing and talking much to one another, have the big picture of the project.
Besides the regular same-site daily meeting, they should have daily meetings among the leads of different locations. This is to ensure that information and issues are quickly shared and addressed. Yet, this introduces new problems with globally distributed teams—namely, that of time zone differences, language barriers, and cultures.
Again, teams need to effectively leverage venues such as phone, IM, and other collaboration tools. But these can’t solve some of the issues that arise from language and cultural barriers. I feel that, in conjunction with these venues, the effective use of visual communication is key. And this is what we mean when we talk about visual communications: How better for a tester across the world to get his point across than to literally capture his test steps, keystrokes, and screen movements annotated with comments, all in one document that his audience has access to prior to the discussion?
Sticky ToolLook: How do agile practices improve the feedback loop between finding and fixing bugs?
Vu Lam: Having a short feedback loop is essential to agile development, so it comes as no surprise that agile processes come with multiple practices enabling feedback to emerge as quickly as possible. Communication practices like daily standup meetings and an open workspace are the first that come to mind, but again we have the issue of globalization. We believe based on our experience that visual communication also minimizes the time between finding and fixing bugs.
Sticky ToolLook: Are there any challenges that agile practices actually introduce to software testing? If so, how do agile teams overcome or work around them, and what are some potential long-term solutions?
Vu Lam: The gist of agile development is about delivering value fast. For software testers, it means they have to find ways to continuously add value to the project. It is simply no longer possible to follow predefined checklists from a preferred one-size-fits-all methodology. They have to constantly ask themselves whether the things they are doing and how they are doing them support, defeat, or simply do not matter to that purpose.
On the other hand, they need to assist in making sure working software can be released quickly and regularly even in the face of changes. Self-educating on how to plan, design, and execute tests on the fly is essential. They have to learn to embrace changes and act upon emerging discoveries about the system. They can no longer rely on thick requirement documents or assume they have the time to create and maintain giant test cases. And they have to learn to engage more often in visual communication irrespective of location, for it is usually the fastest and most effective way to get the most up-to-date information across among team members.
Agile development works but is not without challenges. Yet, there is no shortcut, only solutions.
