Everything we do has an economic impact because what we do has costs and benefits. Testing is about getting real feedback quickly, reducing wasteful testing activities, and putting a mirror in front of our applications. It becomes advantageous to understand the costs of these activities and direct the effort investment where it’s most beneficial.
You might think that when someone decides to write a book, everything is set up in his mind. Wrong. With my unit testing book I had a couple of topics to start with, but only after something like four months did a common thread appear. The thread was economics, which surprised even me. But I’ve started to see how economics fits into everything we do in software development, and specifically testing.
If you’ve done some kind of professional development, you probably developed certain skills. When we apply these skills, we call that work. We don’t think about it much while doing the work—sometimes even less after work—but everything we do has an economic impact because what we do has costs and benefits. Testing activities, regardless of who’s actually doing them, are the same.
History
Software is quite a young field, and testing as a profession is even more so. We needed testing right from the start, before a tester existed. After World War II, the software field grew bigger, and along with it came economic risks such as quality and maintenance costs and reputation loss. Back then, developers tested the software; it was part of the job. Then, in the 1980s, things changed: Computers became cheaper, and operating systems were finally stable and accommodating. Software was now cheaper to write.
At least, if you had developers. Developers were a scarce resource even before. Now the market needed more of them. This led to one of the industry’s most logical and stupid decisions. Many organizations decided they could replace those doing the testing activities with cheaper people and keep the existing developers working on features. This makes economic sense, but once developers were free from the shackles of code responsibility, quality suffered big time.
The first testers were bug catchers. Today’s testers do much more: They report the status of the product from all sides. To do that, they need to understand the risks, the market, and the users. They find ways to reduce those risks by exploring areas of uncertainty in the product, proving assumptions, and suggesting modifications. They define testing strategies, knowing that they don’t have all the time in the world and they need to prioritize their activities. In order to maximize the effectiveness of testing, they also need to be aware of the testing skills and activities in their team. Good testers understand economics.
All these qualities in a tester? No wonder we can’t find many good ones, and those we do find cost like (gasp!) developers!
Organizations
Let’s see how testing activities impact the bottom line, and begin with where the testers are. We all know that in agile teams, testers are embedded in the team. There’s an economical benefit for that: When they are collocated with the team, feedback flows quickly and collaboration speeds up and becomes more effective. However, mature agile teams are now facing another challenge.
Imagine we have five teams working on a web application, and one of the testers is a performance-testing expert. All the teams need her help, but how do we manage it? We can build different models of work to satisfy the growing need of performance testing in the organization. We can develop the performance capabilities of every tester in every team (although that may be excessive—should everyone know everything?). The tester can rotate among the teams, but then she won’t really belong to any of them. Or she could offer her consulting services to the other teams and limit her time in the current team.
That’s not a process issue. Step back and you’ll see a bigger issue: how recruitment, placement, and retention of testers should work in the agile era. How many do we need, and what should their skill set be? What’s their status in the team? And shouldn’t we ask the same about developers? That’s a big HR question that can make or break an organization.
Bugs
We all know bugs are waste. We need to reproduce them, document them, and then fix them.
But that’s not all: After we’ve documented them, we need to discuss them in a triage meeting. If they survive a round or two, we get to discuss them again. The product owner gets to shuffle them around the backlog, time after time.
The emergency bugs are worse. Sure, we need to fix them quickly, so there’s no further discussion. But while we drop everything to fix them, we go through context switches, work through the night, and enter some new bugs without noticing because we’re in a hurry. We mean well, but we hurt quality, which costs us later.
Debugging is the most deceiving part of fixing bugs because we’ve learned that it’s so useful, we can’t live without it. Mastery, however, does not always speed things up. The time we put into debugging easily can be reduced with tests. This frees time to write more new features.
Testing is not the right medicine here. Proper prevention is, which requires the developers to have testing skills, code reviews, and collaboration with the testers.
Testable Architecture
Mostly, architecture is built once. Maybe we thought hard about it before we built it. Or maybe it got patched up a lot. Regardless, without testability in mind, it becomes hard to test. This either leads to not testing those areas that are hard to test, or big efforts in order to test it. In economics terms, we either introduce big risks or slow development down. Neither is desired.
When we have a testable architecture, we can modify it according to the requirements we have. It may still cost us, but not enough to discard the idea completely. With automated tests around it, we can do the necessary changes.
Having tests around the application can also help delay “the big rewrite.” The code we write today should last for twenty years. Without tests, it will survive until the developers decide that it’s quicker to write the application from the ground up rather than continue maintaining the existing code. Big rewrites are costly and risky, and as such we want to delay them as much as possible. Tests help us do that.
Strategy
A good tester has the skills of a project manager. When devising a strategy, she needs to take into account how well the developers test their code so she can concentrate on covering the less tested parts. She consistently checks the feedback cycle to see where it can be reduced. She looks at the length of the automated test runs and thinks of ways to reduce it; where to invest in integration tests, which give better confidence but are more costly to set up and debug; and how much manual testing is needed based on the number of automated tests.
If she decides to use acceptance test-driven development combined with test-driven development as part of her strategy, she gets a definite economic impact boost. Working with a test-first approach aligns everyone with the business requirements. That means more code that application wants and less YAGNI (You Ain’t Gonna Need It) code. Test-driven development helps there, too, because it gets development to think before they actually write the code. Thinking before coding is one of the best bug prevention methods I know.
A strategy does not end with just testing activities, but with understanding the market. Today it is important for many organizations, especially start-ups, to be first to market. That requires a different strategy. Instead of big investment in automation, our tester does just-enough manual testing. The developers don’t automate what may not be the final product at all, so a good-enough manually tested product is enough to test (intended pun) the waters.
We professionals usually look down on such low-quality products. However, it may be the right decision at the time, and we can always add quality later, though maybe at bigger costs.
Economics Everywhere
Whatever we do has value and costs. Testing is about getting real feedback quickly, reducing wasteful testing activities, and putting a mirror in front of our applications. It becomes advantageous to understand the costs of these activities and direct the effort investment where it’s most beneficial.
Being a professional is about understanding economics. Are you a professional? Start thinking about those numbers.
