Further Thoughts on Boutique Testing

Three years ago, back when I had one of those day job things, I published a paper called "The Boutique Tester." Since that time, I've gone independent, sold testing and training services, had a chance to test the business model, met a bunch of people who were also trying it out, and watched the world change. Between cloud computing, crowd-sourced testing, and the recent claim that "test is dead," I daresay the freelance software landscape looks different than it did three years ago.

If you want to become a boutique tester, or if you're thinking about collaborating with one, I have a few suggestions.

Any New, Disruptive Service Has Sales Challenges

The Boutique Model requires a fair bit of speed; your contract will be a short one, where you are air-dropped into a project and take off. If the paperwork takes a few weeks to process and the contract then needs approval from HR, legal, and a vice president, then the software may have already shipped by the time the contract is approved! This can make working with large companies difficult. In addition, those large companies tend to want employees on long contracts for forty hours a week, and they prefer to deal with institutions, not individuals. You can see how selling boutique testing services might be harder than it seems.

Large companies do have one advantage: They have budget, while startup and small companies might not. Yet, the small company has a lot of other benefits, like the ability to make quick decisions and the kinds of development practices to which a boutique tester can add the most value. Several of my colleagues have managed to carve out a niche by testing a few hours here and there for multiple startups at the same time, but I’ve found that medium-sized companies are the easiest to get into. By “medium-sized,” I am not talking about number of people but rather the company’s mentality. A medium-sized organization is successful enough to have budget but small enough to make decisions quickly. Like large organizations, medium organizations tend to want extended contracts for forty hours a week, which makes losing a single customer kind of a big deal—the sort of risk the Boutique Model is designed to avoid.

It’s not business nirvana, but I've been independent for nine months now and am looking to stay that way.

Everyone Wants Test Automation

Ten years after James Bach published "Test Automation Snake Oil," I continue to get calls from companies that want me to come in for a three-month project to fix a broken test automation framework. Why is it broken? Some contractor came in for a three-month contract, wrote a bunch of stuff, then left. The company hit a crunch time, modified the GUI, and now the test automation is a big mess on the floor.
 
In a year, is this company going to call someone else to fix the code that I wrote?

The potential client often says that the great Matthew Heusser “really gets this stuff” and can design a “maintainable framework.” That's flattering and all, yet I can't help but notice that the system forces are exactly the same for me as they were for the previous contractor. Nothing has changed. I do think I have something to offer these companies, but I'm not excited about answering that specific call or tricking them by coming in to do something other than what they’re asking. So far, I have always had a better opportunity to pursue.

Don't get me wrong. All the teams I've been working with automate some checks. The developers use test-driven development, the testers write scripts on demand, and we often have a high-level tool like Fitnesse to communicate requirements or have developers write Selenium code to demonstrate that the feature is “done.” The thing I am talking about is having someone external—even contract—develop automation that drives a GUI after the code is “passed” to QA.

This can work and has for some organizations with certain types of problems. The work we did at Socialtext is often held up as a case study for a good framework. A few years later, I'm less excited. So few companies have the perfect mix of problems that lends itself to this sort of automation approach. Our case study may have been an exception, not the rule, but it's open for debate.

Adam Yuret, a boutique provider for management and test services recently put it this way: "I think test automation fetish is a pretty good heuristic for organizational dysfunction—not to say automation is always bad, but I think many dysfunctional organizations misidentify their problem as being a lack of automation."

Boutique Testing Services Can Be Sold by a Larger Company

When I wrote the original article on the Boutique Tester, I thought high-end testers would contract directly with companies. But, like I wrote above, it turns out that many companies want testing as a true service—like tap water they can turn on and off, with little concern over the "who" or if that person is available right now. The easiest way to do that is to hire a service-providing company, one with hundreds or thousands of testers who can be accessed at will.

Since that article was published, I've seen two companies move into that space: uTest, which offers crowd-sourced testing (as well as specialized test) services, and PQA in Canada. It turns out that PQA has been around for a long time, but they only recently got aggressive about competing in the US. Two of my friends, John Kotzian and Elena Houser, have done independent test work for uTest, accepting projects as they come and working on them when they want to from home. John tells me he even exceeded the cash salary of his old day job in his first year with uTest, working about forty hours a week.

On a related note, another challenge is that boutique testing only offers part of the solution, not the whole thing, so it creates a little more management work for the client. One fix for this is to partner with a developer and test on the projects the developer leads. My colleague Catherine Powell did this for several years. If you want to live the boutique life but can't take the risk, it might make sense to find a partner.

Experience in the Subject Area Matters

The first article downplayed the importance of subject-matter experience. It implied that any properly trained tester could drop into any project and be extremely valuable immediately. Three years later, I'd like to clarify things a bit. Yes, there are certain sorts of "common attacks" or "quick attacks" that can be used on most software applications. They are skills many testers don't have, and with those skills you can be valuable on most things. Still, I don't tolerate someone with an MBA claiming that a good manager can manage absolutely anything, and I'm not sure we can claim something similar as testers. Those "quick attack" skills are easy to learn, easy to apply, and mechanical. There other aspects to testing, like understanding the problem domain, the combination of inputs that have the greatest risk to the business, and what business functions matter the most to the end customer—the "getting inside the clients head" stuff—and they take time to develop.

A tester with quick attack skills can be strong and can be air-dropped, but one with domain expertise will be far stronger, especially for business technology tied to a specific industry instead of general end-customer technology. That leads me to either creating longer-term alliances with companies (where I will be very valuable and the logical choice to bring back for the next project) or focusing in specific industry slices like health care, banking, and medical devices.

There Are All Kinds of Calls for Testers Beyond Traditional GUIs

You might be surprised at the number of calls I get to test a REST API or a service-oriented system, to do performance testing, or even to help a company with monitoring of production applications. Companies have a hard time filling these slots because so many testers think of and market themselves as hunt-and-peck button pushers. That's an important and valuable service, but as the cloud gains traction, testers with above-average technical chops—or those willing to learn—will provide value by testing the orchestration of services.

The bottom line is that the boutique tester market is larger than I realized at first, because it can expand to cover more things. At the same time, there are more barriers to entry than I initially realized. Can a few high-end providers survive under fire? Ask me again in three years.