Why Testers Need to Get Used to Change

Growing up with computers, I have seen them change from business machines to personal entertainment devices. For the past ten years or so, I’ve watched computers go back to being more business-oriented. Now most entertainment is moving from computers to mobile devices. Why own a giant computer when you don't type much text? This change in the way people are using computers appears to create new challenges that I would like to focus on.

For example, people are accepting buggier software. Mobile users are now more like power-users within their touch-only world, which lowers traditional input errors. However, input-method variation is also going up and likely to cause new failures. Consider the case of Microsoft’s Kinect, which had problems recognizing people with darker-colored skin and problems with voice processing for high- and low-pitched voices or voices with heavy accents.

Additionally, I see another change in how performance is becoming more important; you can't, for example, wait a long time between traffic lights to send texts or check the weather. Since we don’t have a standard platform set, we have a divergence of systems that causes more difficulty in testing the hardware. User-usage patterns continue to split further between the consumer, business, and government.

Mobile devices are also not using traditional user interface design patterns, and with single-use apps, designs are now less consistent. We are seeing functionality changing all the time, sometimes just between devices due to A/B testing. New devices that were not originally designed to hook on to the Internet, like your air conditioner or heating appliance, will now be able join the web. Finally, we are witnessing high-profile (think blood pressure) medical device apps becoming standard for many people's mobile devices.

I see at least two major themes here: divergent use patterns and divergent expectations. Big business will keep having high expectations while consumers will have much lowered expectations and less patience. When all the applications cost a dollar, expectations for quality drops, yet if it takes more than thirty seconds to use, you might as well throw it away. This shift in attitude is an interesting one and one that I don't see discussed much by our community.

Sorry, I'm Not a Smart Phone App Testing “Expert,” but Who Is?
When I go to conferences, I don’t see many QA folk discussing how to test mobile apps. In my personal life I use a dumb phone and laptops. Maybe in the testing world, few of us are testing consumer mobile apps? That is to say, who hires a QA tester for a dollar application, anyway? There are over a million apps in Apple's app store but 60 billion downloads. This makes 60 thousand downloads for every app on average. An average of $60,000 take away 30 percent for Apple's commission often is not enough money to justify testing. Even if those numbers are off significantly for popular applications, it might still not justify a dedicated tester.

What does this do to our jobs? I'm invested in the job of testing. If you are reading these words, it's likely you have invested in testing as well. And our world is changing.

It seems to me, QA will likely be involved in the "free" apps as they are likely to be involved in economic transactions that pay a good deal more. Amazon, Delta, Hotels.com, and most other “free” applications for business actually involve much more value that the “free” label suggests; the cost of the app is built into the goods you buy. The same applies to games, which have add-ons that cost but are free-to-play.

What Will Change?
One large pattern of change that seems most obvious to me is the lack of attention to details and how that will be exploited. We are seeing one-person companies popping up, in which this one person does all the development, testing, operations, and marketing. He may also work in the office next to you during the day. With these developers lacking the resources to carefully run their app business, other small teams of black-hats are taking advantage, creating fake copies of the apps, and selling them, sometimes with malware.

The app market is currently the Wild West of software. In addition to this, recently high-end device manufactures are looking at using phones as commodity hardware to function with additional hardware, like medical devices. The US government may try to regulate these manufacturers, but other nations do not always follow the same rules. With so many apps being developed, the government can’t even provide the scrutiny these apps might need.

Then there are some smaller changes that are less easily categorized. Businesses will have reams of detailed customer data that will become a commodity due to aggressive data collection methods. With apps that are leaky and insecure, it's easy to get this kind of data. The large organizations that have been developing websites don’t have the knowledge (yet) to build apps, so they may choose to not create them, but instead build mobile websites (m-dots). Or, those organizations may invest in creating apps, but outsource that development work as the app market is still crazy.

In either case, these large organizations are still learning to deal with the app market, which is still mostly unmanaged and difficult to develop for. Due to all the fraud we see regarding apps, these large companies may eventually push for change and demand a more secure platform, but that will take time.

So What Happens to QA?
I think testing is diverging and see at least four different futures for four different careers:

1. Traditional Business: I don't see much change in the next few years. Traditional testing problems like functional testing and load testing will continue to exist. Mobile and Bring Your Own Device (BYOD) happen in business software, but I will consider that separately, outside of traditional business.
2. Innovative Hardware: With Internet-connected TVs, Kinect, medical devices, we will see new and interesting mash ups of our hardware. This takes intensive research and development and will include testing. In specific markets, testing by different cultures will be important as cultural patterns vary. If you want to do this, you may need to move to where the jobs are.
3. Mobile (App Store) Development: This area may not have testers involved. While some big companies will have apps, the majority of the apps will be without testing and maybe lower quality. Often these apps are replaceable and aren't expected to provide much value.
4. Government: As our market becomes more divergent and if a government wants to be relevant, it will need to handle new methods to interact with its constituents.

Considering the cost of hardware development and the fact that it's slower to develop, I don't expect to see much job growth in that sector. While government may add some jobs, it's relatively slow moving, due to the risks with technology going wrong (see the recent US healthcare problems for an example). The traditional business is rather traditional and should not be expected to go anywhere.

The mobile piece certainly has some growth to see, but it is a paradigm shift in how we test. With the majority of the mobile test market being developed by the Wild West, in which there are many untested applications, testing that business application becomes more difficult. User expectations have also changed and our QA techniques need to grow. Our previously finite list of OS browser configurations that were well understood are now looking nearly infinite.

Full Circle
I have now gone through my thoughts on how the industry has changed and while writing this I was struck how this all sounds like what was said in the early 1990s. Computers are faster! Beginners are using AOL! Geocities! Customers will accept services of lower quality, like losing a dial-up connection when the phone rings, and so on.

Anyone old enough to recall those days has seen all of this before. It's not always obvious that while all things change, things stay the same. As I get older it becomes more obvious. Handling change is important, but don't throw away your experience. Often things are a remix of a previous era. Jerry Weinberg, the master of consulting, recently wrote that, "We were doing incremental development as early as 1957... where the technique used was, as far as I can tell, indistinguishable from XP."

The reason we don't know how to handle this new technology and shift is we are all busy trying to understand it and get consensus. In the past, Microsoft just handed us consensus. Right now it's hard to give useful advice other than high level testing strategy with all the changes in the market. As the market creates winners and losers, things will settle down some and it will become slightly easier to have common test techniques. For now, get used to change. Learn something new, and try something different. Then try something different again tomorrow.