Offshore resources have proved useful, and companies continue to try and cash in on the cost savings. But those savings might not be what you or your company expected. In this column, Linda Hayes warns that outsourcing has some pitfalls we should always look out for.
The use of offshore resources for development, while not a panacea, has been accomplished with varying degrees of success by corporations in many industries. The promise of the same value proposition—a reduction in resource costs—has encouraged some to pursue the transfer of testing offshore as well. Interestingly, the value proposition is not as clear for testing as it is for development. In fact, some companies have reversed course and returned functional testing back onshore. The past five years or so of experience has illuminated four key factors affecting the success or failure of these efforts: process maturity, independent verification, time to value, and security.
P: Process Maturity
Just as companies found that their own development processes and project oversight had to be significantly improved before offshore development could be effective, they also discovered that their test processes lacked the maturity, necessary detail, and structure needed to be successful offshore.
High-level test scenarios, rife with phrases such as "enter valid data" and "verify correct results," presuppose the tester has domain and application expertise. When working with offshore resources that have neither expertise, the test documentation must be reworked to provide explicit step detail and data values in order to be usable. And while this investment might be worthwhile if the tests are reused often, a high rate of change in the application under test may negate the value.
I: Independent Verification
For mission-critical software with a high component of specialized domain knowledge, it is essential to independently verify the functionality. And offshore development already poses a risk of misinterpretation of requirements due to the lack of domain knowledge and experience.
Some have sought to mitigate a portion of this risk by engaging separate organizations for development and testing, but this approach drives up time and costs due to extra overhead for contract and project management. Internal experts—the very resources whose time is supposed to be saved—may end up with even more responsibility. Thus, acceptance testing often ends up being performed internally as a check against external development.
T: Time to Value
Offshore resources are never a quick fix, especially for testing. A 2005 report from AMR Research found that it took between fourteen months and three years before the offshore testers had sufficient familiarity to be effective in finding the root cause of problems. Others have found that the lack of domain knowledge resulted in spurious defects being reported that actually increased development overhead due to review and response times. In fact, one organization identified that as many as 33 percent of reported issues were traceable to tester error.
So companies that choose offshore resources as a way to remedy a resource deficiency, or to handle an influx of projects while keeping costs flat, might find themselves with higher costs and resource demands for one or more years. Without careful planning and realistic expectations, this confluence may end up eroding quality and extending schedules.S: Security
In many cases comprehensive testing requires access to a complete test environment, which may include a copy of production data. Although data scrambling utilities exist, they are not always used. Even when they are employed, they are time-consuming to implement and may not completely mask sensitive information. The potential for disclosing private or protected data is substantial and may violate laws and regulations, particularly in the financial services and health industries.
This risk exists for both internal and external resources but may be harder to manage in an offshore facility. Internal protections are usually already in place and must be extended to cover outside organizations.
Of course these factors have been successfully managed or overcome to yield a return on an offshore testing investment. For purely ad hoc testing, less-trained resources might actually have an advantage as they might be more likely to make the same mistakes as users. For the classic "brute force" approach, the sheer numbers of resources might prove beneficial.
Others have found that while less costly resources are useful for manually intensive execution, they are still more expensive than automated execution. And while using offshore resources to develop automated tests might appear to be less costly than doing it onshore, internal resources are nevertheless required to provide highly detailed test documentation as well as internal results analysis and verification.
As always, there are no easy answers or surefire formulas. Has your company moved testing offshore? What worked? What didn't? If you could do it over, what would you do differently...or would you? Leave your comments below.
