Testing in Production: Which Version Wins?

Would your marketing department like to know which website feature will excite online customers to buy more products, return to your site again and again, and increase revenue and profits? Harish Narayan describes how his team uses risk-based testing and statistical test design to optimally check features deployed with multiple website options. At Vistaprint, their measurement-focused marketing department requires live production tests of multiple web options-split runs in their jargon-that expose different features for different customer sessions; they choose to retain the one that “wins” to maximize returns. Preproduction testing of split-run features, especially when multiple runs are deployed in every release, presented challenges for Vistaprint’s testers. They found that assuring each new split-run works correctly on the website, not only with the existing functionality but also with the other marketing tests running at the same time, is critical. Join Harish to learn how his test team overcame these challenges to help their company deliver better customer service and more profits.