When you release a website or web application, it’s going to face a lot of very public load testing. If it performs poorly, there’s a good chance that you’re going to lose a lot of customers. Colin Mason offers some tips for load testing in order to ensure a better customer experience.
In this day and age, the web tends to be the number-one way businesses interact with their customers and prospects. Web performance—or the speed and reliability of your website, mobile site, and web applications—can have a huge impact on your business. According to the Aberdeen Group, a single-second delay in website response time translates to an 11 percent reduction in the number of pages customers and prospects view on your site, a 16 percent drop in customer satisfaction, and a 7 percent decline in conversions.
As a software tester or quality assurance professional, your job is to ensure superior website, mobile site, and web application performance, particularly during peak traffic periods when a business often has the most to gain (or lose). If the site is slow, you’re going to lose customers to competition. If customers can’t get through their transactions, you’re not going to make money.
Years ago, load tests were designed to validate that your database, server, and internal network were ready to handle peak traffic loads (in the vicinity of one hundred users!). As websites, mobile sites and web applications have grown significantly more complex over the past decade, they now require a whole new set of techniques and practices for load testing.
Load Test from Your Customers’ Point of View, Not Your Data Center’s
Today’s websites and web applications have evolved from simple, single-function tools to feature-rich composites offering extensive functionality from external third-parties like shopping carts, ratings, and reviews and analytics, as well as RSS feeds and social media plug-ins. In fact, the average website today connects to more than twelve hosts. While these extensive functionalities can enable a richer online customer experience, they also introduce performance risks, since any one component can cause a website or web application to slow down. Case in point: Tagman recently determined that Google’s +1 plug-in drags down page load times by a full second.
In addition to third-party applications, there are many other performance-impacting elements standing between your data center and your customers around the world, including cloud service providers, regional ISPs, local ISPs, content delivery networks, and even browsers and devices. Collectively, this concept is known as the web application delivery chain, and poor performance anywhere in the chain will degrade the cumulative customer experience and reflect poorly on you, the website owner, regardless of the actual cause.
The reality of modern websites and web applications is that even if your inside-the-firewall tools indicate that everything is running OK, that’s no guarantee your customers are happy. You can no longer just test inside the firewall, because this gives you only partial coverage and leaves you with significant blind spots. You must load test the entire web application delivery chain in order to identify and fix any weak links. Some businesses believe they can’t load test external elements because those elements are beyond their control. But, if you can do a load test from the customer’s point of view and include that entire web application delivery chain in your load test, then you really are in a stronger position of control.
Know What an Excellent Experience Looks Like
As a software tester, not only should you be able to clearly communicate to various functional groups with a stake in web application performance (IT, line of business, and marketing) the direct, bottom-line impact of web performance, but you should also be able to define “strong experience.” How fast and available do you need to be during peak periods? A recent industry survey found that 41 percent of online shoppers have no qualms about abandoning a preferred website to shop elsewhere after only one or two bad experiences during a peak time. In addition, a vast majority of online shoppers report being very much inclined to tell people they know about poor web experiences. Today, your customers expect superior experiences regardless of whether there are ten users on a site or ten million. Industry benchmarks can give you a quick view of how your competitors are performing and the expectation they’re setting in the marketplace regarding what constitutes a “good” online experience.
Besides speed and reliability, there are other pieces that, when put together, also contribute to the customer online experience, such as: Does the page load correctly? Does it look right? Are all the objects on the page? Are there a lot of 404 errors? Are there objects missing? By articulating the definition of “excellent experience,” you can create a shared language and basis of understanding among cross-functional teams, helping to unite them behind a common goal—which leads to the next point.
Get the Whole Team Involved in the Load Test
Even if they don’t necessarily need to be up in the middle of the night while the load test is running, all cross-functional groups with a stake in website and web application performance—including business, marketing, and IT executives—must be involved in load test goal setting, planning, and analysis, because they each possess the knowledge needed to develop and execute a proper test that yields critical insights.
For example, business executives often have information regarding sales and growth projections and ask: Is this web application going to be good for our brand? Are we going to get the revenue we expect out of it, and will our customers be happy with it during periods of both normal and peak traffic? Marketing executives often have information regarding upcoming campaigns and ask: Will our website and web applications be ready to handle a “best-case” uptick in traffic resulting from our campaign? IT executives know the capabilities and limitations of the existing infrastructure, and they ask: Do we have enough infrastructure in the system to support peak load?
Here is where software testers can assume a strategic role by rallying teams around the web application performance imperative and eliciting the types of responses that promote it. As indicated above, it starts by articulating the web performance goal. If a load test demonstrates that this goal is not being met, business executives may be more inclined to pay for the necessary investments, understanding that this expenditure can offset the heavier cost of disgruntled customers and lost sales. Marketing executives can make smarter decisions that maximize the impact of campaigns (for example, timing to coincide with more IT resources being available). IT executives can plan better and strike a more accurate balance between having enough resources but not so many resources that they sit idle. IT executives can also show marketing teams how the addition of certain “heavy” website content (for example, graphics or video) may impact web performance, leading to more-informed content management decisions.
Plan Early, and Test Early and Often So That You Can Address Problems
Planning a load test early in the development cycle—weeks or even months in advance of website or web application deployment—allows you to develop the smartest, most strategic load tests, revealing critical, useful insights versus just load testing “in the dark.” For example, by collaborating with business, marketing, and IT teams early, you can better understand and answer the questions: What areas of the website and which web applications are most important and in need of load testing? For which geographies, at what times of day, and across which types of devices?
By answering these questions well in advance, software testers can prioritize their load testing efforts in accordance with business impact. Planning early also enables you to include any third-party service providers in your test, which is important because your ability to deliver performance under load is increasingly a byproduct of their abilities.
When it comes to actually executing the load test, you should test to the peak traffic you’ve ever seen on your site and go beyond it. Consider going anywhere from two to ten times your peak traffic, depending on what you’re testing for and the potential volumes you expect. Rarely do businesses meet their performance goals in the first load test. If you know you are going live on Tuesday but leave testing to Monday, the only answer you’re going to get is a yes or no—as in, we’re going to succeed or fail. If you test well in advance of application deployment, you’re still apt to get a yes-or-no answer, but at least you have time to address it, and that’s the key.
Stop Thinking of the Web and Mobile Web as Separate Entities
The 2010 holiday season was a breakout season for mobile phones, with digital marketplace eBay offering compelling proof. The company reported that Black Friday 2010 sales across its suite of mobile apps almost doubled over the previous year, while its Cyber Monday mobile sales jumped 146 percent year over year.
As smartphones continue to increase in popularity, customers are coming to expect the same level of web performance on their mobile phones as they do on a desktop PC. A new consumer survey reported that 71 percent of mobile web users expect websites to load as quickly or even faster on their mobile phone compared to the computer they use at home—up from 58 percent in 2009. Seventy-three percent of consumers (up from 20 percent in 2009) are willing to wait only five seconds or less for a single web page to load before leaving the site. And, a third of mobile web users who encounter a poorly performing website or application will go to a competitor instead.
What does all of this tell us? It’s no longer good enough for your mobile site to be prepared for “ordinary” business conditions. You also have to be prepared for peak conditions, which may mean load testing. When higher-than-usual mobile web traffic volumes lead to unbearably slow load times, revenues, brand image, and customer relationships can be jeopardized. Businesses must now carefully ensure their mobile sites and infrastructure can handle peak traffic conditions when customer expectations are at their highest, such as the holidays or when you’re leveraging a mobile ad network to drive increased traffic to your site.
Like “traditional” websites, load testing for mobile sites and their applications requires careful planning, such as determining which devices are most prevalent among our customer base and should therefore be prioritized from a load testing perspective. And, mobile web benchmarks can give you an idea of the mobile web experience being offered by your industry counterparts.
Conclusion
At the end of the day, ensuring the best possible customer experience during periods of heavy traffic is the goal. You want to find problems before they do. To achieve this goal, you must build your load test from your customer’s point of view at the end of the web application delivery chain. Many businesses find it useful to leverage load generated from the cloud combined with load generated from users’ desktops and mobile devices. This approach delivers a bird’s eye view into the experiences of real customers under load across a wide variety of geographies and device platforms.
In addition, clear definition of performance goals, input from all key stakeholders, early planning and testing, and equal emphasis on both desktop and mobile are keys to an effective load testing strategy. Today, software testers have an opportunity to drive this strategy, paving the way not only to more effective load tests but also higher-performing websites, mobile sites, and web applications that help their businesses capitalize on peak traffic periods.
