Simulating production conditions to test enterprise applications calls for some careful considerations. Learn how you can borrow the architect's hat without trying to fill his shoes.
A tester testing real-time enterprise applications requires technical know-how and expertise to be able to think through scenarios in which the application under test could behave abnormally in production. Beyond the functional, exploratory, boundary, regression and performance testing, a quality engineer has to wear a software architect's hat and start thinking like one. He needs to consider the hardware and software modules on which the application will be deployed in production and ensure application availability when a module or a combination of modules partially or completely fails.
Figure 1
Client-side Scenarios
An application’s client-side architectural test scenarios should include tests to check for over usage of Ajax, as this may often downgrade the application’s usability. JavaScript usage is an important test, too, as this can lead to variances in page-rendering times across browsers that have different JavaScript engines. On the same note, tests to check for high throughput, page load times under different bandwidths, and rendering the page on the cloud are also required. Tools such as traffic shapers are used to simulate these scenarios.
On the negative scenarios side are tests to check that there is no client data loss when the server side has a failover, how ctrl+N behavior (the keys to spawn new browser instances) has been implemented when simultaneous logins to the application are disallowed, and how user notifications are handled when required plug-ins such as applets and flash are not installed.
Server-side Scenarios
On the server side, it is always important to ensure that the pre-production, staging, or user-acceptance testing environment is as close as possible to the production environment in terms of network architecture and ports hardening. Often, an application that works in QA might fail in production, as it may not have been tested for port availability.
During load testing, the tester should use the cloud to simulate external load, as using internal servers does not simulate actual user bandwidth usage. Also, for applications with load balancers that serve multiple servers, more than one cloud machine should be used to distribute the load and avoid all requests going to just one server. Component failover at all levels is a very realistic scenario, and test cases should be defined to test all component levels—web server, application server, and database server.
For testing messaging applications that are processing or publishing live data, testing should be done by adding a “slow” consumer or subscriber to avoid system-health deterioration caused by messages that are in memory for a long time. Herein, tests should also check for proper system notifications and alarms. Also, use a full data set during load testing to expose potential issues related to simultaneous data actions. Such issues may not surface with partial data sets
Component-level Testing
A quality engineer thinking like an architect can unearth potential production issues by focusing on the architectural components that the application will run on. Resources are a constraint, especially in real time applications, so it is important that the test plan contains design tests to cover such cases.
To promote efficient usage of resources on the web-server side, the tester should ensure that static content is rendered from the web server instead of the application server. Similarly, system compression levels should be adequately tested, as bandwidth usage is often a concern in real-time applications.
On the application-server side, the tester should try things including forcefully shutting down all parts of the application, like the admin server and SNMP server, to verify that the application is responding as expected. Verifying that that components reconnect and communicate with each other when they are restarted is also important. To verify real-time server clustering, tests should cover test-environment hardening and check for port usage in production. While testing for clustering, the tester should use a similar number of nodes as in production to uncover any performance degradation as the number of nodes increases. Finally, testing the server-side cache’s performance should be done with a full set of data to help tune the cache effectively, since responses vary heavily as data sets increase.
As we move to the final component of the database server, tests should aim to uncover indexing issues and table lock issues with data sets that mimic the production environment. If the application under test is new, it should be tested with a projected data set of at least one year. Database failover tests should also be included to check for any application downtime when the database servers get into a hot-cold mode. During this time, if the application is still in use, tests need to verify different errors that the user would encounter.
In summary, in the current application-testing scenario, a tester’s role is not limited only to functional, automation, or performance testing. He also needs to understand the application’s architecture and challenge the design without worrying about stepping into an architect’s shoes. Executing detailed architectural testing helps document the application’s behavior, which can then be published to eliminate any surprises or anxieties and to minimize the application’s components’ glitches and hiccups.
