Testing Services: Isolate, Automate, and Add Value

Understanding what’s going on “behind the scenes” provides us both insights into test design and details on failure, which makes our work more valuable. For example, imagine an application with three tiers: a data tier, a web-services tier, and a GUI tier with a different developer for each. A tester who is running tests only through the GUI finds a defect and assigns it to the GUI developer. The GUI developer, after checking his code, realizes that the bug is not in the GUI and assigns it to the web-service developer who, in turn, assigns it to the database developer. As our GUI testers find bugs that exist lower in the architecture, defects will begin taking longer and longer to pinpoint, troubleshoot, and resolve. Tests that are one layer down at the service level can catch the issue earlier and eliminate debugging and handoffs.

Some application architectures make it inherently easier to move beyond black box GUI testing. Service-oriented architectures (SOAs) generally separate the GUI from the business logic, creating a “loose coupling.” This provides greater opportunities for testers to “plug” tests into individual services. Many tools exist to aid in this testing—some open source, others free, and some commercial. While having some knowledge of SOA concepts will help a tester get up and running more quickly on these tools, some of these tools are so easy to use that you can get started and get effective during the same day—no knowledge of web protocols is required.

The tools take care of much of the overhead involved in testing services, including generating the request and reading the response from the service. Adding parameters to run the test with different values is generally straightforward with these tools, which allows you to quite easily move from one test to a suite with boundaries and positive and negative tests, etc.. With the request-response cycle of SOA services generally measured in seconds or less, a large number of test cases can be run against a service in a short period of time.

An Example
To get things started, let’s consider this scenario: a simple web form that takes a zip code into a form and then responds with the city and state represented by that zip code. As intuitive testers, we talk to our development team and learn that this function is driven by an underlying service which uses Simple Object Access Protocol, commonly known as SOAP. The developer tells us that the service is called USZipSoap. All we really need to test this service directly is a copy of SoapUI (free and open source) installed, the location of the Web Services Description Language (WSDL) file, which is the contract in XML form that specifies exactly what the service does and how requests and responses are to be formatted, and the location of the service itself, commonly called the endpoint.

Creating a test to verify the basic function of the USZipSoap service is straightforward and can be accomplished by following these steps. In this example, we will send a valid United States zip code to the USZipsoap service and receive information from the service pertaining to the city, state, and time zone for that zip code.

1. In SoapUI, create a project. This will give you a place to view and store the WSDL as well as your tests.
2. Point SoapUI to the WSDL using the location provided on the website. In real life, your developer would provide the WSDL location to you. This step is not strictly required, but I highly recommend doing this as the WSDL tells you almost everything you need to know about the service.
3. In the navigator window, Soap UI will display the operations made available by the USZipSoap service. Operations are the useful functions provided by the web service. A web service can provide a single operation or multiple operations. In this case, the web service USZipSoap provides four operations. We will use the operation called GetInfoByZIP.
4. The SOAP protocol uses a request and response scheme in which a client makes a formatted request and the service returns a formatted response. At this point, you need to tell SoapUI to add a request by clicking the plus sign next to the name of the operation.
5. Add your data. Using the WSDL as a guide, SoapUI will create and format most of the request for you. You just need to find the XML tags indicating the data needed by the request. In this case, the tags are <web:USZip> </web:USZip>. You add the zip code to be sent to the service between these two tags. If you are using the zip code for Phoenix, AZ in your test, your entry would look like this: <web:USZip> 85040</web:USZip>. SoapUI Pro, the paid version of the tool, provides a form to enter any necessary test data into a request, but this is not necessary in our case.
6. Create a test case by right clicking on the request and selecting “Add to Test Case.” SoapUI will prompt you to create a new test suite into which you can create a new test case.
7. Add one or more assertions to your test case. Assertions tell SoapUI what to look for in the service’s response. There are over a dozen assertions available to validate things like proper response formatting, response time, SOAP faults, and data returned. In this case, we add two “simple contains” assertions to look for the text string “Phoenix” and the text string “AZ.” We also add assertions to validate that the response is valid in the SOAP format and it does not contain a SOAP fault.There are four assertions in total in this example.
8. Execute your test! As soon as the service takes your request and produces a response, SoapUI will indicate whether your assertions were all met and will also allow you to look at the response in case errors are indicated. In this example, all four assertions pass, so green indicators appear next to each assertion. You can see the formatted request and response in the top-center and top-right panels of the screenshot provided.

While SOA is used here as an example, it is definitely not the only architecture in which service testing can be applied. Making the transition from black-box testing involves selecting (or building) the right tool for your architecture, and there are tools available for a wide variety of architectures. The key is to find or work with your development teams to create the “plugs” into which your tool can send tests and data (inputs) and from which it can receive results (outputs).

One (free and open!) test tool that can be used for service testing across a broader range of architectures and programming languages is FitNesse. FitNesse allows (or more accurately it requires) testers and developers to work together to generate test cases and to plug those tests into the application under test. While FitNesse generally requires more developer effort than tools developed for specific architectures, the level of developer effort required when using FitNesse is generally much less than with pure white-box unit testing tools, and once the developer has created the plugs into the application (known as fixtures in FitNesse lingo), the test team can easily add or modify a large number of test cases as their understanding of the application changes. FitNesse has native support or plug-ins to support Java, .Net, Ruby, Python, PHP, and a number of additional languages so it can fit a wide variety of application architectures. 

Moving from strictly GUI-driven tests to service tests has the potential to improve a team’s results, not only through greater test coverage and better long-term efficiency. It also improves the testers’ understanding of their applications. Testers, who are likely already the subject-matter experts on their applications from a business perspective, become much more powerful assets for their teams when their level of technical knowledge of their application increases through the process of service testing. This is a win-win-win for the tester, the test team, and the project team.

Want More?
You can find the WSDL, the definition of the zip code service, at http://www.webservicex.net/uszip.asmx?WSDL

For more information about the USZipSoap service , check out http://www.webservicex.net

Additional Images

Figure 1. SoapUI is organized so users see the hierarchy of tests, the tester’s request, the service’s response, and the test results all in one unified view.

Figure 2. After the tester enters the zip code in the request pane and clicks the green arrow, the response appears to the right, giving the tester a full view of both the request and its corresponding response.

Figure 3. Test results, based on the tester’s assertions, appear at the bottom of the screen and are color-coded to quickly show whether the assertions were valid or invalid.