Testing a SOAP service in Java

The earlier lessons each looked at one piece: a call, a WSDL, a fault, a query. This lesson puts them together into a test suite for the calculator service, the kind you'd write for a real SOAP service before a release. It covers every operation, checks every response against the contract in the WSDL, and probes the edges where services usually break. The class is CalculatorServiceTest.

What to test

A WSDL gives you the list. For each operation it describes:

Calling an operation

Every test goes through one helper, which builds the request, checks it succeeded and returns the response element:

private Element call(String operation, String a, String b) {
    String request = Soap.envelope(SOAP_1_1, "<" + operation + " xmlns=\"" + CALC + "\">"
            + "<intA>" + Xml.escape(a) + "</intA><intB>" + Xml.escape(b) + "</intB></" + operation + ">");
    Api.Response response = Soap.call("/api/soap/calculator", SOAP_1_1, CALC + operation, request);
    assertEquals(200, response.status(), response.body());
    // Not Body's getFirstChild(): the response is indented, so the first child is a text node of whitespace.
    return (Element) response.xml().getElementsByTagNameNS(CALC, operation + "Response").item(0);
}

Two details. Both values go through Xml.escape, for reasons the escaping test below makes clear. And the result is found by its namespace and name, AddResponse in http://tempuri.org/, rather than as the first child of Body. The first version of this test did the latter, and failed with a ClassCastException:

<soap:Body>↵
    <AddResponse xmlns="http://tempuri.org/">
    ↑
    the first child of Body is this line break and the spaces, a text node

The server indents its responses, and the line break and spaces between <soap:Body> and <AddResponse>are a text node in the DOM. Code that assumes the first child is an element works against a server that doesn't indent, and breaks the day someone turns pretty-printing on. Find elements by name.

Every operation in one test

Four operations with the same shape are a good fit for a parameterized test, which runs one method for each row of values:

@ParameterizedTest(name = "{0}({1}, {2}) = {3}")
@CsvSource({
    "Add,      7, 3, 10",
    "Subtract, 7, 3, 4",
    "Multiply, 7, 3, 21",
    "Divide,   7, 3, 2",    // whole numbers only: the remainder is dropped
    "Divide,  -7, 2, -3",   // and the result is rounded toward zero
    "Add,     -2147483648, 0, -2147483648",
})
@DisplayName("every operation gives the right answer")
void operations(String operation, String a, String b, String expected) {
    Element result = call(operation, a, b);
    assertEquals(expected, q.string(result, "t:" + operation + "Result"));
}

Each row is chosen for a reason. The first four check each operation once. The fifth checks how integer division rounds negative numbers: toward zero, so -7 / 2 is -3, not -4. That's a real difference between languages and a common source of bugs when a client and server disagree. The last uses the smallest 32-bit integer, the edge of what the schema's int allows. The name pattern gives each row a readable name in the test report:

every operation gives the right answer "Add"("7", "3") = "10"
every operation gives the right answer "Subtract"("7", "3") = "4"
every operation gives the right answer "Multiply"("7", "3") = "21"
every operation gives the right answer "Divide"("7", "3") = "2"
every operation gives the right answer "Divide"("-7", "2") = "-3"
every operation gives the right answer "Add"("-2147483648", "0") = "-2147483648"

Checking every response against the WSDL's schema

The WSDL's types section, from lesson 4, is an XML Schema. Java can load it and check that a response follows it, a process called validation. That turns the contract into a test that checks every field at once: types, required elements and order. The schema is loaded once, straight from the live WSDL:

@BeforeAll
static void loadSchemaFromWsdl() throws Exception {
    XmlQuery wsdl = new XmlQuery(Map.of("wsdl", "http://schemas.xmlsoap.org/wsdl/", "s", XMLConstants.W3C_XML_SCHEMA_NS_URI));
    Element schemaElement = (Element) Api.get("/api/soap/calculator?wsdl").xml()
            .getElementsByTagNameNS(XMLConstants.W3C_XML_SCHEMA_NS_URI, "schema").item(0);
    assertEquals(CALC, wsdl.string(schemaElement, "@targetNamespace"));
    schema = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI).newSchema(new DOMSource(schemaElement));
}

It finds the schema element by its namespace, checks it's for the calculator's namespace, and builds a Schema from it. Each test then validates the element inside the Body:

@ParameterizedTest(name = "{0} response matches the schema")
@CsvSource({"Add", "Subtract", "Multiply", "Divide"})
@DisplayName("every response matches the schema in the WSDL")
void responsesMatchTheSchema(String operation) throws Exception {
    schema.newValidator().validate(new DOMSource(call(operation, "8", "2")));
}

A passing validation does nothing; a failing one throws with a message. To see what it catches, here are three made-up responses that break the contract, and what the validator said about each:

<AddResult>eight</AddResult>
cvc-datatype-valid.1.2.1: 'eight' is not a valid value for 'integer'.

<AddResponse> with no AddResult
cvc-complex-type.2.4.b: The content of element 'AddResponse' is not complete. One of '{"http://tempuri.org/":AddResult}' is expected.

<AddResult>8</AddResult><Extra>1</Extra>
cvc-complex-type.2.4.d: Invalid content was found starting with element 'Extra'. No child element is expected at this point.

The messages start with codes from the XML Schema specification, but the text after them says plainly what's wrong: not an integer, a missing element, an extra element. Fetching the schema from the live WSDL, rather than a copy saved in the project, means the test checks the service against the contract it publishes today.

The edges

Limits

intholds values up to 2,147,483,647. Multiplying that by 2 doesn't fit, and the service has to say so rather than return a wrapped-around negative number:

@Test
@DisplayName("a result that doesn't fit in 32 bits is a Client fault")
void overflow() {
    String request = Soap.envelope(SOAP_1_1, "<Multiply xmlns=\"" + CALC + "\"><intA>2147483647</intA><intB>2</intB></Multiply>");
    Api.Response response = Soap.call("/api/soap/calculator", SOAP_1_1, null, request);
    assertEquals("Client", Soap.fault(response.xml()).codeName());
}

Escaping

Values that contain & or < have to be escaped before they go into an envelope, as lesson 1 explained. This test shows why, using a country code containing an ampersand:

@Test
@DisplayName("values must be escaped before they go into an envelope")
void escapeValues() {
    String code = "I&N"; // a value with a character that means something in XML

    // Pasted in as it is, it breaks the envelope, and the server can't even read the request.
    String broken = Soap.envelope(SOAP_1_1, "<GetCountryInfo xmlns=\"" + COUNTRIES + "\"><countryCode>" + code + "</countryCode></GetCountryInfo>");
    Soap.Fault fault = Soap.fault(Soap.call("/api/soap/countries", SOAP_1_1, null, broken).xml());
    assertEquals("Client", fault.codeName());
    assertEquals(true, fault.reason().startsWith("Malformed XML"), fault.reason());

    // Escaped, it arrives intact and gets a sensible answer.
    String escaped = Soap.envelope(SOAP_1_1, "<GetCountryInfo xmlns=\"" + COUNTRIES + "\"><countryCode>" + Xml.escape(code) + "</countryCode></GetCountryInfo>");
    assertEquals("Country code 'I&N' not found.", Soap.fault(Soap.call("/api/soap/countries", SOAP_1_1, null, escaped).xml()).reason());
}

Unescaped, the request isn't well-formed XML, so the server can't read it and answers with a Malformed XML fault. Escaped, the server receives exactly I&Nand answers that there's no such country, which is the correct response. The same mistake in a production client sends broken requests whenever a customer's name or address contains an ampersand, and a value containing < could even change the structure of the request. Always escape, or let an XML library build the document.

Strictness

What does the service do with a request that is almost right? Here, an Add element in the wrong namespace, and one with no namespace at all:

@Test
@DisplayName("an operation in the wrong namespace is a Client fault")
void wrongNamespace() {
    String request = Soap.envelope(SOAP_1_1, "<Add xmlns=\"http://example.com/wrong\"><intA>5</intA><intB>3</intB></Add>");
    Soap.Fault fault = Soap.fault(Soap.call("/api/soap/calculator", SOAP_1_1, null, request).xml());
    assertEquals("Client", fault.codeName());
    assertEquals("Operation 'Add' must be in namespace 'http://tempuri.org/', but it is in namespace 'http://example.com/wrong'.",
            fault.reason());
}

@Test
@DisplayName("an operation with no namespace is a Client fault too")
void noNamespace() {
    String request = Soap.envelope(SOAP_1_1, "<Add><intA>5</intA><intB>3</intB></Add>");
    Soap.Fault fault = Soap.fault(Soap.call("/api/soap/calculator", SOAP_1_1, null, request).xml());
    assertEquals("Operation 'Add' must be in namespace 'http://tempuri.org/', but it is in no namespace.", fault.reason());
}

Both get a Client fault that names the namespace the WSDL requires, http://tempuri.org/. That's the strict behavior you want. When this lesson was first written, the service didn't do it: it looked only at the local name, Add, and answered whatever the namespace. Leniency like that feels harmless, but clients with a wrong namespace then work against one server and fail against a stricter one, such as the next version of the same service.

Running the suite

$ mvn test
[INFO] Running com.testingapis.soap.CalculatorServiceTest
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0
[INFO] Running com.testingapis.soap.FaultsTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] Tests run: 52, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

Forty-eight tests in a few seconds, against the public API. The same suite runs in CI with no changes: the REST Assured tutorial's lesson on running tests in CI has a GitHub Actions workflow for a Maven project that fits this one as it is.