XML namespaces

Why names need namespaces, prefixes versus URIs, and the mistakes that make queries find nothing.

Open almost any SOAP response and the tags have prefixes and colons in them, like soap:Envelope, along with attributes full of URLs. Those are XML namespaces. They confuse more people testing XML than anything else, mostly because one parser setting and one XPath rule make queries silently find nothing. This lesson explains what namespaces are, then shows exactly where the traps are.

The problem namespaces solve

XML lets anyone invent element names, so the same name often means different things. In one document, name can belong to three different things:

Same name, three meanings

<order>
  <name>Spring sale order</name>      ← the order's name?
  <customer>
    <name>Ana Silva</name>            ← or a person's name?
  </customer>
  <product>
    <name>Desk lamp</name>            ← or a product's?
  </product>
</order>

Within one design, nesting tells them apart. The real trouble starts when a document mixes vocabularies written by different people. A SOAP message is the standard example: the envelope comes from the SOAP specification, and the content inside it comes from whoever built the service. If the service had its own element called Body, a program couldn't tell it from SOAP's Body. Namespaces fix that by giving every element a second part to its name, which says which vocabulary it belongs to.

How a namespace is written

Here is a real response from this API's SOAP calculator:

Response

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <AddResponse xmlns="http://tempuri.org/">
      <AddResult>8</AddResult>
    </AddResponse>
  </soap:Body>
</soap:Envelope>

A namespace is identified by a URI, here http://schemas.xmlsoap.org/soap/envelope/ and http://tempuri.org/. It's only a unique name. Nothing ever fetches it, and it doesn't have to point to a real page. URLs are used because whoever owns a domain can make up unique names under it.
xmlns:soap="..." declares a prefix. From here down, soap:in front of a name means "in the namespace with this URI". So soap:Envelope is the element Envelope in the SOAP envelope namespace.
xmlns="http://tempuri.org/", with no prefix, declares a default namespace. It applies to this element and every element inside it that has no prefix. So AddResponse and AddResult are both in the http://tempuri.org/ namespace, even though neither has a prefix.

Every element therefore has a two-part name: the namespace URI and the local name. Written out in full, which some tools do, AddResult here is {http://tempuri.org/}AddResult. Two elements are the same element only if both parts match.

Written as	Namespace URI	Local name
`soap:Envelope`	http://schemas.xmlsoap.org/soap/envelope/	Envelope
`soap:Body`	http://schemas.xmlsoap.org/soap/envelope/	Body
`AddResponse`	http://tempuri.org/ (the default)	AddResponse
`AddResult`	http://tempuri.org/ (inherited)	AddResult

The prefix doesn't matter, the URI does

These two documents look different, but they contain exactly the same elements:

The same response, written two ways

<!-- 1: a prefix for the envelope, a default namespace for the result -->
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <AddResponse xmlns="http://tempuri.org/"><AddResult>8</AddResult></AddResponse>
  </soap:Body>
</soap:Envelope>

<!-- 2: different prefixes for both -->
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:calc="http://tempuri.org/">
  <env:Body>
    <calc:AddResponse><calc:AddResult>8</calc:AddResult></calc:AddResponse>
  </env:Body>
</env:Envelope>

The second uses env and calc where the first uses soap and a default namespace. The prefix is a local nickname for the URI, chosen by whoever wrote the document, and different servers, libraries and versions choose differently. So never test the prefix. A test that looks for the text soap:Envelope breaks the day the server starts writing s:Envelope, though nothing meaningful changed. Test the namespace URI and the local name.

In practice that means telling your XPath library which prefixes youwant to use in queries, and which URI each one stands for. The project's XmlQuery takes them as a map:

NamespacesTest.java

@Test
@DisplayName("prefixes are only nicknames: both documents give the same answer")
void prefixesDontMatter() {
    String path = "/s:Envelope/s:Body/t:AddResponse/t:AddResult";
    assertEquals("8", q.string(Xml.parse(DEFAULT_NS), path));
    assertEquals("8", q.string(Xml.parse(PREFIXED), path));
}

The test uses s and t, which appear in neither document, and the same query works on both. Lesson 6 covers XPath properly; for now, read /s:Envelope/s:Body/t:AddResponse/t:AddResult as a path from the root down to the result.

The traps

An unprefixed name in XPath means no namespace

In the document, AddResult without a prefix is in the default namespace. In an XPath 1.0 query, AddResultwithout a prefix means "in no namespace at all". So the most natural query finds nothing:

NamespacesTest.java

@Test
@DisplayName("an unprefixed name in XPath means no namespace, so it finds nothing here")
void noPrefixMeansNoNamespace() {
    Document doc = Xml.parse(DEFAULT_NS);
    assertEquals("", q.string(doc, "//AddResult"));
    assertEquals("8", q.string(doc, "//t:AddResult"));
}

This catches nearly everyone. The query //AddResult returns an empty string, no error, and a test that compares it with "8"fails with a message that suggests the server returned nothing. The fix is to give the namespace a prefix in your query, even though the document didn't use one.

Same local name, different namespace

If the server changed the namespace of its elements, a correct query stops matching:

NamespacesTest.java

@Test
@DisplayName("the same local name in another namespace is a different element")
void sameNameOtherNamespace() {
    Document doc = Xml.parse(DEFAULT_NS.replace(CALC, "http://example.com/other"));
    assertEquals("", q.string(doc, "//t:AddResult"));
}

That's the right behavior. A different namespace means a different element, and in SOAP it often means a different version of the service. Your test should notice.

Java's parser ignores namespaces unless told otherwise

Java's DocumentBuilderFactoryis not namespace-aware by default. It keeps prefixes as part of the name and forgets the URIs. The result is the exact opposite of what you'd expect:

NamespacesTest.java

@Test
@DisplayName("the JDK's default parser ignores namespaces, which hides mistakes")
void defaultParserIsNotNamespaceAware() throws Exception {
    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); // namespace awareness off
    Document doc = factory.newDocumentBuilder()
            .parse(new ByteArrayInputStream(DEFAULT_NS.getBytes(StandardCharsets.UTF_8)));

    assertEquals("", q.string(doc, "//t:AddResult")); // the correct query fails
    assertEquals("8", q.string(doc, "//AddResult")); // and the wrong one works
}

The correct, namespaced query finds nothing, and the naive one works. Teams sometimes "fix" this by writing all their queries without namespaces, and their tests then pass while checking nothing about namespaces at all. One line avoids it, and the project's Xml.parse has it: factory.setNamespaceAware(true).

local-name(): the escape hatch

XPath can match on the local name alone, ignoring the namespace, with //*[local-name()='AddResult']:

NamespacesTest.java

@Test
@DisplayName("local-name() matches any namespace, for better or worse")
void localName() {
    assertEquals("8", q.string(Xml.parse(DEFAULT_NS), "//*[local-name()='AddResult']"));
    assertEquals("8", q.string(Xml.parse(DEFAULT_NS.replace(CALC, "http://example.com/other")), "//*[local-name()='AddResult']"));
}

It works whatever the namespace, which is useful for a quick look at an unfamiliar document. In a test, it's usually a mistake for the same reason: it would still pass if the server sent elements in the wrong namespace.

The whole test class, for reference:

NamespacesTest.java

package com.testingapis.soap;

import static com.testingapis.soap.Soap.SOAP11;
import static org.junit.jupiter.api.Assertions.assertEquals;

import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import javax.xml.parsers.DocumentBuilderFactory;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.w3c.dom.Document;

/** Lesson 2: what a namespace is, and the mistakes that make queries find nothing. */
class NamespacesTest {

    private static final String CALC = "http://tempuri.org/";

    // The same response written two ways: a default namespace in one, a "calc" prefix in the other.
    private static final String DEFAULT_NS = "<soap:Envelope xmlns:soap=\"" + SOAP11 + "\"><soap:Body>"
            + "<AddResponse xmlns=\"" + CALC + "\"><AddResult>8</AddResult></AddResponse>"
            + "</soap:Body></soap:Envelope>";
    private static final String PREFIXED = "<env:Envelope xmlns:env=\"" + SOAP11 + "\" xmlns:calc=\"" + CALC + "\"><env:Body>"
            + "<calc:AddResponse><calc:AddResult>8</calc:AddResult></calc:AddResponse>"
            + "</env:Body></env:Envelope>";

    // Our own prefixes. They don't match either document, and they don't need to.
    private final XmlQuery q = new XmlQuery(Map.of("s", SOAP11, "t", CALC));

    @Test
    @DisplayName("prefixes are only nicknames: both documents give the same answer")
    void prefixesDontMatter() {
        String path = "/s:Envelope/s:Body/t:AddResponse/t:AddResult";
        assertEquals("8", q.string(Xml.parse(DEFAULT_NS), path));
        assertEquals("8", q.string(Xml.parse(PREFIXED), path));
    }

    @Test
    @DisplayName("an unprefixed name in XPath means no namespace, so it finds nothing here")
    void noPrefixMeansNoNamespace() {
        Document doc = Xml.parse(DEFAULT_NS);
        assertEquals("", q.string(doc, "//AddResult"));
        assertEquals("8", q.string(doc, "//t:AddResult"));
    }

    @Test
    @DisplayName("the same local name in another namespace is a different element")
    void sameNameOtherNamespace() {
        Document doc = Xml.parse(DEFAULT_NS.replace(CALC, "http://example.com/other"));
        assertEquals("", q.string(doc, "//t:AddResult"));
    }

    @Test
    @DisplayName("the JDK's default parser ignores namespaces, which hides mistakes")
    void defaultParserIsNotNamespaceAware() throws Exception {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); // namespace awareness off
        Document doc = factory.newDocumentBuilder()
                .parse(new ByteArrayInputStream(DEFAULT_NS.getBytes(StandardCharsets.UTF_8)));

        assertEquals("", q.string(doc, "//t:AddResult")); // the correct query fails
        assertEquals("8", q.string(doc, "//AddResult")); // and the wrong one works
    }

    @Test
    @DisplayName("local-name() matches any namespace, for better or worse")
    void localName() {
        assertEquals("8", q.string(Xml.parse(DEFAULT_NS), "//*[local-name()='AddResult']"));
        assertEquals("8", q.string(Xml.parse(DEFAULT_NS.replace(CALC, "http://example.com/other")), "//*[local-name()='AddResult']"));
    }
}

Check your understanding

1.Does the parser fetch the namespace URI http://tempuri.org/ from the internet?Show answer
No. A namespace URI is only an identifier. Nothing downloads it, and it doesn't need to exist as a page.
2.In <AddResponse xmlns="http://tempuri.org/"><AddResult>8</AddResult></AddResponse>, which namespace is AddResult in?Show answer
http://tempuri.org/. A default namespace applies to the element that declares it and every unprefixed element inside.
3.Your XPath //AddResult returns an empty string, though the response clearly contains it. Why?Show answer
In XPath 1.0, an unprefixed name means no namespace, but the element is in a default namespace. Bind a prefix to its URI and query //t:AddResult.
4.The server switches from soap:Envelope to s:Envelope with the same URI. Should your tests fail?Show answer
No. Only the prefix changed; the element is the same. Tests should depend on the URI and the local name.
5.What does setNamespaceAware(true) change?Show answer
The parser records each element's namespace URI. Without it, Java's parser ignores namespaces, so correct namespaced queries find nothing.

Exercise

Call the countries service's ListCountries operation (lesson 3 shows how) and look at the response. Which namespace are the Countryelements in, and how is it declared? Write a query that gets the name of the first country using a prefix of your own choosing. Then parse the same response with a factory that isn't namespace-aware and see which of your queries still work.

XML namespaces

The problem namespaces solve

How a namespace is written

The prefix doesn't matter, the URI does

The traps

An unprefixed name in XPath means no namespace

Same local name, different namespace

Java's parser ignores namespaces unless told otherwise

local-name(): the escape hatch

Check your understanding

Exercise

Practise and go further