Errors and status codes

In REST, the status code does most of the work: 404 means not found, 400 means you sent something wrong, 500 means the server broke. GraphQL moves most of that into the response body, and the status code becomes much less reliable. That's the biggest single change for anyone testing a GraphQL API, and it's where most GraphQL tests go wrong.

Two kinds of error

A GraphQL request can fail at two different moments, and the response looks different for each.

Request errors

Ask for a field the User type doesn't have:

{
  "errors": [
    {
      "message": "The field `email` does not exist on the type `User`.",
      "locations": [{ "line": 1, "column": 20 }],
      "path": ["user"],
      "extensions": {
        "type": "User",
        "field": "email",
        "responseName": "email",
        "specifiedBy": "https://spec.graphql.org/October2021/#sec-Field-Selections-on-Objects-Interfaces-and-Unions-Types"
      }
    }
  ]
}

Each error object has up to four parts:

1. message, written for a developer. Wording varies between server libraries and versions, so don't make a test depend on the exact text unless the API documents it.
2. locations, the line and column in the query where the problem is. Useful when a long query fails.
3. path, the route through the response to the field that failed. Here it's ["user"]. For a field inside a list it includes the index, like ["books", 3, "price"].
4. extensions, anything else the server wants to add. This server adds the field and type names, a link to the rule in the GraphQL specification, and for some errors a code like HC0011. Codes are the stable part to assert on when an API documents them.

There's no data key at all, which is how you know nothing ran.

The status code depends on what you accept

Here is the same broken query sent twice. The only difference is the Accept header. First without one, as curl sends it by default:

$ curl -i -X POST https://api.testingapis.com/graphql \
  -H "Content-Type: application/json" \
  -d '{"query": "{ product(id: 1) { nope } }"}'

HTTP/1.1 400 Bad Request
Content-Type: application/graphql-response+json; charset=utf-8

{"errors":[{"message":"The field `nope` does not exist on the type `Product`.", ...}]}

Then asking for plain JSON:

$ curl -i -X POST https://api.testingapis.com/graphql \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"query": "{ product(id: 1) { nope } }"}'

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{"errors":[{"message":"The field `nope` does not exist on the type `Product`.", ...}]}

Same query, same error, same body. One is a 400 and the other is a 200. This isn't a bug. It follows the GraphQL over HTTP specification, which defines a newer media type, application/graphql-response+json, alongside the older application/json. With the new type, a request that fails before running gets a 4xx status. With the old one, servers keep the long-standing GraphQL habit of answering 200 to any request they could read, and leave the rest to the body.

For testing, this means two things. A test that asserts on 400 can pass in curl and fail when the same request comes from an app that sends Accept: application/json. And if your tests pin the Accept header, pin it to what the real client sends, not to what makes the status code look tidy. The body is the reliable signal. Check errors first, and treat the status as secondary.

{"query":"{ product(id: 1) { nope } }"}

Field errors

Field errors happen during execution, after the query passed validation. The addTodo mutation checks that the priority is high, medium or low. Send "urgent":

{
  "errors": [
    {
      "message": "Field 'priority' must be one of: high, medium, low.",
      "locations": [{ "line": 1, "column": 12 }],
      "path": ["addTodo"]
    }
  ],
  "data": null
}

The status is 200 whatever Accept says, because the request itself was fine. This time data is present, and it's null.

Why null for all of data, not just for addTodo? Because addTodo returns Todo!, which can't be null. When a non-null field fails, GraphQL can't put null there, so the null moves up to the parent. If the parent can't be null either, it moves up again. Here the parent is the top of the response, so the whole of datagoes. On a query where the failing field is nullable, the null stays where it is and the rest of the response survives. That's called a partial response, and a test has to handle both.

A null response doesn't mean nothing happened

Put the two ideas together: mutations run in order, and a failed non-null field nulls out the whole of data. Now send a mutation that toggles todo 2 and then makes a bad todo:

mutation {
  done: toggleTodo(id: 2) { id completed }
  broken: addTodo(title: "x", userId: 1, priority: "urgent") { id }
}

{
  "errors": [
    {
      "message": "Field 'priority' must be one of: high, medium, low.",
      "locations": [{ "line": 1, "column": 51 }],
      "path": ["broken"]
    }
  ],
  "data": null
}

One error, about the second field, and "data": null. There's no mention of the toggle at all. A client reading this response would reasonably conclude that the request failed and nothing changed. Now read the todos:

{ todos { id completed } }

{ "data": { "todos": [ { "id": 1, "completed": false }, { "id": 2, "completed": true }, ... ] } }

Todo 2 was falsebefore. The toggle ran, its result was thrown away when the null moved up, and the response gave no sign of it. (We toggled it back afterwards. If you try this yourself, do the same.) A client that retries this "failed" request toggles todo 2 again, back to where it started, and the user wonders why their tick vanished.

Null with no error

Ask for user 9999, who doesn't exist:

{
  "data": {
    "user": null
  }
}

No errorskey. On this API that's how "not found" looks. It's also exactly how "found, but the field is legitimately empty" would look on a nullable field. Tests need to know which case they're in. A test for a missing record should assert user is null and that there is no error. A test for an existing record should fail loudly if user is null, with a message that says so, rather than crashing three lines later on user.name.

HTTP errors still exist

Some failures happen before the GraphQL server sees the query, and those use ordinary status codes. On this API:

Gateways and proxies in front of a GraphQL server add their own: 401 for a missing token, 429 for rate limits, 502 and 504 when the server is down or slow. Those bodies often aren't GraphQL responses at all, so client code that always expects data or errors will trip over them. Check what the client does with each.

What to assert

For each GraphQL test, in this order:

1. On a request that should succeed, the errorskey is absent. Not an empty list: absent. If it's there, print it in the failure message, because it says exactly what went wrong.
2. On a request that should fail, errors is present, and the error you expected is in it, matched by path and code rather than by message text where you can.
3. Whether data is absent, null, or partly filled, and that it matches the kind of error.
4. The shape and values of data.
5. The status code, last, and only if you've pinned the Accept header.