Security and performance

A REST API has many endpoints, and each one does a fixed amount of work. A GraphQL API has one endpoint that does whatever the query asks for. That flexibility is the reason to use GraphQL, and it's also the source of most of its security and performance problems. This lesson goes through the ones to test. The general techniques are in security testing basics and performance testing; this is what's different about GraphQL.

Introspection

This API answers introspection queries, so anyone can list every type and field. That's deliberate for a practice API. Production APIs often switch it off, especially when the schema contains fields the app doesn't use yet, or internal ones.

Turning introspection off hides the map. It doesn't lock any doors. Every field still exists and still answers if someone guesses its name, and tools exist that guess from common words. Some servers also help the guesser: they reply to a near-miss field name with a suggestion, as in Did you mean "email"?. When an API claims introspection is off, test three things:

• A { __schema { types { name } } } query is refused.
• Misspelled field names don't get suggestions back.
• The fields that should be private are protected by an authorization check, not by nobody knowing they exist.

Authorization, one field at a time

In REST, access control usually sits on the route: /admin/usersneeds an admin. GraphQL has no routes to hang that on. Checks have to live on types and fields, and the classic mistake is protecting a piece of data in one place while it's reachable from another. This API has no accounts or relations, so imagine one that does:

# Blocked, correctly: you can't read another user's email
{ user(id: 42) { email } }

# The same email, reached through a post they wrote
{ post(id: 7) { title author { email } } }

The first query is checked, because someone thought about the user entry point. The second reaches the same User object through Post.author, and the check never runs. When you test authorization on a GraphQL API, list every path through the schema that ends at sensitive data, and try each one as a user who shouldn't see it. The schema makes this possible: follow every field whose type is User, wherever it appears.

Query cost

One request can ask for a lot of work. On an API where types link to each other, nesting multiplies:

{
  users {                # 100 users
    posts {              # x 50 posts each
      comments {         # x 20 comments each
        author {
          posts {        # x 50 posts again
            title
          }
        }
      }
    }
  }
}

That's a 100 × 50 × 20 × 50 tree: five million post titles from a request of a few hundred bytes. Rate limits that count requests see one request. The defences are a maximum depth, a maximum number of items per list, and a cost limit that adds up the expected work before running anything. Test them the way you'd test any limit:

• Find the documented depth or cost limit, and send a query just under it and just over it.
• Over the limit, expect a clear error before execution, not a timeout after it.
• Measure response time as you add one level of nesting at a time. A sudden jump tells you where the real limit is, documented or not.
• Check list arguments like first: 100000 are capped.

Aliases and batches

A schema with no nesting at all, like this one, can still be made expensive. Aliases let one query repeat the same field as many times as it likes:

{
  a1: books { title }
  a2: books { title }
  a3: books { title }
  # ...and so on, up to a500
}

Five hundred list lookups, one HTTP request. The same trick is used against login mutations: hundreds of password guesses in one request, so a limit of five attempts per request does nothing. A cost limit should count every aliased field, and rate limits on sensitive mutations should count calls, not requests.

Batching is the other multiplier: sending a JSON array of whole operations in one body. Some servers allow it. This one doesn't:

$ curl -X POST https://api.testingapis.com/graphql \
  -H "Content-Type: application/json" \
  -d '[{"query": "{ book(id: 1) { title } }"}, {"query": "{ book(id: 2) { title } }"}]'

{"errors":[{"message":"Invalid GraphQL Request.","extensions":{"code":"HC0009"}}]}

If the API you test accepts batches, the questions are the same as for aliases. Is there a maximum batch size, and does the rate limiter count operations or requests?

GET requests and CSRF

GraphQL over HTTP allows queries to be sent as GET, with the query in the URL. It doesn't allow mutations over GET, because GET is supposed to be safe to repeat, and because a GET can be triggered by any image tag on any website. This API gets both right:

$ curl -i "https://api.testingapis.com/graphql?query=%7B%20book(id%3A%201)%20%7B%20title%20%7D%20%7D"
HTTP/1.1 200 OK

{"data":{"book":{"title":"Clean Code"}}}

$ curl -i "https://api.testingapis.com/graphql?query=mutation%20%7B%20deleteTodo(id%3A%2099999)%20%7D"
HTTP/1.1 405 Method Not Allowed

{"errors":[{"message":"The specified operation kind is not allowed."}]}

Test the same on your API. A mutation that runs over GET can be fired by a link in an email or an <img>on a forum, using the victim's cookies.

POST has its own version of this problem. Browsers let any website submit a form to any other site, without asking permission, as long as the content type is a plain one like text/plain. A form can be built so its body happens to be valid JSON:

<!-- On some other website -->
<form action="https://api.example.com/graphql" method="POST" enctype="text/plain">
  <input name='{"query":"mutation { deleteAccount }", "x":"' value='"}'>
</form>
<script>document.forms[0].submit()</script>

If the GraphQL server accepts text/plainbodies and parses them as JSON anyway, and the API authenticates with cookies, that form deletes the visitor's account. Send a mutation with Content-Type: text/plain and with application/x-www-form-urlencoded, and expect a refusal. This API answers both with 404.

Caching

Browsers and CDNs cache GET responses, not POST ones. Since almost every GraphQL request is a POST to the same URL, the HTTP caching that REST APIs get for free mostly doesn't apply. GraphQL apps cache on the client instead, by type and ID, and that creates its own bugs: a mutation that returns too few fields leaves stale data on screen, because the client cache merges the new fields with old ones.

Some APIs use persisted queries: the client sends a hash of a query the server already knows, often over GET, so CDNs can cache the response. If yours does, check that an unknown hash is refused, that a hash can't be registered by an outsider, and that cached responses never carry one user's data to another. This API sends no caching headers on GET responses, so nothing in between should store them.

The N+1 problem

GraphQL servers resolve each field separately. On an API with relations, a naive server runs this query as 1 + N database queries:

{
  posts {          # 1 database query for the posts
    title
    author {       # 1 query per post for its author
      name
    }
  }
}

Fifty posts means 51 database queries. It works fine in development with five posts and falls over in production with five thousand. The usual fix is batching inside the server, often with a library called DataLoader, which collects the 50 author lookups and runs them as one.

You can find N+1 problems from the outside without seeing the database. Time the query with a list of 10, then 100, then 1,000 items, if the API lets you choose the size. Roughly flat times are good. Times that grow in step with the list, and more steeply when you add a nested field, point to one query per item. If you have access to the server's logs or tracing, count the database calls per request directly. That's the most reliable test there is.

What error messages give away

Errors in development builds often include stack traces, SQL statements, file paths and internal hostnames, in message or in extensions. That's helpful on a laptop and dangerous in production. Cause a few field errors on purpose, with bad input, a missing record or a huge number, and read every key of every error object. Compare: