Why APIs go async

Most API calls are over in a fraction of a second: the client asks, the server answers, done. That model breaks as soon as the work takes longer than anyone is willing to wait. A client can only hold a request open for so long, and so can everything between it and the server. Async APIs are the set of patterns for work that doesn't fit inside one request.

Why one request isn't enough

Three things put a ceiling on how long a request can take:

• Timeouts along the way. Load balancers, proxies and CDNs close requests that run too long, often after 30, 60 or 100 seconds depending on the product and its settings. Your server can be working perfectly and the client still gets a 504.
• Clients. Browsers, mobile apps and HTTP libraries have their own timeouts, and users give up long before those.
• Capacity. A request held open for two minutes ties up a connection and a worker on the server for two minutes. Under load, slow requests crowd out fast ones.

Some results also simply don't exist yet when the request arrives: a refund the bank will confirm tomorrow, a build that hasn't started, the next price change. No amount of waiting inside one request fixes that.

Four patterns

This tutorial covers the first three. WebSockets have a tutorial of their own. Real systems mix them: a payment provider might accept a request with a 202, tell you the outcome by webhook, and offer a status endpoint for polling in case the webhook never arrives.

A first look: a job

This API runs fake background jobs that take as long as you ask. Start one and the server doesn't make you wait for it:

$ curl -i -X POST https://api.testingapis.com/api/jobs \
  -H "Content-Type: application/json" \
  -d '{"type": "report", "durationSeconds": 4}'

HTTP/1.1 202 Accepted
Location: /api/jobs/job_1009
Retry-After: 2

{"message":"Job accepted. Poll the Location URL for status.","data":{"id":"job_1009","type":"report","status":"queued","progress":0,...}}

{"type":"report","durationSeconds":4}

202 Accepted means "I've taken this on, it isn't done". Location says where to ask about it, and Retry-After says how many seconds to wait before asking. Everything a client needs is in the response; the next lesson builds a client that uses it properly.

What changes for testing

Async APIs break the habits built on request and response:

• One request no longer proves anything. A 202 says the work was accepted, not that it will succeed. The test has to follow the work to its end.
• Time becomes part of every test. Tests wait, and they need deadlines, or a job that never finishes hangs the whole suite.
• Some requests come to you. A webhook test needs somewhere for the webhook to land, which is what the bins in lesson 4 are for.
• Delivery isn't guaranteed to be clean. Messages can arrive twice, late, or out of order, and correct code has to cope with all three.