Streaming JSON and chunked responses

Not every streaming API uses Server-Sent Events. Many send an ordinary HTTP response that simply arrives in pieces: a large export written row by row, a search that returns results as it finds them, an AI model producing text a few words at a time. The client can start using the data before the response is finished, if it's written to. This lesson covers the three common shapes and the timeouts that go with them, using the streaming API.

Chunked transfer encoding

Normally a response says how long it is, in Content-Length. A server that doesn't know the length in advance uses Transfer-Encoding: chunked instead, sending the body as a series of pieces until a final empty one. Any HTTP client handles it; the question is whether your code waits for the end or reads as it goes.

This script prints each piece as it arrives, with the time since the request started:

// Times when each piece of a streamed response arrives
async function timed(label, path) {
  const t0 = performance.now();
  const res = await fetch("https://api.testingapis.com" + path);
  const decoder = new TextDecoder();
  console.log(`=== ${label}: ${res.status} ${res.headers.get("content-type")} transfer-encoding=${res.headers.get("transfer-encoding")}`);
  for await (const chunk of res.body) {
    console.log(`  +${Math.round(performance.now() - t0)}ms`, JSON.stringify(decoder.decode(chunk, { stream: true })));
  }
}

await timed("chunked", "/api/stream/chunked/3?intervalMs=400");
await timed("ndjson", "/api/stream/ndjson/3?intervalMs=300");
await timed("json-array", "/api/stream/json-array/3?intervalMs=300");

=== chunked: 200 text/plain; charset=utf-8 transfer-encoding=chunked
  +341ms "chunk 1/3 sent at 12:46:34.121\n"
  +739ms "chunk 2/3 sent at 12:46:34.521\n"
  +1137ms "chunk 3/3 sent at 12:46:34.922\n"
=== ndjson: 200 application/x-ndjson transfer-encoding=chunked
  +123ms "{\"id\":1,\"name\":\"Alpha-001\",\"value\":3.14,\"even\":false}\n"
  +418ms "{\"id\":2,\"name\":\"Bravo-002\",\"value\":6.28,\"even\":true}\n"
  +721ms "{\"id\":3,\"name\":\"Charlie-003\",\"value\":9.42,\"even\":false}\n"
=== json-array: 200 application/json transfer-encoding=chunked
  +32ms "[\n  {\"id\":1,\"name\":\"Alpha-001\",\"value\":3.14,\"even\":false},\n"
  +333ms "  {\"id\":2,\"name\":\"Bravo-002\",\"value\":6.28,\"even\":true},\n"
  +640ms "  {\"id\":3,\"name\":\"Charlie-003\",\"value\":9.42,\"even\":false}\n]\n"

All three arrive in pieces, at the interval each request asked for: about 400 ms apart for the chunked text, 300 ms for the other two. A client that calls await res.json() or await res.text() waits for the last piece and sees none of that.

NDJSON: one JSON object per line

Newline-delimited JSON, content type application/x-ndjson, is the easiest streaming format to consume: every line is a complete JSON document. Read the body as it comes, split on newlines, and parse each complete line:

// Parse NDJSON line by line as it arrives, not after the whole body
const t0 = performance.now();
const res = await fetch("https://api.testingapis.com/api/stream/ndjson/4?intervalMs=250");
const decoder = new TextDecoder();
let buffer = "";
for await (const chunk of res.body) {
  buffer += decoder.decode(chunk, { stream: true });
  let newline;
  while ((newline = buffer.indexOf("\n")) >= 0) {
    const line = buffer.slice(0, newline).trim();
    buffer = buffer.slice(newline + 1);
    if (line) console.log(`ndjson line at +${Math.round(performance.now() - t0)}ms:`, JSON.parse(line));
  }
}

ndjson line at +151ms: { id: 1, name: 'Alpha-001', value: 3.14, even: false }
ndjson line at +398ms: { id: 2, name: 'Bravo-002', value: 6.28, even: true }
ndjson line at +649ms: { id: 3, name: 'Charlie-003', value: 9.42, even: false }
ndjson line at +904ms: { id: 4, name: 'Delta-004', value: 12.57, even: true }

Each record is usable about a quarter of a second after the one before, as it's sent. The bufferis the important part. Network chunks don't line up with lines: one chunk can hold half a line, or three lines and a bit of a fourth. Code that calls JSON.parse on each chunk works on a fast local network, where chunks happen to match lines, and breaks in production. Test the parser with a stream whose lines are split across chunks.

A JSON array that arrives in pieces

The third shape in the output above is a normal JSON array, valid only once it's complete, sent element by element.JSON.parsecan't read it until the closing bracket arrives, so a client either waits for the whole thing or uses a streaming JSON parser that emits elements as they complete. Servers choose this shape because old clients can still read it as plain JSON. When you test one, check both: that a simple client gets a valid array at the end, and that a streaming client gets each element as it's sent.

Timeouts for slow bodies

A streamed response breaks the usual idea of a timeout. The headers arrive quickly, so a timeout that only covers "until the response starts" never fires, while the body can trickle in for minutes. The drip endpoint is built for testing this: it sends headers at once and spreads a few bytes over a chosen time. Five bytes over two seconds arrived as a byte at 36 ms, then the rest over the next 2.4 seconds.

// A whole-request deadline: the drip endpoint sends headers at once and the body slowly
const controller = new AbortController();
const t0 = performance.now();
const timer = setTimeout(() => controller.abort(new Error("no complete body within 1000 ms")), 1000);
try {
  const res = await fetch("https://api.testingapis.com/api/stream/drip?durationMs=3000&bytes=6", { signal: controller.signal });
  console.log(`drip headers at +${Math.round(performance.now() - t0)}ms, status ${res.status}`);
  const body = await res.text();
  console.log("drip body", body);
} catch (e) {
  console.log(`drip aborted at +${Math.round(performance.now() - t0)}ms:`, e.message);
} finally {
  clearTimeout(timer);
}

drip headers at +156ms, status 200
drip aborted at +1007ms: no complete body within 1000 ms

The headers came back in well under a second with a 200, and the deadline still caught the slow body, because the abort signal covers reading the body too, not just the first byte. Many HTTP clients have separate settings for connect time, time to first byte and time between bytes, and the defaults differ between libraries. Find out which ones your client applies, and test each against a slow stream like this one.

Streaming test checklist

• Data arrives in pieces at the expected rate, through every proxy in front of the API.
• Parsers handle records split across network chunks.
• Clients that wait for the whole body still get valid data at the end.
• Deadlines cover the whole body, and a stalled stream fails cleanly.
• Closing the connection early stops the server producing more, rather than letting it finish for nobody.