Anatomy of a request

In the previous lesson you sent a request by typing a URL into a browser. The browser filled in everything else for you. This lesson takes a request apart so you know what each piece is, because every test you write will set some of these pieces on purpose and check how the server reacts.

An HTTP request has four parts:

• a method, which says what kind of action you want
• a URL, which says which thing on which server you want it done to
• headers, which carry extra information about the request
• an optional body, which carries data you are sending

What a request looks like on the wire

HTTP/1.1 is a text protocol. Before any encryption, a request is a few lines of plain text. This is the request curl sends for https://api.testingapis.com/api/echo?color=red. You can see it yourself by running curl with -v (verbose): the lines that start with > are what went out.

GET /api/echo?color=red HTTP/1.1
Host: api.testingapis.com
User-Agent: curl/8.18.0
Accept: */*

The first line is the request line: method, the path with its query string, and the protocol version. Each line after that is a header, a name and a value separated by a colon. An empty line ends the headers. If there were a body, it would come after that empty line. Newer versions of HTTP (HTTP/2 and HTTP/3) send the same information in a binary format, but the parts are identical, and every tool still shows them to you in this text form.

Seeing what the server received

The easiest way to learn requests is to send one to a server that tells you what it got. /api/echo does exactly that. It accepts any method and replies with a description of your request. Send this one:

Here is the reply from curl, trimmed to the interesting fields. If you send it from the page, the headers will be your browser's instead.

{
  "method": "GET",
  "url": "http://api.testingapis.com/api/echo?color=red&size=10",
  "path": "/api/echo",
  "queryString": "?color=red&size=10",
  "query": { "color": "red", "size": "10" },
  "headers": {
    "Accept": "*/*",
    "Host": "api.testingapis.com",
    "User-Agent": "curl/8.18.0",
    "Accept-Encoding": "gzip, br",
    "Cf-Ipcountry": "IN",
    "X-Forwarded-For": "203.0.113.7, 172.71.198.178, 10.30.107.46",
    "X-Forwarded-Proto": "https"
  },
  "contentType": null,
  "contentLength": 0,
  "body": null
}

Two things in there deserve a second look. The server saw the URL as http://, not https://. And it received headers you never sent, like Cf-Ipcountry and X-Forwarded-For. Your request went through a CDN and a load balancer before it reached the application, and each of them rewrote or added something. The first address in X-Forwarded-For is yours (the one above is a placeholder). That is normal in production systems, and it is the reason an echo endpoint is useful: it shows what arrived, not what you think you sent.

The method

The method is the verb. GET asks for data. POST sends data to create something. PUT, PATCH and DELETE change or remove things. A browser address bar can only send GET, which is why you need other tools for the rest. Methods get a whole lesson of their own, HTTP methods, so for now it is enough to know that the same URL can do different things depending on the method.

The URL

Break this URL into its parts:

https://api.testingapis.com/api/Products?category=books&limit=5

URLs can also carry a port (:8080 after the host) and a fragment (#section at the end). You will see ports a lot on test and local environments. Fragments never reach the server at all: the browser keeps them.

Path parameters and query parameters

Values can travel in the path or in the query string. In /api/Products/1, the 1 is a path parameter. It identifies one specific product, and the documentation writes the path as /api/Products/{id}. In /api/Products?limit=5, limit is a query parameter. Query parameters usually filter, sort or page through results, and they are usually optional. That split is a convention, not a rule, so check the docs for each API.

Paths are often case-sensitive and query parameter names usually are too. That depends on the server framework, so run a quick test on any new API: send /api/products instead of /api/Products and see what happens.

Percent-encoding

Some characters have a job inside a URL. ? starts the query string, & separates parameters, # starts the fragment, /separates path segments, and spaces aren't allowed at all. To send one of these as data, you replace it with % and its hexadecimal byte value. A space becomes %20 and & becomes %26. This is called percent-encoding, or URL encoding.

Say you want to search for "red shoes" and filter by a tag whose value is the literal text a&b. Encoded, it works:

{
  "queryString": "?q=red%20shoes&tag=a%26b",
  "query": { "q": "red shoes", "tag": "a&b" }
}

Now send the tag without encoding the ampersand:

{
  "queryString": "?tag=a&b",
  "query": { "tag": "a", "b": "" }
}

The server now sees two parameters: tag with the value a, and an empty parameter called b. No error, no warning. The request "worked" and the data is wrong. This is a real class of bug in clients that build URLs by gluing strings together, and it is why search fields are a good place to type ampersands, plus signs, slashes and non-English text during testing. The URL encoder on this site does the conversion if you need it by hand.

Headers

Headers are name and value pairs that describe the request. Names are case-insensitive, so content-type and Content-Type mean the same thing. Most clients add several headers automatically. The ones you will set or check most often are these:

Accept matters when an API can answer in more than one format. /api/formats/negotiatesign in to see returns the same user as JSON by default and as XML if you ask for it:

Compare the XML with the JSON you get without the header. Both carry the same seven fields, written in each format's own way: the roles list becomes repeated role elements, and the null manager becomes an empty element marked xsi:nil. When an API offers several formats, test each one and don't assume they match. When this lesson was first written, the XML version of this user was missing three fields.

Authorization is covered properly in Authentication and authorization. One thing to know already: never paste real tokens into screenshots, bug reports or shared collections. They are passwords.

The body

GET requests normally have no body. POST, PUT and PATCH usually do, because they send data to the server. The body can be any format, and the Content-Type header tells the server how to read it. For most APIs today the body is JSON. This sends a small JSON object to the echo endpoint with curl:

curl -X POST https://api.testingapis.com/api/echo \
  -H "Content-Type: application/json" \
  -d '{"title":"Dune","year":1965}'

{
  "method": "POST",
  "path": "/api/echo",
  "query": {},
  "contentType": "application/json",
  "contentLength": 28,
  "body": "{\"title\":\"Dune\",\"year\":1965}",
  "json": { "title": "Dune", "year": 1965 }
}

The server read the header, saw application/json, and parsed the 28 bytes into an object. You can send the same request from here:

{"title":"Dune","year":1965}

When Content-Type and the body disagree

Now send the same kind of body with curl but leave out the header:

curl -X POST https://api.testingapis.com/api/echo \
  -d '{"title":"Dune"}'

{
  "contentType": "application/x-www-form-urlencoded",
  "contentLength": 16,
  "body": "{\"title\":\"Dune\"}",
  "json": null,
  "form": { "{\"title\":\"Dune\"}": "" }
}

curl's -d option defaults to application/x-www-form-urlencoded, the format HTML forms use. The server believed the header, tried to read the body as a form, and ended up with one form field whose name is the entire JSON text. No error came back. Many real servers behave the same way, and others return 415 Unsupported Media Type. A missing or wrong Content-Typeis one of the most common reasons a request "looks right" and still fails, so it is the first thing to check when a server ignores your data.

If the header is right but the JSON is broken, a good server says so. Here is part of the echo response for the body {"title":"Dune",}, which has a trailing comma:

"json": null,
"jsonError": "The JSON object contains a trailing comma at the end which is not supported in this mode. Change the reader options. LineNumber: 0 | BytePositionInLine: 16."