Running collections with Newman in CI

A collection that only runs when someone remembers to open Postman catches bugs late. Newman, Postman's open-source command-line runner, runs an exported collection from a terminal, so the same tests can run on every commit, every deployment, or every morning, without anyone clicking anything. Every run shown in this tutorial was a Newman run, so its output is the output you'll get.

Installing and running

Newman is an npm package and needs Node.js. This tutorial used version 6.2.3:

$ npm install --global newman
$ newman --version
6.2.3

# or, without installing anything
$ npx newman@6 run testingapis-postman-collection.json

It runs the same JSON files Postman exports: choose Export on a collection or an environment in Postman, or use the tutorial's downloads directly. A full run of the tutorial collection:

$ newman run testingapis-postman-collection.json -e testingapis-postman-environment.json

...
│                requests │                27 │                 0 │
│            test-scripts │                24 │                 0 │
│              assertions │                29 │                 0 │
│ total run duration: 8.2s                                        │

$ echo $?
0

Newman needs no Postman account; it only reads the files. Postman also has its own command-line tool, the Postman CLI, which is designed around collections stored in a Postman account. Either can run in CI. This tutorial uses Newman because it works with plain files and nothing else.

Exit codes

What makes Newman useful in CI is its exit code: 0 when every test passed, 1 when anything failed. CI systems mark a step as failed on any non-zero exit code, so a failing test fails the build. Here is a run with a bad row in the data file:

$ newman run testingapis-postman-collection.json -e testingapis-postman-environment.json \
    --folder "06 Data-driven" -d bad.csv

 1.  AssertionError  product 2 is in books
     iteration: 2    expected 'electronics' to deeply equal 'books'
                     at assertion:1 in test-script
                     inside "06 Data-driven / Product by ID, from the data file"

$ echo $?
1

Failures are listed at the end, numbered, with the request and the iteration they happened in. That last block is what someone reads first in a failed build log, which is why test names matter so much: a CI log shows nothing else.

The options you'll use

Secrets from the command line

--env-var is how secrets reach a run without being stored in a file. The environment file keeps apiKey empty, and the CI system passes the real key at run time. Command-line values override the file:

$ newman run testingapis-postman-collection.json \
    --env-var "baseUrl=https://api.testingapis.com" \
    --folder "02 Variables"

  √  the product matches the productId variable
  │ 'baseUrl resolved to', 'https://api.testingapis.com'

Reports

CI systems understand JUnit XML, the test report format from the Java world. Most of them turn it into a list of passed and failed tests on the build page, with history:

$ newman run testingapis-postman-collection.json -e testingapis-postman-environment.json \
    --reporters cli,junit --reporter-junit-export results/newman.xml

<testsuites name="TestingAPIs: Postman tutorial" tests="2" time="0.195">
  <testsuite name="06 Data-driven / Product by ID, from the data file" tests="2" failures="1" errors="0" time="0.098">
    <testcase name="product 1 returns 200" time="0.038" classname="06DataDrivenProductByIdFromTheDataFile"/>
    <testcase name="product 1 is in electronics" time="0.038" classname="06DataDrivenProductByIdFromTheDataFile"/>
    <testcase name="product 2 returns 200" time="0.038" classname="06DataDrivenProductByIdFromTheDataFile"/>
    <testcase name="product 2 is in books" time="0.038" classname="06DataDrivenProductByIdFromTheDataFile">
      <failure type="AssertionFailure" message="expected 'electronics' to deeply equal 'books'">
      ...

Each Postman test becomes a testcase, grouped by request, with the failure message attached. The json reporter writes everything, including full requests and responses, which is handy for debugging a failure that only happens in CI. Community reporters such as newman-reporter-htmlextraproduce an HTML page for people who'd rather not read XML.

A GitHub Actions workflow

Commit the exported collection, environment and data file to the repository, in a folder such as postman/, then add a workflow:

# .github/workflows/api-tests.yml
name: API tests

on:
  push:
  pull_request:
  schedule:
    - cron: "0 6 * * *"   # every morning, to catch changes on the API side too

jobs:
  postman:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Run the collection
        run: >
          npx newman@6 run postman/testingapis-postman-collection.json
          -e postman/testingapis-postman-environment.json
          --env-var "apiKey=${{ secrets.TESTINGAPIS_KEY }}"
          --reporters cli,junit
          --reporter-junit-export results/newman.xml

      - name: Run the data-driven checks
        run: >
          npx newman@6 run postman/testingapis-postman-collection.json
          -e postman/testingapis-postman-environment.json
          --env-var "apiKey=${{ secrets.TESTINGAPIS_KEY }}"
          --folder "06 Data-driven" -d postman/products.csv

      - name: Keep the report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: newman-results
          path: results/

The API key lives in the repository's secrets as TESTINGAPIS_KEY, and GitHub masks it in the logs. The schedule matters as much as the push trigger: an API can change underneath you without any commit to your repository, and a morning run is how you find out before your users do. if: always() keeps the report even when the tests fail, which is exactly when you need it. Other CI systems work the same way: a step that runs Newman, a secret passed with --env-var, and the JUnit file handed to the test report feature.

The REST tutorial's lesson on running tests in CI covers the strategy questions: which tests to run on every commit, which to run nightly, and what to do about flaky ones.

CI checklist

• The collection, environment and data files are in version control, exported from Postman after each change.
• No secret is in any committed file; CI passes them with --env-var.
• A failing test makes the build fail. Never add --suppress-exit-code to make a pipeline green.
• JUnit results are published, so failures show up as named tests, not a wall of log.
• A scheduled run catches changes on the API side.