Running REST Assured tests in CI

A test suite earns its keep when it runs without anyone deciding to run it: on every commit, on every pull request, and on a schedule. REST Assured tests are ordinary JUnit tests run by Maven, so any CI system that can run mvn test can run them. This lesson covers the pieces that make them work well there: groups of tests, exit codes, reports, secrets, and a complete GitHub Actions workflow.

Smoke tests with tags

Not every test needs to run every time. A few fast tests that touch the most important endpoints make a smoke suite, good for a check after each deployment. JUnit marks them with @Tag:

@Test
@Tag("smoke")
void healthCheck() {
    given()
        .spec(api())
    .when()
        .get("/api/health")
    .then()
        .spec(fastResponse())
        .statusCode(200)
        .body("status", equalTo("healthy"));
}

And Maven runs only those with -Dgroups:

$ mvn test -Dgroups=smoke
[INFO] Running com.testingapis.examples.FirstTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.896 s -- in com.testingapis.examples.FirstTest
[INFO] Running com.testingapis.examples.SpecsTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.210 s -- in com.testingapis.examples.SpecsTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

Two tests in about three seconds, out of 23. -DexcludedGroups=slow does the opposite, which is handy for keeping long tests out of the commit build. Tags are free text, so agree on a few names as a team and stick to them.

Exit codes

CI decides pass or fail from the exit code of each step. Maven exits with 0 when every test passed and 1 when any failed, so a failing test fails the build without any extra setup:

$ mvn test -Dgroups=smoke
...
$ echo $?
0

$ mvn test            # with a failing test
[ERROR] Tests run: 23, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURE
$ echo $?
1

Resist the options that turn this off, such as -Dmaven.test.failure.ignore=true. A pipeline that stays green while tests fail trains everyone to stop looking at it.

Reports

Surefire writes two files per test class:

target/surefire-reports/
├── com.testingapis.examples.AuthTest.txt
├── TEST-com.testingapis.examples.AuthTest.xml
├── com.testingapis.examples.FirstTest.txt
├── TEST-com.testingapis.examples.FirstTest.xml
└── ...

The .txt files are for people. The TEST-*.xmlfiles are JUnit XML, which nearly every CI system can turn into a list of passed and failed tests on the build page. The project's usePhrasedTestCaseMethodName setting, from lesson 7, makes those names readable, so a failure shows up as product 9999 does not exist rather than [3]. Combined with the failure logging from lesson 6, a failed test in CI comes with the request and response that caused it.

Servers and secrets

Nothing secret lives in the repository. The key reaches the tests through the environment, and only ApiSpecsreads it. Remember that failure logs print request headers, keys included, so either mask that header in REST Assured's log settings or make sure build logs aren't public.

A GitHub Actions workflow

# .github/workflows/api-tests.yml
name: API tests

on:
  push:
  pull_request:
  schedule:
    - cron: "0 6 * * *"   # every morning, to catch changes on the API side

jobs:
  rest-assured:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 21
          cache: maven

      - name: Smoke tests
        run: mvn -B test -Dgroups=smoke
        env:
          TESTINGAPIS_KEY: ${{ secrets.TESTINGAPIS_KEY }}

      - name: Full suite
        run: mvn -B test
        env:
          TESTINGAPIS_KEY: ${{ secrets.TESTINGAPIS_KEY }}

      - name: Keep the reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: surefire-reports
          path: target/surefire-reports/

The workflow does five things:

• setup-java installs Java 21 and caches Maven's downloads between runs, so builds after the first are quicker.
• The smoke tests run first and fail fast if the API is down, before the full suite spends minutes finding out.
• The key comes from the repository's secrets, and GitHub hides it in the logs.
• The reports are uploaded even when tests fail, thanks to if: always(), which is exactly when you need them.
• The daily schedule catches changes on the API side, with no commit in your repository to trigger a run.

Other CI systems work the same way: install Java, run mvn -B test with the secrets in the environment, and hand the JUnit XML to the test report feature. -Bis batch mode, which keeps Maven's output plain and free of progress animations in logs.

CI checklist

• A smoke group that runs in seconds, and the full suite after it.
• The server and the key come from the environment; nothing secret is committed.
• Test failures fail the build; nothing ignores them.
• JUnit XML reports are published, with readable names, and kept when tests fail.
• A scheduled run catches changes on the API side.