Testing strategy inside the pipeline

Layer unit, integration and end-to-end tests, use service containers for real dependencies, handle flaky tests honestly, split and parallelise, and publish results.

Which test runs when

jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: npm run lint
      - run: npm run test:unit -- --reporter=junit --output=reports/unit.xml
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: unit-results
          path: reports/

  integration:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: ci
        ports: ['5432:5432']
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s --health-timeout 5s --health-retries 10
      redis:
        image: redis:7
        ports: ['6379:6379']
    env:
      DATABASE_URL: postgres://postgres:ci@localhost:5432/postgres
      REDIS_URL: redis://localhost:6379
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run migrate
      - run: npm run test:integration

• A service container needs a health check, otherwise the first test connects before the database is accepting connections.
• Use a real database engine in integration tests. An in-memory substitute with different semantics lets bugs through.
• Each job gets a clean database; sharing one across parallel test files creates ordering dependencies that surface as random failures.
• Upload results with if: always(), or a failing suite produces no report at exactly the moment you need one.

Flaky tests

# retry at the job level, not inside the test, so retries are visible
test:e2e:
  runs-on: ubuntu-latest
  continue-on-error: false
  strategy:
    fail-fast: false
  steps:
    - uses: actions/checkout@v4
    - run: npm ci
    - run: npm run test:e2e
      continue-on-error: true        # first attempt
    - id: retry
      if: steps.first.outcome == 'failure'
      run: npm run test:e2e

# publish the JUnit report so the flaky test is named, not buried
    - if: always()
      uses: actions/upload-artifact@v4
      with:
        name: e2e-results
        path: reports/e2e.xml

Splitting and reporting

# split by timing so each shard takes a similar time
npx jest --shard=1/4 --reporters=jest-junit
npx jest --shard=2/4 --reporters=jest-junit

# pytest: distribute by test duration from the previous run
pytest -n auto --dist loadfile --junitxml=reports/junit.xml

# once, record durations so the split is informed rather than arbitrary
pytest --store-durations --durations-path=.test_durations

Run only the tests affected by a change once the suite is large. It requires a reliable dependency graph, so start with splitting by duration and add affected-only runs when the graph is trustworthy.

FAQ

Related

Triggers, concurrency and matrix builds Observability and speeding up slow pipelines

Last refreshed 2026-09-18.