Skip to content

test: fix flaky test-runner coverage threshold test#64051

Open
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage
Open

test: fix flaky test-runner coverage threshold test#64051
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage

Conversation

@trivikr

@trivikr trivikr commented Jun 21, 2026

Copy link
Copy Markdown
Member

This fixes a flaky assertion in parallel/test-runner-run-coverage.mjs.

The threshold subtest was asserting that process.exitCode was not already
1 before starting its coverage run. That value is process-global, so it
can be affected by other run() coverage activity in the same process.

The test now clears process.exitCode before the threshold check and
restores the original value with a cleanup hook.

Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20is%3Aissue%20state%3Aopen%20%22test-runner-run-coverage%22

Example
not ok 3959 parallel/test-runner-run-coverage
  ---
  duration_ms: 710.47400
  severity: fail
  exitcode: 1
  stack: |-
    Test failure: 'should run with coverage and fail when below line threshold'
    Location: test/parallel/test-runner-run-coverage.mjs:157:11
    AssertionError [ERR_ASSERTION]: Expected "actual" to be strictly unequal to: 1
        at TestContext.<anonymous> (file:///home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-runner-run-coverage.mjs:160:14)
        at Test.runInAsyncScope (node:async_hooks:226:14)
        at Test.run (node:internal/test_runner/test:1382:25)
        at Suite.processPendingSubtests (node:internal/test_runner/test:960:18)
        at Test.postRun (node:internal/test_runner/test:1522:19)
        at Test.run (node:internal/test_runner/test:1447:12)
        at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
        at async Suite.processPendingSubtests (node:internal/test_runner/test:960:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: 1,
      operator: 'notStrictEqual',
      diff: 'simple'
    }

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Jun 21, 2026
@atlowChemi

Copy link
Copy Markdown
Member

@trivikr I think the commit should be under the test: submodule

Reset process.exitCode before running the coverage threshold check.
This lets the test verify the exit code set by that run() call without
depending on process-global state from earlier coverage runs.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
@trivikr trivikr force-pushed the test-runner-run-coverage branch from 1d1026a to d73ab26 Compare June 22, 2026 06:31
@trivikr trivikr changed the title test_runner: fix flaky run coverage threshold test test: fix flaky test-runner coverage threshold test Jun 22, 2026
@trivikr

trivikr commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

I think the commit should be under the test: submodule

Yes. Since the code changes are only in test files.
I've updated the commit message and requested a review.

@trivikr trivikr requested a review from atlowChemi June 22, 2026 06:32
@trivikr trivikr added flaky-test Issues and PRs related to the tests with unstable failures on the CI. request-ci Add this label to start a Jenkins CI on a PR. labels Jun 22, 2026
@trivikr trivikr added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 22, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 22, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@trivikr trivikr added commit-queue Add this label to land a pull request using GitHub Actions. fast-track PRs that do not need to wait for 72 hours to land. labels Jun 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Fast-track has been requested by @trivikr. Please 👍 to approve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. commit-queue Add this label to land a pull request using GitHub Actions. fast-track PRs that do not need to wait for 72 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants