Skip to content

fix(tests): eliminate flaky timers and silent false positives#75

Merged
robertsLando merged 4 commits intomasterfrom
fix/flaky-request-manager-tests
Mar 3, 2026
Merged

fix(tests): eliminate flaky timers and silent false positives#75
robertsLando merged 4 commits intomasterfrom
fix/flaky-request-manager-tests

Conversation

@robertsLando
Copy link
Member

@robertsLando robertsLando commented Mar 3, 2026

Summary

Test fixes

  • RequestManager unit tests: replaced real setTimeout/wait() with t.mock.timers (Node.js mock timers API) so all timing is fully deterministic — eliminates the root cause of flaky failures where the 100ms test delay matched the floor in #scheduleClear()
  • Unit + integration tests: replaced try/catch patterns with assert.rejects() to prevent silent false positives (previously, if the promise resolved instead of rejecting, the test passed with zero assertions)
  • Integration tests: moved client.close() outside the catch block so it always runs, preventing socket leaks

CI updates

  • Node matrix: [20, 22][22, 24] across all workflows
  • Bumped all GitHub Actions to latest major versions:
    • actions/checkout v4 → v6
    • actions/setup-node v4 → v6
    • actions/upload-artifact v4 → v7
    • actions/download-artifact v4 → v8
    • actions/upload-pages-artifact v3 → v4
    • github/codeql-action v3 → v4
    • coverallsapp/github-action v2.3.4 → v2.3.6
  • Simplified publish.yml by removing unnecessary single-entry matrix

Closes #73

Test plan

  • request-manager.spec.ts passes 50/50 consecutive runs with zero failures
  • Full unit suite passes (182 tests)
  • Full integration suite passes (33 tests)
  • CI workflows pass on PR (Node 22 + 24 matrix)

🤖 Generated with Claude Code

… positives

RequestManager unit tests failed intermittently because the 100ms test
delay matched the floor in #scheduleClear(), causing timer jitter to
double the effective timeout. Replace all real-timer waits with
t.mock.timers for fully deterministic execution.

Also replace try/catch patterns (that silently passed when the promise
resolved instead of rejecting) with assert.rejects() across both unit
and integration tests, and ensure client.close() always runs.

Closes #73

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 3, 2026 16:41
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves the reliability of the Node.js test suite by removing timing-dependent waits in RequestManager unit tests and by replacing try/catch rejection assertions with assert.rejects() to avoid silent false positives.

Changes:

  • Refactors test/unit/request-manager.spec.ts to use t.mock.timers for deterministic timeout behavior.
  • Updates multiple integration tests to use assert.rejects() instead of try/catch patterns that could pass with zero assertions.
  • Adjusts integration test cleanup flow to call client.close() after assertions (but still needs unconditional cleanup on assertion failure).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
test/unit/request-manager.spec.ts Replaces real timer waits with Node mock timers to eliminate flakiness and make timeout behavior deterministic.
test/integration/write-property.spec.ts Uses assert.rejects() to ensure rejection is actually asserted in timeout case.
test/integration/write-property-multiple.spec.ts Uses assert.rejects() for deterministic rejection assertions.
test/integration/read-property.spec.ts Uses assert.rejects() to prevent silent false positives on expected timeouts.
test/integration/read-property-multiple.spec.ts Uses assert.rejects() for the timeout test case; keeps existing decode tests.
test/integration/add-list-element.spec.ts Uses assert.rejects() to assert expected timeout rejection.
test/integration/acknowledge-alarm.spec.ts Uses assert.rejects() to assert expected timeout rejection.
Comments suppressed due to low confidence (1)

test/integration/write-property-multiple.spec.ts:32

  • client.close() is executed only after assert.rejects(...) completes successfully. If the assertion fails, the test will throw before closing the client and may leak an open socket into later tests. Ensure the close runs unconditionally via try/finally or t.after(() => client.close()).
		await assert.rejects(
			client.writePropertyMultiple({ address: '127.0.0.2' }, values, {}),
			(err: Error) => {
				assert.strictEqual(err.message, 'ERR_TIMEOUT')
				return true
			},
		)
		client.close()
	})
})

robertsLando and others added 2 commits March 3, 2026 17:47
- Node matrix: [20, 22] → [22, 24] across CI, test, and publish workflows
- actions/checkout v4 → v6
- actions/setup-node v4 → v6
- actions/upload-artifact v4 → v7
- actions/download-artifact v4 → v8
- actions/upload-pages-artifact v3 → v4
- actions/configure-pages stays at v5 (already latest)
- actions/deploy-pages stays at v4 (already latest)
- github/codeql-action v3 → v4
- coverallsapp/github-action v2.3.4 → v2.3.6
- publish.yml: remove single-entry matrix, use node 22 directly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Addresses Copilot review: client.close() now runs via t.after()
regardless of assertion outcome, preventing UDP socket leaks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@robertsLando robertsLando merged commit cd8704a into master Mar 3, 2026
8 checks passed
@robertsLando robertsLando deleted the fix/flaky-request-manager-tests branch March 3, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: flaky tests

2 participants