Add Playwright flaky & metrics reporters/configs#400
Open
SteveChurchillSWO wants to merge 4 commits intomainfrom
Open
Add Playwright flaky & metrics reporters/configs#400SteveChurchillSWO wants to merge 4 commits intomainfrom
SteveChurchillSWO wants to merge 4 commits intomainfrom
Conversation
Introduce dedicated Playwright configs and reporters for flaky-test detection and tag-based metrics. Adds playwright.flaky.config.ts and playwright.metrics.config.ts, new reporters (flaky-test-reporter.ts, metrics-reporter.ts), tags-helper and report-utils helpers, and wiring to output reports to reports/. Updates e2etests package.json with new scripts (playwright:metrics, playwright:flaky) and adds @reportportal/agent-js-playwright as a dependency. Ignore reports/ in .gitignore and apply related adjustments across many e2etests specs to integrate the reporting/metrics workflow.
Import TAGS_MODULE, TAGS_SPEED, TAGS_TYPE and TAGS_DEBUG from tags-helper and remove the duplicated local tag definitions in metrics-reporter.ts. Updated the comment to note tags are sourced from tags-helper (single source of truth). This is a refactor to centralize tag definitions and avoid duplication; no functional changes intended.
dwolosz
reviewed
Apr 17, 2026
| }); | ||
| } else if (failures > 0 && passes === 0) { | ||
| // ── Consistently failing ────────────────────────────────────────── | ||
| console.log(`❌ FAILED: ${test.title} (${failures}/${totalRuns} failed)`); |
There was a problem hiding this comment.
Do you want to show console.log here or should we use debugLog instead ? If use debugLog please apply for whole commited code.
| * Usage — add to any playwright config's reporter array: | ||
| * reporter: [ | ||
| * ['list'], | ||
| * ['./reporting/flaky-test-reporter.ts'], |
There was a problem hiding this comment.
if all reports are moved to reporting maybe its good idea to default test results (playwright-report, test-results) to /report folder as well, then all will be in one place
Print process.env.ENVIRONMENT in global setup for easier environment debugging. Import LARGE_DATA_TIMEOUT in expenses-tests.spec.ts and pass it to page.waitForResponse for the /pools_expenses/ request to handle large payloads and reduce flaky timeouts during heavy data loads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Introduce dedicated Playwright configs and reporters for flaky-test detection and tag-based metrics. Adds playwright.flaky.config.ts and playwright.metrics.config.ts, new reporters (flaky-test-reporter.ts, metrics-reporter.ts), tags-helper and report-utils helpers, and wiring to output reports to reports/. Updates e2etests package.json with new scripts (playwright:metrics, playwright:flaky) and adds @reportportal/agent-js-playwright as a dependency. Ignore reports/ in .gitignore and apply related adjustments across many e2etests specs to integrate the reporting/metrics workflow.
Related issue number
https://softwareone.atlassian.net/browse/MPT-20249