Skip to content

E2E: parallel workers#7309

Open
phyllis-sy-wu wants to merge 1 commit intopsyw-0413-E2E-store-isolationfrom
psyw-0414-E2E-parallel-workers
Open

E2E: parallel workers#7309
phyllis-sy-wu wants to merge 1 commit intopsyw-0413-E2E-store-isolationfrom
psyw-0414-E2E-parallel-workers

Conversation

@phyllis-sy-wu
Copy link
Copy Markdown
Contributor

@phyllis-sy-wu phyllis-sy-wu commented Apr 15, 2026

Addressed #7268

WHY are these changes introduced?

E2E tests currently run sequentially with 1 worker, making the suite slow (~25 min on CI). With per-test store creation (PR #7279), each store-dependent test adds ~30s of overhead, compounding the problem.

WHAT is this pull request doing?

Enables parallel test execution in playwright.config.ts:

  • fullyParallel: true — tests run concurrently across workers
  • workers: 3 — 3 parallel Playwright workers
  • globalTimeout: 20 * 60 * 1000 — reduced from 35 min to 20 min (parallel execution is much faster)
  • GitHub Actions timeout-minutes reverted from 40 back to 20 (no longer needs the temporary bump from E2E: temporary time limit bump #7282)

Why 3 workers, not more? Tested with 4 and 5 workers, but the app-scaffold test consistently failed on CI (fine locally). That test uses npm (not pnpm) with the full reactRouter template to mimic real user behaviour. On CI runners (2-core Ubuntu), 4+ concurrent workers cause npm install to fail due to resource contention (network/disk I/O). 3 workers is the sweet spot — still a significant speedup while keeping npm-based tests reliable.

How to test your changes?

# Run full suite — should complete in ~8-12 min instead of ~25 min
pnpm --filter e2e exec playwright test

# Verify no port conflicts with headed mode
E2E_HEADED=1 DEBUG=1 pnpm --filter e2e exec playwright test

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing, so I've added a changelog entry with pnpm changeset add

Copy link
Copy Markdown
Contributor Author

phyllis-sy-wu commented Apr 15, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0413-E2E-store-isolation branch from dbcae8d to 898e174 Compare April 15, 2026 14:33
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0414-E2E-parallel-workers branch from 6c00722 to 0cf54f5 Compare April 15, 2026 14:33
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0413-E2E-store-isolation branch from 898e174 to fd1b216 Compare April 15, 2026 17:05
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0414-E2E-parallel-workers branch 3 times, most recently from 3af2880 to 6e9781e Compare April 15, 2026 18:17
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0413-E2E-store-isolation to graphite-base/7309 April 15, 2026 18:25
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0414-E2E-parallel-workers branch from 6e9781e to 50dc619 Compare April 15, 2026 18:35
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7309 to psyw-0413-E2E-store-isolation April 15, 2026 18:35
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0414-E2E-parallel-workers branch from 50dc619 to aa94ef3 Compare April 15, 2026 18:48
@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/analytics.d.ts
@@ -23,7 +23,6 @@ interface EnvironmentData {
     env_auth_method: string;
     env_is_wsl: boolean;
     env_build_repository: string;
-    env_auto_upgrade_enabled: boolean | null;
 }
 export declare function getEnvironmentData(config: Interfaces.Config): Promise<EnvironmentData>;
 export declare function getSensitiveEnvironmentData(config: Interfaces.Config): Promise<{
packages/cli-kit/dist/public/node/metadata.d.ts
@@ -34,7 +34,7 @@ export type SensitiveSchema<T> = T extends RuntimeMetadataManager<infer _TPublic
  * @returns A container for the metadata.
  */
 export declare function createRuntimeMetadataContainer<TPublic extends AnyJson, TSensitive extends AnyJson = Record<string, never>>(defaultPublicMetadata?: Partial<TPublic>): RuntimeMetadataManager<TPublic, TSensitive>;
-type CmdFieldsFromMonorail = PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'> & PickByPrefix<MonorailEventPublic, 'env_auto_upgrade_'>;
+type CmdFieldsFromMonorail = PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'>;
 declare const coreData: RuntimeMetadataManager<CmdFieldsFromMonorail, {
     commandStartOptions: {
         startTime: number;
packages/cli-kit/dist/public/node/monorail.d.ts
@@ -2,7 +2,7 @@ import { JsonMap } from '../../private/common/json.js';
 import { DeepRequired } from '../common/ts/deep-required.js';
 export { DeepRequired };
 type Optional<T> = T | null;
-export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.21";
+export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.20";
 export interface Schemas {
     [MONORAIL_COMMAND_TOPIC]: {
         sensitive: {
@@ -46,10 +46,6 @@ export interface Schemas {
             cmd_all_timing_network_ms?: Optional<number>;
             cmd_all_timing_prompts_ms?: Optional<number>;
             cmd_all_timing_active_ms?: Optional<number>;
-            env_auto_upgrade_enabled?: Optional<boolean>;
-            env_auto_upgrade_accepted?: Optional<boolean>;
-            env_auto_upgrade_skipped_reason?: Optional<string>;
-            env_auto_upgrade_success?: Optional<boolean>;
             cmd_extensions_binary_from_source?: Optional<boolean>;
             cmd_scaffold_required_auth?: Optional<boolean>;
             cmd_scaffold_template_custom?: Optional<boolean>;
packages/cli-kit/dist/public/node/upgrade.d.ts
@@ -2,7 +2,7 @@
  * Utility function for generating an install command for the user to run
  * to install an updated version of Shopify CLI.
  *
- * @returns A string with the command to run, or undefined if the package manager cannot be determined.
+ * @returns A string with the command to run.
  */
 export declare function cliInstallCommand(): string | undefined;
 /**

@phyllis-sy-wu phyllis-sy-wu marked this pull request as ready for review April 15, 2026 19:05
@phyllis-sy-wu phyllis-sy-wu requested a review from a team as a code owner April 15, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2E: Parallel test execution with multiple workers for faster CI

3 participants