TEST-INFRA: [ci-execution-time] Parallelize E2E tests for faster CI #2298

New issue

Closed

opened 2026-04-03 13:21:56 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 13:21:56 +00:00

Owner

Metadata

Branch: test/m3-parallelize-e2e-tests
Commit Message: test(ci): parallelize E2E test suite using matrix sharding strategy
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The E2E test suite is likely a significant contributor to the overall CI execution time. Running the entire suite sequentially can create a bottleneck in the CI/CD pipeline, leading to longer wait times for developers.

Expected Behavior

The E2E test suite should be split into multiple independent shards and executed in parallel across multiple CI runners, dramatically reducing the overall execution time of the E2E test job.

Acceptance Criteria

The E2E test suite is successfully parallelized using a matrix sharding strategy.
The overall execution time of the e2e_tests job is measurably and significantly reduced.
The CI pipeline remains stable and reliable with no flaky test failures introduced by parallelization.
The test runner accepts a shard number and total shards as input parameters.

Subtasks

Analyze the E2E test suite to identify logical groups of tests that can be run independently.
Update the .forgejo/workflows/ci.yml file to configure a matrix strategy for the e2e_tests job.
Configure the test runner to accept a shard number and total shards as input.
Validate that all shards produce correct results and no tests are skipped or duplicated.
Monitor the performance of the parallelized E2E tests to ensure they are running efficiently.
Run nox (all default sessions), fix any errors.
Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(ci): parallelize E2E test suite using matrix sharding strategy), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/m3-parallelize-e2e-tests).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/m3-parallelize-e2e-tests` - **Commit Message**: `test(ci): parallelize E2E test suite using matrix sharding strategy` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The E2E test suite is likely a significant contributor to the overall CI execution time. Running the entire suite sequentially can create a bottleneck in the CI/CD pipeline, leading to longer wait times for developers. ## Expected Behavior The E2E test suite should be split into multiple independent shards and executed in parallel across multiple CI runners, dramatically reducing the overall execution time of the E2E test job. ## Acceptance Criteria - The E2E test suite is successfully parallelized using a matrix sharding strategy. - The overall execution time of the `e2e_tests` job is measurably and significantly reduced. - The CI pipeline remains stable and reliable with no flaky test failures introduced by parallelization. - The test runner accepts a shard number and total shards as input parameters. ## Subtasks - [ ] Analyze the E2E test suite to identify logical groups of tests that can be run independently. - [ ] Update the `.forgejo/workflows/ci.yml` file to configure a matrix strategy for the `e2e_tests` job. - [ ] Configure the test runner to accept a shard number and total shards as input. - [ ] Validate that all shards produce correct results and no tests are skipped or duplicated. - [ ] Monitor the performance of the parallelized E2E tests to ensure they are running efficiently. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(ci): parallelize E2E test suite using matrix sharding strategy`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/m3-parallelize-e2e-tests`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 13:22:18 +00:00

freemo added a new dependency

2026-04-03 13:23:33 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo referenced this issue

2026-04-03 14:18:56 +00:00

TEST-INFRA: [ci-execution-time] Parallelize E2E and integration tests #2319

freemo referenced this issue

2026-04-03 14:20:32 +00:00

TEST-INFRA: [ci-execution-time] Parallelize the E2E test suite #2322

freemo referenced this issue

2026-04-03 14:21:32 +00:00

TEST-INFRA: [ci-execution-time] Parallelize the E2E test suite #2322

freemo commented

2026-04-03 17:22:21 +00:00

Author

Owner

Closing as duplicate of #2336.

This issue ("Parallelize E2E tests for faster CI") covers the same scope as #2336 ("Parallelize E2E tests using pabot for faster CI execution"). Issue #2336 is more specific and actionable. Consolidating to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2336. This issue ("Parallelize E2E tests for faster CI") covers the same scope as #2336 ("Parallelize E2E tests using pabot for faster CI execution"). Issue #2336 is more specific and actionable. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo

2026-04-03 17:22:30 +00:00