TEST-INFRA: [ci-execution-time] Parallelize E2E and integration tests #2319

Closed
opened 2026-04-03 14:18:56 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-parallelize-e2e-integration-tests
  • Commit Message: chore(ci): parallelize E2E and integration test suites in CI workflow
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The CI pipeline currently runs the E2E and integration test suites sequentially, which contributes significantly to the overall CI execution time. Both suites are long-running (E2E: 14m+, integration: 22m+) and represent the largest bottleneck in the pipeline. Splitting these suites into parallelizable chunks and running them concurrently in the .forgejo/workflows/ci.yml workflow will substantially reduce total CI wall-clock time.

Expected Behavior

The E2E and integration test suites are split into smaller, independently runnable chunks. The CI workflow is updated to execute these chunks in parallel jobs, reducing the overall CI execution time significantly.

Acceptance Criteria

  • The E2E test suite is split into at least two parallelizable chunks using an appropriate splitting strategy (e.g., by feature file, by tag, or by estimated runtime).
  • The integration test suite is split into at least two parallelizable chunks using an appropriate splitting strategy.
  • The .forgejo/workflows/ci.yml file is updated to run the split chunks as parallel CI jobs (e.g., using a matrix strategy or explicit parallel job definitions).
  • All existing tests continue to pass after the restructuring.
  • The overall CI execution time is measurably reduced compared to the baseline.
  • No test is omitted — the union of all parallel chunks must equal the full original test suite.

Supporting Information

  • Related: #2298 (Parallelize E2E tests for faster CI)
  • Related: #2315 (Investigate and optimize slow integration_tests suite)
  • Related: #2278 (Parallelize E2E tests for faster execution)
  • Parent Epic: #1678 (CI Execution Time Optimization)

Subtasks

  • Investigate test splitting strategies for the E2E test suite (by feature file, tag, or runtime estimate).
  • Investigate test splitting strategies for the integration test suite (by module, tag, or runtime estimate).
  • Modify .forgejo/workflows/ci.yml to run the split E2E test chunks in parallel (e.g., matrix strategy).
  • Modify .forgejo/workflows/ci.yml to run the split integration test chunks in parallel.
  • Verify all tests still pass and no tests are omitted after the split.
  • Monitor and document the CI execution time improvement after the changes.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage ≥ 97% via nox -s coverage_report.

Definition of Done

  • All subtasks above are completed and checked off.
  • The E2E and integration test suites are split into smaller, parallelizable chunks with no tests omitted.
  • The CI workflow (.forgejo/workflows/ci.yml) is updated to run these chunks in parallel.
  • The overall CI execution time is significantly reduced (documented with before/after measurements).
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-parallelize-e2e-integration-tests` - **Commit Message**: `chore(ci): parallelize E2E and integration test suites in CI workflow` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The CI pipeline currently runs the E2E and integration test suites sequentially, which contributes significantly to the overall CI execution time. Both suites are long-running (E2E: 14m+, integration: 22m+) and represent the largest bottleneck in the pipeline. Splitting these suites into parallelizable chunks and running them concurrently in the `.forgejo/workflows/ci.yml` workflow will substantially reduce total CI wall-clock time. ## Expected Behavior The E2E and integration test suites are split into smaller, independently runnable chunks. The CI workflow is updated to execute these chunks in parallel jobs, reducing the overall CI execution time significantly. ## Acceptance Criteria - The E2E test suite is split into at least two parallelizable chunks using an appropriate splitting strategy (e.g., by feature file, by tag, or by estimated runtime). - The integration test suite is split into at least two parallelizable chunks using an appropriate splitting strategy. - The `.forgejo/workflows/ci.yml` file is updated to run the split chunks as parallel CI jobs (e.g., using a matrix strategy or explicit parallel job definitions). - All existing tests continue to pass after the restructuring. - The overall CI execution time is measurably reduced compared to the baseline. - No test is omitted — the union of all parallel chunks must equal the full original test suite. ## Supporting Information - Related: #2298 (Parallelize E2E tests for faster CI) - Related: #2315 (Investigate and optimize slow `integration_tests` suite) - Related: #2278 (Parallelize E2E tests for faster execution) - Parent Epic: #1678 (CI Execution Time Optimization) ## Subtasks - [ ] Investigate test splitting strategies for the E2E test suite (by feature file, tag, or runtime estimate). - [ ] Investigate test splitting strategies for the integration test suite (by module, tag, or runtime estimate). - [ ] Modify `.forgejo/workflows/ci.yml` to run the split E2E test chunks in parallel (e.g., matrix strategy). - [ ] Modify `.forgejo/workflows/ci.yml` to run the split integration test chunks in parallel. - [ ] Verify all tests still pass and no tests are omitted after the split. - [ ] Monitor and document the CI execution time improvement after the changes. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage ≥ 97% via `nox -s coverage_report`. ## Definition of Done - [ ] All subtasks above are completed and checked off. - [ ] The E2E and integration test suites are split into smaller, parallelizable chunks with no tests omitted. - [ ] The CI workflow (`.forgejo/workflows/ci.yml`) is updated to run these chunks in parallel. - [ ] The overall CI execution time is significantly reduced (documented with before/after measurements). - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 14:19:15 +00:00
Author
Owner

Closing as duplicate of #2336.

This issue ("Parallelize E2E and integration tests") overlaps with #2336 (E2E parallelization via pabot) and #2315 (integration test optimization). The E2E portion is covered by #2336 and the integration portion by #2315. Consolidating to avoid duplicate work.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2336. This issue ("Parallelize E2E and integration tests") overlaps with #2336 (E2E parallelization via pabot) and #2315 (integration test optimization). The E2E portion is covered by #2336 and the integration portion by #2315. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo 2026-04-03 17:22:29 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2319
No description provided.