ci(e2e): Parallelize E2E tests using pabot for faster CI execution #2336

Open
opened 2026-04-03 14:44:38 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: task/ci-execution-time-parallelize-e2e-tests
  • Commit Message: feat(ci): parallelize e2e test session using pabot to reduce pipeline execution time
  • Milestone: v3.8.0
  • Parent Epic: #1678

Problem

The End-to-End (E2E) test suite is currently executed sequentially, which significantly increases the overall CI pipeline execution time. The e2e_tests job in .forgejo/workflows/ci.yml has a timeout of 45 minutes, indicating that it is one of the longest-running jobs in the pipeline.

The noxfile.py confirms that the e2e_tests session invokes robot directly, without using a parallel execution tool like pabot. In contrast, the integration_tests session already uses pabot for parallel execution.

Proposed Solution

Modify the e2e_tests nox session in noxfile.py to use pabot for parallel test execution, mirroring the existing implementation of the integration_tests session.

Subtasks

  • Modify the e2e_tests session in noxfile.py to invoke pabot instead of robot
  • Ensure pabot is listed as a dependency for the e2e_tests nox session
  • Verify that all E2E tests run correctly in parallel without race conditions or shared-state issues
  • Confirm that test reports are generated and aggregated correctly under pabot
  • Update features/ci_workflow_validation.feature if any CI workflow assertions need updating
  • Monitor CI execution time to confirm wall-clock reduction

Definition of Done

  • The e2e_tests nox session in noxfile.py uses pabot instead of robot
  • The E2E test suite runs in parallel in the CI pipeline
  • All E2E tests continue to pass reliably with no flakiness introduced
  • Test reports are correctly generated and aggregated
  • The overall CI execution time for the e2e_tests job is measurably reduced
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-parallelize-e2e-tests` - **Commit Message**: `feat(ci): parallelize e2e test session using pabot to reduce pipeline execution time` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Problem The End-to-End (E2E) test suite is currently executed sequentially, which significantly increases the overall CI pipeline execution time. The `e2e_tests` job in `.forgejo/workflows/ci.yml` has a timeout of 45 minutes, indicating that it is one of the longest-running jobs in the pipeline. The `noxfile.py` confirms that the `e2e_tests` session invokes `robot` directly, without using a parallel execution tool like `pabot`. In contrast, the `integration_tests` session already uses `pabot` for parallel execution. ## Proposed Solution Modify the `e2e_tests` nox session in `noxfile.py` to use `pabot` for parallel test execution, mirroring the existing implementation of the `integration_tests` session. ## Subtasks - [ ] Modify the `e2e_tests` session in `noxfile.py` to invoke `pabot` instead of `robot` - [ ] Ensure `pabot` is listed as a dependency for the `e2e_tests` nox session - [ ] Verify that all E2E tests run correctly in parallel without race conditions or shared-state issues - [ ] Confirm that test reports are generated and aggregated correctly under `pabot` - [ ] Update `features/ci_workflow_validation.feature` if any CI workflow assertions need updating - [ ] Monitor CI execution time to confirm wall-clock reduction ## Definition of Done - [ ] The `e2e_tests` nox session in `noxfile.py` uses `pabot` instead of `robot` - [ ] The E2E test suite runs in parallel in the CI pipeline - [ ] All E2E tests continue to pass reliably with no flakiness introduced - [ ] Test reports are correctly generated and aggregated - [ ] The overall CI execution time for the `e2e_tests` job is measurably reduced - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 14:44:43 +00:00
Author
Owner

Duplicate of #2278 and #1961.

This is the fourth issue requesting E2E test parallelization using pabot. Previous issues:

  • #1961 ("Parallelize E2E test execution in CI", v3.8.0) — triaged cycle 2
  • #2278 ("Parallelize E2E tests for faster execution", v3.8.0) — triaged cycle 85
  • #2322 ("Parallelize the E2E test suite", v3.2.0) — closed as duplicate cycle 99

Closing as a duplicate.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

**Duplicate of #2278 and #1961.** This is the fourth issue requesting E2E test parallelization using pabot. Previous issues: - #1961 ("Parallelize E2E test execution in CI", v3.8.0) — triaged cycle 2 - #2278 ("Parallelize E2E tests for faster execution", v3.8.0) — triaged cycle 85 - #2322 ("Parallelize the E2E test suite", v3.2.0) — closed as duplicate cycle 99 Closing as a duplicate. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo reopened this issue 2026-04-03 14:46:47 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High (already set, correct — E2E tests are a major CI bottleneck)
  • Milestone: v3.8.0 (correct)
  • MoSCoW: Could Have — CI optimization is valuable but not blocking any milestone delivery. The v3.8.0 milestone description explicitly states "No deadline assigned. Development effort should focus on M1-M6 milestones first."
  • Parent Epic: #1678

This is the canonical issue for E2E test parallelization. Duplicates #2322, #2319, #2298, #2341, #2318 have been closed and consolidated here.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified ✅ - **Priority**: High (already set, correct — E2E tests are a major CI bottleneck) - **Milestone**: v3.8.0 (correct) - **MoSCoW**: Could Have — CI optimization is valuable but not blocking any milestone delivery. The v3.8.0 milestone description explicitly states "No deadline assigned. Development effort should focus on M1-M6 milestones first." - **Parent Epic**: #1678 This is the canonical issue for E2E test parallelization. Duplicates #2322, #2319, #2298, #2341, #2318 have been closed and consolidated here. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo removed this from the v3.8.0 milestone 2026-04-07 01:01:08 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2336
No description provided.