TEST-INFRA: [ci-execution-time] Parallelize E2E tests for faster execution #2278

Closed
opened 2026-04-03 12:40:04 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: task/ci-execution-time-parallelize-e2e-tests
  • Commit Message: perf(ci): parallelize E2E tests using pabot to reduce e2e_tests job execution time
  • Milestone: v3.8.0
  • Parent Epic: #2273

Summary

The e2e_tests nox session currently executes all end-to-end tests sequentially using robot. This is a significant performance bottleneck, especially given the 45-minute timeout for this job in the CI configuration.

Subtasks

  • Modify the e2e_tests session in noxfile.py to use pabot for parallel execution, similar to the integration_tests session.
  • Ensure that the test reports are correctly aggregated when running with pabot.
  • Verify the CI execution time for the e2e_tests job is significantly reduced.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

  • The e2e_tests nox session runs tests in parallel.
  • The CI execution time for the e2e_tests job is significantly reduced.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-parallelize-e2e-tests` - **Commit Message**: `perf(ci): parallelize E2E tests using pabot to reduce e2e_tests job execution time` - **Milestone**: v3.8.0 - **Parent Epic**: #2273 ## Summary The `e2e_tests` nox session currently executes all end-to-end tests sequentially using `robot`. This is a significant performance bottleneck, especially given the 45-minute timeout for this job in the CI configuration. ## Subtasks - [ ] Modify the `e2e_tests` session in `noxfile.py` to use `pabot` for parallel execution, similar to the `integration_tests` session. - [ ] Ensure that the test reports are correctly aggregated when running with `pabot`. - [ ] Verify the CI execution time for the `e2e_tests` job is significantly reduced. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done - [ ] The `e2e_tests` nox session runs tests in parallel. - [ ] The CI execution time for the `e2e_tests` job is significantly reduced. - [ ] All nox stages pass. - [ ] Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 12:41:56 +00:00
Author
Owner

Potential duplicate of #1961.

This issue describes the same optimization as #1961 ("Parallelize E2E test execution in CI to reduce pipeline wall-clock time", v3.8.0). Both request using pabot to parallelize E2E tests. #1961 was triaged in cycle 2 with MoSCoW: Could Have.

Triaging as verified but flagging the overlap. The backlog groomer should consolidate these issues.

Issue triaged by project owner:

  • State: Verified
  • Priority: High (confirmed)
  • Milestone: v3.8.0 (confirmed — CI infrastructure)
  • MoSCoW: Could Have — E2E test parallelization is a CI performance optimization. Overlaps with #1961.
  • Parent Epic: #2273 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

**Potential duplicate of #1961.** This issue describes the same optimization as #1961 ("Parallelize E2E test execution in CI to reduce pipeline wall-clock time", v3.8.0). Both request using `pabot` to parallelize E2E tests. #1961 was triaged in cycle 2 with MoSCoW: Could Have. Triaging as verified but flagging the overlap. The backlog groomer should consolidate these issues. Issue triaged by project owner: - **State**: Verified - **Priority**: High (confirmed) - **Milestone**: v3.8.0 (confirmed — CI infrastructure) - **MoSCoW**: Could Have — E2E test parallelization is a CI performance optimization. Overlaps with #1961. - **Parent Epic**: #2273 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Closing as duplicate of #2336. Both issues track parallelizing the E2E test suite using pabot. Issue #2336 is the more detailed and up-to-date version with State/Verified and a complete Definition of Done. All work should be tracked there.


Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

Closing as duplicate of #2336. Both issues track parallelizing the E2E test suite using pabot. Issue #2336 is the more detailed and up-to-date version with `State/Verified` and a complete Definition of Done. All work should be tracked there. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2278
No description provided.