TEST-INFRA: [ci-execution-time] Parallelize e2e tests for faster execution #3777

Open
opened 2026-04-06 06:15:57 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: feat/ci-execution-time/parallelize-e2e-tests
  • Commit Message: feat(ci): parallelize e2e tests using matrix strategy for faster execution
  • Milestone: (none — see backlog note below)
  • Parent Epic: #1678

Duplicate Check

  • Search queries: "parallelize e2e tests", "e2e test performance", "split e2e tests"
  • Results: 0 issues found for each query. This is a new proposal.

Description

The e2e_tests job in the ci.yml workflow has a timeout of 45 minutes, which indicates that it is a long-running job. Currently, the tests are run in parallel using pabot with 4 processes, but this might not be sufficient to reduce the execution time significantly.

Proposal

To improve the execution time of the e2e_tests job, we should investigate splitting the tests into smaller, independent suites that can be run in parallel across multiple CI runners. This can be achieved by using a matrix strategy in the CI configuration.

Subtasks

  • Analyze the E2E test suite to identify logical groups of tests that can be run independently.
  • Create a matrix strategy in the ci.yml file to run the E2E test suites in parallel.
  • Update features/ci_workflow_validation.feature to assert the matrix strategy is present and correctly configured.
  • Monitor the execution time of the e2e_tests job to verify that the changes have improved the performance.
  • Verify all nox stages pass after changes.
  • Confirm coverage ≥ 97%.

Definition of Done

  • The e2e_tests job is split into at least 4 parallel runs via a matrix strategy.
  • The overall execution time of the E2E tests is reduced by at least 50%.
  • features/ci_workflow_validation.feature is updated to cover the new matrix strategy.
  • All nox stages pass.
  • Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `feat/ci-execution-time/parallelize-e2e-tests` - **Commit Message**: `feat(ci): parallelize e2e tests using matrix strategy for faster execution` - **Milestone**: *(none — see backlog note below)* - **Parent Epic**: #1678 ## Duplicate Check - Search queries: "parallelize e2e tests", "e2e test performance", "split e2e tests" - Results: 0 issues found for each query. This is a new proposal. ## Description The `e2e_tests` job in the `ci.yml` workflow has a timeout of 45 minutes, which indicates that it is a long-running job. Currently, the tests are run in parallel using `pabot` with 4 processes, but this might not be sufficient to reduce the execution time significantly. ## Proposal To improve the execution time of the `e2e_tests` job, we should investigate splitting the tests into smaller, independent suites that can be run in parallel across multiple CI runners. This can be achieved by using a matrix strategy in the CI configuration. ## Subtasks - [ ] Analyze the E2E test suite to identify logical groups of tests that can be run independently. - [ ] Create a matrix strategy in the `ci.yml` file to run the E2E test suites in parallel. - [ ] Update `features/ci_workflow_validation.feature` to assert the matrix strategy is present and correctly configured. - [ ] Monitor the execution time of the `e2e_tests` job to verify that the changes have improved the performance. - [ ] Verify all nox stages pass after changes. - [ ] Confirm coverage ≥ 97%. ## Definition of Done - [ ] The `e2e_tests` job is split into at least 4 parallel runs via a matrix strategy. - [ ] The overall execution time of the E2E tests is reduced by at least 50%. - [ ] `features/ci_workflow_validation.feature` is updated to cover the new matrix strategy. - [ ] All nox stages pass. - [ ] Coverage >= 97%. > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#3777
No description provided.