TEST-INFRA: [ci-execution-time] Investigate splitting integration test suite for parallel execution #2330

Closed
opened 2026-04-03 14:31:13 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-execution-time-split-integration-tests
  • Commit Message: chore(ci): investigate splitting integration test suite for parallel execution
  • Milestone: v3.8.0
  • Parent Epic: #1678

Summary

The integration_tests nox session is the largest test suite and is likely to be the longest-running part of the CI pipeline. While it already runs tests in parallel using pabot, as the number of tests grows, the overall execution time will continue to increase.

To preemptively address this, we should investigate splitting the integration test suite into multiple, smaller suites that can be run in parallel as separate CI jobs. This would allow for much greater scalability and would help keep CI times low.

Subtasks

  • Analyze the execution time of the integration_tests session to confirm if it is a bottleneck.
  • Identify logical ways to split the test suite (e.g., by feature, by directory, or by test type).
  • Create new nox sessions for each of the smaller test suites.
  • Update the CI configuration to run these new nox sessions in parallel.
  • Add or update BDD scenarios in the relevant feature file to cover the new parallel session behaviour.
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

  • The integration_tests suite is split into smaller, more manageable suites.
  • The new test suites are run in parallel in the CI pipeline.
  • The overall CI execution time is significantly reduced.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-execution-time-split-integration-tests` - **Commit Message**: `chore(ci): investigate splitting integration test suite for parallel execution` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Summary The `integration_tests` nox session is the largest test suite and is likely to be the longest-running part of the CI pipeline. While it already runs tests in parallel using `pabot`, as the number of tests grows, the overall execution time will continue to increase. To preemptively address this, we should investigate splitting the integration test suite into multiple, smaller suites that can be run in parallel as separate CI jobs. This would allow for much greater scalability and would help keep CI times low. ## Subtasks - [ ] Analyze the execution time of the `integration_tests` session to confirm if it is a bottleneck. - [ ] Identify logical ways to split the test suite (e.g., by feature, by directory, or by test type). - [ ] Create new nox sessions for each of the smaller test suites. - [ ] Update the CI configuration to run these new nox sessions in parallel. - [ ] Add or update BDD scenarios in the relevant feature file to cover the new parallel session behaviour. - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] The `integration_tests` suite is split into smaller, more manageable suites. - [ ] The new test suites are run in parallel in the CI pipeline. - [ ] The overall CI execution time is significantly reduced. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 14:31:32 +00:00
Author
Owner

Closing as duplicate of #2315.

This issue ("Investigate splitting integration test suite for parallel execution") covers the same scope as #2315 ("Investigate and optimize slow integration_tests suite (22m+ runtime)"). Consolidating to avoid duplicate work.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2315. This issue ("Investigate splitting integration test suite for parallel execution") covers the same scope as #2315 ("Investigate and optimize slow integration_tests suite (22m+ runtime)"). Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2330
No description provided.