TEST-INFRA: [ci-pipeline-design] Split tests for parallel execution #1835

Open
opened 2026-04-02 23:56:34 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: feature/ci-split-tests-parallel-execution
  • Commit Message: feat(ci): split unit and integration test sessions for parallel execution
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The unit_tests and integration_tests nox sessions currently run all tests in a single serial session. As the test suite grows, this creates a bottleneck in CI wall-clock time. Splitting the test suite into smaller chunks and running them in parallel across CI matrix jobs would significantly reduce overall execution time and improve developer feedback loops.

Expected Behavior

The unit_tests and integration_tests sessions are split into smaller, independently runnable chunks. The CI workflow is updated to run these chunks in parallel using a matrix strategy, reducing total test execution time.

Acceptance Criteria

  • The unit_tests and integration_tests sessions support splitting into smaller chunks (e.g., via pytest-split or a matrix of test directories/markers).
  • The CI workflow runs the split test chunks in parallel.
  • The overall wall-clock execution time of the test sessions is measurably reduced.
  • All tests continue to pass and coverage remains ≥ 97%.

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization
  • Related to the broader effort to reduce CI pipeline execution time via concurrency, timeouts, and artifact sharing.

Subtasks

  • Analyze the unit_tests and integration_tests suites to identify the optimal split strategy (by directory, marker, or pytest-split).
  • Modify noxfile.py to support parameterized/split test execution (e.g., accept a --group or --splits argument).
  • Modify .forgejo/workflows/ci.yml to use a matrix strategy for running split test chunks in parallel.
  • Update features/ci_workflow_validation.feature to assert that the parallel test matrix is present in the CI workflow.
  • Verify all tests pass and coverage ≥ 97% after changes.
  • Run nox (all default sessions) and fix any errors.

Definition of Done

  • The unit_tests and integration_tests sessions are split into smaller chunks runnable independently.
  • The CI workflow runs the split chunks in parallel via a matrix strategy.
  • The overall execution time of the test sessions is measurably reduced.
  • features/ci_workflow_validation.feature is updated to cover the parallel matrix steps.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `feature/ci-split-tests-parallel-execution` - **Commit Message**: `feat(ci): split unit and integration test sessions for parallel execution` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `unit_tests` and `integration_tests` nox sessions currently run all tests in a single serial session. As the test suite grows, this creates a bottleneck in CI wall-clock time. Splitting the test suite into smaller chunks and running them in parallel across CI matrix jobs would significantly reduce overall execution time and improve developer feedback loops. ## Expected Behavior The `unit_tests` and `integration_tests` sessions are split into smaller, independently runnable chunks. The CI workflow is updated to run these chunks in parallel using a matrix strategy, reducing total test execution time. ## Acceptance Criteria - The `unit_tests` and `integration_tests` sessions support splitting into smaller chunks (e.g., via pytest-split or a matrix of test directories/markers). - The CI workflow runs the split test chunks in parallel. - The overall wall-clock execution time of the test sessions is measurably reduced. - All tests continue to pass and coverage remains ≥ 97%. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization - Related to the broader effort to reduce CI pipeline execution time via concurrency, timeouts, and artifact sharing. ## Subtasks - [ ] Analyze the `unit_tests` and `integration_tests` suites to identify the optimal split strategy (by directory, marker, or pytest-split). - [ ] Modify `noxfile.py` to support parameterized/split test execution (e.g., accept a `--group` or `--splits` argument). - [ ] Modify `.forgejo/workflows/ci.yml` to use a matrix strategy for running split test chunks in parallel. - [ ] Update `features/ci_workflow_validation.feature` to assert that the parallel test matrix is present in the CI workflow. - [ ] Verify all tests pass and coverage ≥ 97% after changes. - [ ] Run `nox` (all default sessions) and fix any errors. ## Definition of Done - [ ] The `unit_tests` and `integration_tests` sessions are split into smaller chunks runnable independently. - [ ] The CI workflow runs the split chunks in parallel via a matrix strategy. - [ ] The overall execution time of the test sessions is measurably reduced. - [ ] `features/ci_workflow_validation.feature` is updated to cover the parallel matrix steps. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:56:41 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1835
No description provided.