TEST-INFRA: [ci-execution-time] Split integration tests into smaller, parallelizable suites #4407

Open
opened 2026-04-08 12:16:36 +00:00 by HAL9000 · 1 comment
Owner

Metadata

  • Branch: feat/ci-execution-time/split-integration-test-suites
  • Commit Message: feat(ci): split integration tests into smaller, parallelizable suites
  • Milestone: (none — see backlog note below)
  • Parent Epic: #1678

Backlog note: This issue was discovered during autonomous operation
on milestone v3.7.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Overview

The current integration_tests nox session runs a large number of tests in parallel using pabot. While this is effective, the entire suite runs as a single, monolithic block in the CI pipeline. If this block is slow, it becomes a bottleneck for the entire CI process.

This proposal suggests splitting the integration tests into smaller, more focused suites that can be run as independent, parallel jobs in the CI pipeline. This would allow for better resource utilization and potentially faster overall CI execution time.

Subtasks

  • Identify logical groupings for the integration tests (e.g., by feature, by component, by service).
  • Create new nox sessions for each of the new test suites.
  • Update the CI configuration to run the new nox sessions as parallel jobs.
  • Ensure that the test reports from the parallel jobs are properly collected and aggregated.

Definition of Done

  • The integration tests are split into at least two smaller, independent suites.
  • The new test suites are run as parallel jobs in the CI pipeline.
  • The total execution time of the integration tests is reduced or remains the same.
  • The test reports are correctly collected and available for inspection.
  • All nox stages pass.
  • Coverage >= 97%.

Duplicate Check

  • Search queries: "split integration tests", "parallelize ci jobs", "matrix builds", "parallel integration suites"
  • Results: No existing issues found that specifically address splitting the integration test suite into smaller, parallelizable suites. (Related: #3777 parallelizes e2e tests via matrix strategy; this issue targets the integration_tests nox session specifically.)

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `feat/ci-execution-time/split-integration-test-suites` - **Commit Message**: `feat(ci): split integration tests into smaller, parallelizable suites` - **Milestone**: *(none — see backlog note below)* - **Parent Epic**: #1678 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.7.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Overview The current `integration_tests` nox session runs a large number of tests in parallel using `pabot`. While this is effective, the entire suite runs as a single, monolithic block in the CI pipeline. If this block is slow, it becomes a bottleneck for the entire CI process. This proposal suggests splitting the integration tests into smaller, more focused suites that can be run as independent, parallel jobs in the CI pipeline. This would allow for better resource utilization and potentially faster overall CI execution time. ## Subtasks - [ ] Identify logical groupings for the integration tests (e.g., by feature, by component, by service). - [ ] Create new nox sessions for each of the new test suites. - [ ] Update the CI configuration to run the new nox sessions as parallel jobs. - [ ] Ensure that the test reports from the parallel jobs are properly collected and aggregated. ## Definition of Done - [ ] The integration tests are split into at least two smaller, independent suites. - [ ] The new test suites are run as parallel jobs in the CI pipeline. - [ ] The total execution time of the integration tests is reduced or remains the same. - [ ] The test reports are correctly collected and available for inspection. - [ ] All nox stages pass. - [ ] Coverage >= 97%. ## Duplicate Check - Search queries: "split integration tests", "parallelize ci jobs", "matrix builds", "parallel integration suites" - Results: No existing issues found that specifically address splitting the integration test suite into smaller, parallelizable suites. (Related: #3777 parallelizes e2e tests via matrix strategy; this issue targets the `integration_tests` nox session specifically.) --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
Author
Owner

Issue triaged by project owner:

  • State: Verified — Valid test infrastructure improvement
  • Priority: Backlog (already set) — Does not block any milestone
  • MoSCoW: Could Have — Nice optimization but not essential
  • Story Points: 5 — L — Requires CI pipeline restructuring
  • Assignee: HAL9000

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified ✅ — Valid test infrastructure improvement - **Priority**: Backlog ✅ (already set) — Does not block any milestone - **MoSCoW**: Could Have — Nice optimization but not essential - **Story Points**: 5 — L — Requires CI pipeline restructuring - **Assignee**: HAL9000 --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner
HAL9000 self-assigned this 2026-04-08 12:24:18 +00:00
HAL9000 added this to the v3.5.0 milestone 2026-04-08 17:42:53 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#4407
No description provided.