TEST-INFRA: [ci-execution-time] Split integration tests for better parallelization #1632

Open
opened 2026-04-02 23:19:12 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/ci-execution-time-split-integration-tests
  • Commit Message: chore(ci): split integration tests into parallel matrix jobs for faster CI execution
  • Milestone: v3.8.0
  • Parent Epic: #376

Description

The integration_tests job in ci.yml runs all Robot Framework tests in a single job. While pabot provides parallel execution within the job, we can achieve even better performance by splitting the tests into multiple smaller suites and running them in parallel across multiple CI runners.

Subtasks

  • Analyze the existing Robot Framework tests in the robot/ directory to identify logical groups of tests that can be run independently.
  • Create a new nox session or modify the existing integration_tests session to accept arguments for running a specific subset of tests.
  • Modify the ci.yml file to create a matrix of integration_tests jobs, each running a different subset of the tests.
  • Ensure that the test reports from all parallel jobs are collected and aggregated correctly.

Definition of Done

  • The integration tests are split into at least two parallel jobs in the CI pipeline.
  • The total execution time of the integration test phase is significantly reduced.
  • All tests continue to run and pass as expected.
  • Test reports are correctly aggregated.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-split-integration-tests` - **Commit Message**: `chore(ci): split integration tests into parallel matrix jobs for faster CI execution` - **Milestone**: v3.8.0 - **Parent Epic**: #376 ## Description The `integration_tests` job in `ci.yml` runs all Robot Framework tests in a single job. While `pabot` provides parallel execution within the job, we can achieve even better performance by splitting the tests into multiple smaller suites and running them in parallel across multiple CI runners. ## Subtasks - [ ] Analyze the existing Robot Framework tests in the `robot/` directory to identify logical groups of tests that can be run independently. - [ ] Create a new `nox` session or modify the existing `integration_tests` session to accept arguments for running a specific subset of tests. - [ ] Modify the `ci.yml` file to create a matrix of `integration_tests` jobs, each running a different subset of the tests. - [ ] Ensure that the test reports from all parallel jobs are collected and aggregated correctly. ## Definition of Done - [ ] The integration tests are split into at least two parallel jobs in the CI pipeline. - [ ] The total execution time of the integration test phase is significantly reduced. - [ ] All tests continue to run and pass as expected. - [ ] Test reports are correctly aggregated. - [ ] All nox stages pass. - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:19:35 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Priority/Medium
  • MoSCoW: MoSCoW/Could Have — splitting integration tests for parallelization is a CI performance optimization. Could Have.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Priority/Medium - **MoSCoW**: MoSCoW/Could Have — splitting integration tests for parallelization is a CI performance optimization. Could Have. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
Reference
cleveragents/cleveragents-core#1632
No description provided.