TEST-INFRA: [ci-execution-time] Split integration_tests into smaller chunks #2376

New issue

Closed

opened 2026-04-03 17:25:37 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 17:25:37 +00:00

Owner

Metadata

Branch: chore/ci-split-integration-tests
Commit Message: chore(ci): split integration_tests nox session into parallelizable chunks
Milestone: v3.7.0
Parent Epic: #1678

Background and Context

The integration_tests nox session currently runs all tests in the robot/ directory as a single monolithic session. As the test suite grows, this creates a bottleneck in CI execution time. Splitting the session into smaller, logically grouped chunks would allow parallel execution, reduce overall pipeline duration, and provide more granular control over which test groups are run.

Expected Behavior

The robot/ integration tests are organized into multiple focused nox sessions (e.g., by feature area or sub-directory). These sessions can be run independently or in parallel in CI, reducing total execution time and improving developer feedback loops.

Acceptance Criteria

The robot/ directory is analyzed and tests are grouped into logical, non-conflicting chunks.
New nox sessions are created for each group, replacing or supplementing the monolithic integration_tests session.
The new sessions can be run in parallel without resource conflicts or shared-state issues.
The CI configuration (e.g., Forgejo Actions workflow) is updated to run the new sessions in parallel.
All existing integration tests continue to pass under the new session structure.

Supporting Information

Related Epic: #1678 — CI Execution Time Optimization
Nox session reference: nox -e integration_tests (Robot Framework, robot/ directory)
Parallelization must not introduce mocking — integration tests must exercise real services per CONTRIBUTING.md.

Subtasks

Analyze the robot/ directory and identify logical groupings of tests (by feature, sub-directory, or tag).
Design the new nox session structure (names, scope, entry points).
Implement new nox sessions for each identified group.
Verify that the new sessions can be run in parallel without conflicts (no shared state, no port collisions, etc.).
Update the CI configuration to run the new sessions in parallel.
Confirm all integration tests pass under the new session structure via nox -e <new-sessions>.
Run nox (all default sessions) and fix any errors.
Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): split integration_tests nox session into parallelizable chunks), followed by a blank line, then additional lines providing relevant implementation details.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (chore/ci-split-integration-tests).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-split-integration-tests` - **Commit Message**: `chore(ci): split integration_tests nox session into parallelizable chunks` - **Milestone**: v3.7.0 - **Parent Epic**: #1678 ## Background and Context The `integration_tests` nox session currently runs all tests in the `robot/` directory as a single monolithic session. As the test suite grows, this creates a bottleneck in CI execution time. Splitting the session into smaller, logically grouped chunks would allow parallel execution, reduce overall pipeline duration, and provide more granular control over which test groups are run. ## Expected Behavior The `robot/` integration tests are organized into multiple focused nox sessions (e.g., by feature area or sub-directory). These sessions can be run independently or in parallel in CI, reducing total execution time and improving developer feedback loops. ## Acceptance Criteria - The `robot/` directory is analyzed and tests are grouped into logical, non-conflicting chunks. - New nox sessions are created for each group, replacing or supplementing the monolithic `integration_tests` session. - The new sessions can be run in parallel without resource conflicts or shared-state issues. - The CI configuration (e.g., Forgejo Actions workflow) is updated to run the new sessions in parallel. - All existing integration tests continue to pass under the new session structure. ## Supporting Information - Related Epic: #1678 — CI Execution Time Optimization - Nox session reference: `nox -e integration_tests` (Robot Framework, `robot/` directory) - Parallelization must not introduce mocking — integration tests must exercise real services per CONTRIBUTING.md. ## Subtasks - [ ] Analyze the `robot/` directory and identify logical groupings of tests (by feature, sub-directory, or tag). - [ ] Design the new nox session structure (names, scope, entry points). - [ ] Implement new nox sessions for each identified group. - [ ] Verify that the new sessions can be run in parallel without conflicts (no shared state, no port collisions, etc.). - [ ] Update the CI configuration to run the new sessions in parallel. - [ ] Confirm all integration tests pass under the new session structure via `nox -e <new-sessions>`. - [ ] Run `nox` (all default sessions) and fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): split integration_tests nox session into parallelizable chunks`), followed by a blank line, then additional lines providing relevant implementation details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`chore/ci-split-integration-tests`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-03 17:25:42 +00:00

freemo added a new dependency

2026-04-03 17:26:18 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 17:30:53 +00:00

Author

Owner

Closing as duplicate of #2315.

This issue ("Split integration_tests into smaller chunks") covers the same scope as #2315 ("Investigate and optimize slow integration_tests suite (22m+ runtime)"). Consolidating to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2315. This issue ("Split integration_tests into smaller chunks") covers the same scope as #2315 ("Investigate and optimize slow integration_tests suite (22m+ runtime)"). Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 17:31:04 +00:00

freemo removed this from the v3.7.0 milestone

2026-04-06 20:58:43 +00:00