TEST-INFRA: [ci-execution-time] Parallelize Robot Framework integration test suite #5820

New issue

Open

opened 2026-04-09 10:10:35 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 10:10:35 +00:00

Owner

Metadata

Branch: test-infra/parallelize-robot-integration-tests
Commit Message: perf(ci): parallelize Robot Framework integration test suite to eliminate nox integration_tests timeout
Milestone: (backlog — see note below)
Parent Epic: #5407

Summary

The nox -e integration_tests session is consistently timing out after 20 minutes in CI. This is a significant bottleneck in the development workflow and needs to be addressed.

Problem

The Robot Framework test suite is running sequentially, which is not scalable as the number of tests grows. The current execution time exceeds the CI timeout, preventing PRs from being merged.

Proposed Solution

Implement parallel execution for the Robot Framework test suite. This can be achieved by using tools like pabot or by sharding the test suites and running them on multiple runners.

Subtasks

Research and choose a parallel execution strategy for Robot Framework.
Implement the chosen strategy in the noxfile.py.
Update the CI workflow to use the new parallel execution.
Monitor the execution time of the integration_tests session to ensure it is within acceptable limits.

Definition of Done

The integration_tests session completes successfully in CI without timing out.
The execution time of the integration_tests session is significantly reduced.
All nox stages pass.
Coverage >= 97%.

Duplicate Check

Search queries: robot parallel, parallelize integration tests, integration_tests timeout
Results: No existing issues found that address the initial implementation of parallelization for the Robot Framework test suite. (Note: Issue #5662 covers optimizing existing pabot parallel execution, which is a distinct follow-on concern.)

Backlog note: This issue was discovered during autonomous operation
on milestone v3.6.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `test-infra/parallelize-robot-integration-tests` - **Commit Message**: `perf(ci): parallelize Robot Framework integration test suite to eliminate nox integration_tests timeout` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #5407 ## Summary The `nox -e integration_tests` session is consistently timing out after 20 minutes in CI. This is a significant bottleneck in the development workflow and needs to be addressed. ## Problem The Robot Framework test suite is running sequentially, which is not scalable as the number of tests grows. The current execution time exceeds the CI timeout, preventing PRs from being merged. ## Proposed Solution Implement parallel execution for the Robot Framework test suite. This can be achieved by using tools like `pabot` or by sharding the test suites and running them on multiple runners. ### Subtasks - [ ] Research and choose a parallel execution strategy for Robot Framework. - [ ] Implement the chosen strategy in the `noxfile.py`. - [ ] Update the CI workflow to use the new parallel execution. - [ ] Monitor the execution time of the `integration_tests` session to ensure it is within acceptable limits. ### Definition of Done - [ ] The `integration_tests` session completes successfully in CI without timing out. - [ ] The execution time of the `integration_tests` session is significantly reduced. - [ ] All nox stages pass. - [ ] Coverage >= 97%. ### Duplicate Check - Search queries: `robot parallel`, `parallelize integration tests`, `integration_tests timeout` - Results: No existing issues found that address the *initial implementation* of parallelization for the Robot Framework test suite. (Note: Issue #5662 covers *optimizing existing* pabot parallel execution, which is a distinct follow-on concern.) > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.6.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

HAL9000 added the

labels

2026-04-09 10:22:28 +00:00

HAL9000 added this to the v3.8.0 milestone

2026-04-09 10:26:12 +00:00

HAL9000 commented

2026-04-09 10:32:45 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed