TEST-INFRA: [ci-execution-time] Increase Default Test Parallelism #2296

New issue

Closed

opened 2026-04-03 13:20:03 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 13:20:03 +00:00

Owner

Metadata

Branch: test-infra/ci-execution-time/increase-default-test-parallelism
Commit Message: perf(noxfile): increase default test parallelism for unit and integration sessions
Milestone: v3.8.0
Parent Epic: #1678

Description

The unit_tests (Behave) and integration_tests (Robot Framework) sessions in noxfile.py are configured with a default maximum of 2 parallel processes. This is an extremely conservative setting for modern CI runners which typically have significantly more CPU cores available. This low limit artificially throttles the test suites, leading to unnecessarily long execution times.

The _default_processes function should be updated to a more reasonable default, or the CI configuration itself must be modified to set the TEST_PROCESSES environment variable to a higher value that better utilizes the available hardware.

Subtasks

Benchmark Runner Resources: Determine the typical CPU and memory resources available to the CI runners.
Update Default Value: Modify the _default_processes function in noxfile.py to use a higher, more appropriate default (e.g., max(2, os.cpu_count() - 2)).
(Alternate) Set Environment Variable: If modifying the default is undesirable, update the main CI workflow to explicitly set the TEST_PROCESSES environment variable for the test jobs.
Monitor Stability: After increasing parallelism, closely monitor CI runs for any signs of instability, such as flaky tests or out-of-memory errors.

Definition of Done

The default parallelism for Behave and Robot test sessions is increased from the hardcoded 2.
The execution time for the unit_tests and integration_tests nox sessions is measurably reduced.
The test suites remain stable and reliable after the change.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test-infra/ci-execution-time/increase-default-test-parallelism` - **Commit Message**: `perf(noxfile): increase default test parallelism for unit and integration sessions` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The `unit_tests` (Behave) and `integration_tests` (Robot Framework) sessions in `noxfile.py` are configured with a default maximum of 2 parallel processes. This is an extremely conservative setting for modern CI runners which typically have significantly more CPU cores available. This low limit artificially throttles the test suites, leading to unnecessarily long execution times. The `_default_processes` function should be updated to a more reasonable default, or the CI configuration itself must be modified to set the `TEST_PROCESSES` environment variable to a higher value that better utilizes the available hardware. ## Subtasks - [ ] **Benchmark Runner Resources**: Determine the typical CPU and memory resources available to the CI runners. - [ ] **Update Default Value**: Modify the `_default_processes` function in `noxfile.py` to use a higher, more appropriate default (e.g., `max(2, os.cpu_count() - 2)`). - [ ] **(Alternate) Set Environment Variable**: If modifying the default is undesirable, update the main CI workflow to explicitly set the `TEST_PROCESSES` environment variable for the test jobs. - [ ] **Monitor Stability**: After increasing parallelism, closely monitor CI runs for any signs of instability, such as flaky tests or out-of-memory errors. ## Definition of Done - [ ] The default parallelism for Behave and Robot test sessions is increased from the hardcoded `2`. - [ ] The execution time for the `unit_tests` and `integration_tests` nox sessions is measurably reduced. - [ ] The test suites remain stable and reliable after the change. - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 13:20:46 +00:00

freemo added a new dependency

2026-04-03 13:22:33 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 17:22:52 +00:00

Author

Owner

Closing as duplicate of #2333.

This issue ("Increase Default Test Parallelism") covers the same scope as #2333 ("Investigate and increase test parallelism"), which is more comprehensive and includes investigation before changing defaults. Consolidating to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2333. This issue ("Increase Default Test Parallelism") covers the same scope as #2333 ("Investigate and increase test parallelism"), which is more comprehensive and includes investigation before changing defaults. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 17:23:08 +00:00

freemo referenced this issue

2026-04-03 17:23:59 +00:00

Proposal: improve ca-test-infra-improver — strengthen duplicate avoidance to prevent near-identical issues #2370

freemo referenced this issue

2026-04-03 17:24:35 +00:00

TEST-INFRA: [ci-execution-time] Investigate and increase test parallelism #2333