TEST-INFRA: [ci-execution-time] Investigate increasing parallel processes for Behave and Robot tests #2268

New issue

Closed

opened 2026-04-03 12:23:13 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 12:23:13 +00:00

Owner

Metadata

Branch: task/ci-execution-time-investigate-parallel-processes-behave-robot
Commit Message: perf(ci): make test parallelism configurable via TEST_PROCESSES env var
Milestone: v3.8.0
Parent Epic: #1678

Description

The noxfile.py uses behave-parallel and pabot to run unit and integration tests in parallel. The number of processes is conservatively defaulted to 2 to avoid resource contention on CI runners. There may be an opportunity to significantly speed up these test suites by increasing the number of parallel processes.

Finding Type: Performance Optimization
Area: Test Architecture
File: noxfile.py

Subtasks

Add an environment variable (e.g., TEST_PROCESSES) to the noxfile.py to allow overriding the default number of test processes.
Create a series of experimental CI runs with different values for TEST_PROCESSES (e.g., 4, 8, 16) to benchmark the impact on execution time and stability.
Analyze the results to determine the optimal number of processes that provides the best balance of speed and reliability.
Update the noxfile.py with the new optimal default value.

Definition of Done

The number of parallel test processes is configurable via an environment variable.
Benchmarking has been performed to identify the optimal number of processes.
The default number of processes in noxfile.py is updated to the new optimal value.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-investigate-parallel-processes-behave-robot` - **Commit Message**: `perf(ci): make test parallelism configurable via TEST_PROCESSES env var` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ### Description The `noxfile.py` uses `behave-parallel` and `pabot` to run unit and integration tests in parallel. The number of processes is conservatively defaulted to 2 to avoid resource contention on CI runners. There may be an opportunity to significantly speed up these test suites by increasing the number of parallel processes. **Finding Type**: Performance Optimization **Area**: Test Architecture **File**: `noxfile.py` ## Subtasks - [ ] Add an environment variable (e.g., `TEST_PROCESSES`) to the `noxfile.py` to allow overriding the default number of test processes. - [ ] Create a series of experimental CI runs with different values for `TEST_PROCESSES` (e.g., 4, 8, 16) to benchmark the impact on execution time and stability. - [ ] Analyze the results to determine the optimal number of processes that provides the best balance of speed and reliability. - [ ] Update the `noxfile.py` with the new optimal default value. ## Definition of Done - [ ] The number of parallel test processes is configurable via an environment variable. - [ ] Benchmarking has been performed to identify the optimal number of processes. - [ ] The default number of processes in `noxfile.py` is updated to the new optimal value. - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 12:24:17 +00:00

freemo added a new dependency

2026-04-03 12:30:02 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo referenced this issue

2026-04-03 12:39:56 +00:00

TEST-INFRA: [ci-execution-time] Increase default parallelism for unit and integration tests #2277

freemo commented

2026-04-03 17:30:49 +00:00

Author

Owner

Closing as duplicate of #2333.

This issue ("Investigate increasing parallel processes for Behave and Robot tests") covers the same scope as #2333 ("Investigate and increase test parallelism"). Consolidating to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2333. This issue ("Investigate increasing parallel processes for Behave and Robot tests") covers the same scope as #2333 ("Investigate and increase test parallelism"). Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 17:31:02 +00:00