TEST-INFRA: [ci-execution-time] Make test parallelism level more configurable #2326

Closed
opened 2026-04-03 14:24:54 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: chore/ci-execution-time-test-parallelism
  • Commit Message: chore(ci): make test parallelism level configurable via environment variable
  • Milestone: v3.8.0
  • Parent Epic: #1678

Summary

The noxfile.py currently defaults to a maximum of 2 parallel processes for both unit and integration tests. While the comment in _default_processes() explains this is to avoid resource contention, this conservative limit may underutilize the capacity of modern CI runners, which often have 8, 16, or more cores.

Increasing the parallelism could significantly reduce the execution time of the unit_tests and integration_tests sessions.

Subtasks

  • Modify the _default_processes() function in noxfile.py to allow for a higher default number of processes, or to be more easily overridden by an environment variable in the CI/CD pipeline.
  • Investigate if the resource contention issues still exist on the current CI runners.
  • Update the CI configuration to set the number of processes based on the runner's available cores.
  • Add or update BDD scenarios in the relevant feature file to cover the configurable parallelism behaviour.
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

  • The number of parallel processes for testing is easily configurable in the CI environment (e.g., via an environment variable such as NOX_TEST_PROCESSES).
  • The default number of processes is re-evaluated to better match the available resources on current CI runners.
  • CI execution time for the test suites is measurably reduced.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-execution-time-test-parallelism` - **Commit Message**: `chore(ci): make test parallelism level configurable via environment variable` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Summary The `noxfile.py` currently defaults to a maximum of 2 parallel processes for both unit and integration tests. While the comment in `_default_processes()` explains this is to avoid resource contention, this conservative limit may underutilize the capacity of modern CI runners, which often have 8, 16, or more cores. Increasing the parallelism could significantly reduce the execution time of the `unit_tests` and `integration_tests` sessions. ## Subtasks - [ ] Modify the `_default_processes()` function in `noxfile.py` to allow for a higher default number of processes, or to be more easily overridden by an environment variable in the CI/CD pipeline. - [ ] Investigate if the resource contention issues still exist on the current CI runners. - [ ] Update the CI configuration to set the number of processes based on the runner's available cores. - [ ] Add or update BDD scenarios in the relevant feature file to cover the configurable parallelism behaviour. - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] The number of parallel processes for testing is easily configurable in the CI environment (e.g., via an environment variable such as `NOX_TEST_PROCESSES`). - [ ] The default number of processes is re-evaluated to better match the available resources on current CI runners. - [ ] CI execution time for the test suites is measurably reduced. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 14:25:03 +00:00
Author
Owner

Potential duplicate of #2285.

This issue overlaps significantly with #2285 ("Increase default parallelization for unit and integration tests", v3.8.0). Both request making the test parallelism level more configurable and increasing the default from 2 processes. #2285 was triaged with MoSCoW: Could Have.

Triaging as verified but flagging the overlap. The backlog groomer should consolidate these issues.

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium (confirmed)
  • Milestone: v3.8.0 (confirmed — CI infrastructure)
  • MoSCoW: Could Have — Test parallelism configuration is a CI performance tuning task. Overlaps with #2285.
  • Parent Epic: #1678 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

**Potential duplicate of #2285.** This issue overlaps significantly with #2285 ("Increase default parallelization for unit and integration tests", v3.8.0). Both request making the test parallelism level more configurable and increasing the default from 2 processes. #2285 was triaged with MoSCoW: Could Have. Triaging as verified but flagging the overlap. The backlog groomer should consolidate these issues. Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) - **Milestone**: v3.8.0 (confirmed — CI infrastructure) - **MoSCoW**: Could Have — Test parallelism configuration is a CI performance tuning task. Overlaps with #2285. - **Parent Epic**: #1678 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Closing as duplicate of #2333.

This issue ("Make test parallelism level more configurable") is a subset of #2333 ("Investigate and increase test parallelism"), which already includes configurability as part of its scope. Consolidating to avoid duplicate work.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2333. This issue ("Make test parallelism level more configurable") is a subset of #2333 ("Investigate and increase test parallelism"), which already includes configurability as part of its scope. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2326
No description provided.