TEST-INFRA: [ci-execution-time] Increase default parallelism for `unit_tests` and `integration_tests` #2342

New issue

Closed

opened 2026-04-03 14:57:51 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 14:57:51 +00:00

Owner

Metadata

Branch: task/ci-execution-time-increase-default-parallelism
Commit Message: perf(ci): increase default parallelism for unit_tests and integration_tests nox sessions
Milestone: v3.8.0
Parent Epic: #1678

Description

The unit_tests and integration_tests nox sessions use a very conservative default for the number of parallel processes. This can lead to under-utilization of resources on CI runners with a high number of CPU cores, resulting in longer than necessary execution times.

Analysis

The noxfile.py defines the _default_processes() function as follows:

def _default_processes() -> int:
    env_override = os.environ.get("TEST_PROCESSES")
    if env_override:
        return int(env_override)
    try:
        cpus = len(os.sched_getaffinity(0)) or 1
    except AttributeError:
        cpus = os.cpu_count() or 1
    # Keep default parallelism conservative to avoid timeout/OOM flakes
    # under heavy Robot/pabot subprocess fan-out in CI and shared runners.
    # Callers can still override with TEST_PROCESSES / --processes.
    return min(cpus, 2)

The function defaults to a maximum of 2 processes, which is very low for modern CI systems. While it's good that this can be overridden with the TEST_PROCESSES environment variable, the default value should be more reasonable to get better performance out of the box.

Proposed Solution

Increase the default number of processes: The _default_processes function should be updated to use a higher default value, for example, min(cpus, 4) or min(cpus, 8). This will provide a better balance between parallelism and resource consumption.
Configure TEST_PROCESSES in CI: The CI configuration should be updated to set the TEST_PROCESSES environment variable to an optimal value for the CI runner. This will ensure that the tests are always run with the optimal level of parallelism.

Subtasks

Profile the current unit_tests and integration_tests execution time on CI to establish a baseline.
Evaluate safe upper bounds for parallelism given CI runner CPU/memory constraints (avoid OOM/timeout flakes).
Update _default_processes() in noxfile.py to use a higher default cap (e.g. min(cpus, 4) or min(cpus, 8)).
Update the CI configuration (.forgejo/workflows/ci.yml) to set TEST_PROCESSES to an optimal value for the CI runner.
Run the full test suite and verify no new flakes or OOM failures are introduced.
Measure and document the execution time improvement compared to the baseline.
Verify all nox stages pass after changes.
Confirm coverage ≥ 97% via nox -s coverage_report.

Definition of Done

The _default_processes function in noxfile.py is updated with a higher default value.
The CI configuration is updated to set the TEST_PROCESSES environment variable.
The execution time of the unit_tests and integration_tests sessions is measurably reduced compared to the baseline.
No new test flakes or OOM failures are introduced by the increased parallelism.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-increase-default-parallelism` - **Commit Message**: `perf(ci): increase default parallelism for unit_tests and integration_tests nox sessions` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The `unit_tests` and `integration_tests` nox sessions use a very conservative default for the number of parallel processes. This can lead to under-utilization of resources on CI runners with a high number of CPU cores, resulting in longer than necessary execution times. ## Analysis The `noxfile.py` defines the `_default_processes()` function as follows: ```python def _default_processes() -> int: env_override = os.environ.get("TEST_PROCESSES") if env_override: return int(env_override) try: cpus = len(os.sched_getaffinity(0)) or 1 except AttributeError: cpus = os.cpu_count() or 1 # Keep default parallelism conservative to avoid timeout/OOM flakes # under heavy Robot/pabot subprocess fan-out in CI and shared runners. # Callers can still override with TEST_PROCESSES / --processes. return min(cpus, 2) ``` The function defaults to a maximum of 2 processes, which is very low for modern CI systems. While it's good that this can be overridden with the `TEST_PROCESSES` environment variable, the default value should be more reasonable to get better performance out of the box. ## Proposed Solution 1. **Increase the default number of processes:** The `_default_processes` function should be updated to use a higher default value, for example, `min(cpus, 4)` or `min(cpus, 8)`. This will provide a better balance between parallelism and resource consumption. 2. **Configure `TEST_PROCESSES` in CI:** The CI configuration should be updated to set the `TEST_PROCESSES` environment variable to an optimal value for the CI runner. This will ensure that the tests are always run with the optimal level of parallelism. ## Subtasks - [ ] Profile the current `unit_tests` and `integration_tests` execution time on CI to establish a baseline. - [ ] Evaluate safe upper bounds for parallelism given CI runner CPU/memory constraints (avoid OOM/timeout flakes). - [ ] Update `_default_processes()` in `noxfile.py` to use a higher default cap (e.g. `min(cpus, 4)` or `min(cpus, 8)`). - [ ] Update the CI configuration (`.forgejo/workflows/ci.yml`) to set `TEST_PROCESSES` to an optimal value for the CI runner. - [ ] Run the full test suite and verify no new flakes or OOM failures are introduced. - [ ] Measure and document the execution time improvement compared to the baseline. - [ ] Verify all nox stages pass after changes. - [ ] Confirm coverage ≥ 97% via `nox -s coverage_report`. ## Definition of Done - [ ] The `_default_processes` function in `noxfile.py` is updated with a higher default value. - [ ] The CI configuration is updated to set the `TEST_PROCESSES` environment variable. - [ ] The execution time of the `unit_tests` and `integration_tests` sessions is measurably reduced compared to the baseline. - [ ] No new test flakes or OOM failures are introduced by the increased parallelism. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 14:58:02 +00:00

freemo added a new dependency

2026-04-03 14:59:07 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 15:02:56 +00:00

Author

Owner

Duplicate of #2285.

This issue describes the same optimization as #2285 ("Increase default parallelization for unit and integration tests", v3.8.0) and #2326 ("Make test parallelism level more configurable", v3.8.0). All three request increasing the default _default_processes() value and making it configurable via environment variable.

Closing as a duplicate.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

**Duplicate of #2285.** This issue describes the same optimization as #2285 ("Increase default parallelization for unit and integration tests", v3.8.0) and #2326 ("Make test parallelism level more configurable", v3.8.0). All three request increasing the default `_default_processes()` value and making it configurable via environment variable. Closing as a duplicate. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 15:02:57 +00:00