TEST-INFRA: [ci-execution-time] Increase default parallelism for tests #2209

New issue

Open

opened 2026-04-03 09:31:27 +00:00 by freemo · 0 comments

freemo commented

2026-04-03 09:31:27 +00:00

Owner

Metadata

Branch: task/ci-execution-time-increase-test-parallelism
Commit Message: perf(ci): increase default test parallelism via configurable TEST_PROCESSES env var
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The default parallelism for running unit and integration tests is currently set to a conservative value of 2. This can be a bottleneck on CI runners with more cores. Increasing this value will better utilize available resources and reduce overall test execution time, contributing to the broader CI execution time optimization effort tracked in Epic #1678.

Current Behavior

The number of test processes in noxfile.py is hardcoded to 2 for both unit_tests and integration_tests sessions. This underutilizes CI runners that have more than 2 available cores, resulting in unnecessarily long test execution times.

Expected Behavior

The number of test processes should be configurable via a TEST_PROCESSES environment variable, with a sensible default (e.g., 4 or 8). The CI configuration should set this variable to an optimal value determined through experimentation on the actual CI runners.

Acceptance Criteria

The number of test processes is configurable via a TEST_PROCESSES environment variable.
noxfile.py reads TEST_PROCESSES and uses it to set the number of processes for both unit_tests and integration_tests sessions.
The default value of TEST_PROCESSES is increased to a more reasonable value (e.g., 4 or 8).
The CI configuration sets TEST_PROCESSES to the optimal value found through experimentation.
Test execution time is measurably reduced on CI.

Supporting Information

Parent Epic: #1678 (CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing)
Related: #1911 (Parallelize unit tests), #1968 (Split integration tests for better parallelism)

Subtasks

Make the number of test processes configurable via a TEST_PROCESSES environment variable.
Update noxfile.py to read TEST_PROCESSES and use it to set the number of processes for unit_tests and integration_tests sessions.
Experiment with different values for TEST_PROCESSES on the CI runners to find the optimal value.
Update the CI configuration (.forgejo/workflows/ci.yml) to set the TEST_PROCESSES environment variable to the optimal value.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly: perf(ci): increase default test parallelism via configurable TEST_PROCESSES env var
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly: task/ci-execution-time-increase-test-parallelism
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
The number of test processes is configurable via a TEST_PROCESSES environment variable.
The default number of processes is increased to a more reasonable value (e.g., 4 or 8).
The test execution time is measurably reduced on CI.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-increase-test-parallelism` - **Commit Message**: `perf(ci): increase default test parallelism via configurable TEST_PROCESSES env var` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The default parallelism for running unit and integration tests is currently set to a conservative value of 2. This can be a bottleneck on CI runners with more cores. Increasing this value will better utilize available resources and reduce overall test execution time, contributing to the broader CI execution time optimization effort tracked in Epic #1678. ## Current Behavior The number of test processes in `noxfile.py` is hardcoded to `2` for both `unit_tests` and `integration_tests` sessions. This underutilizes CI runners that have more than 2 available cores, resulting in unnecessarily long test execution times. ## Expected Behavior The number of test processes should be configurable via a `TEST_PROCESSES` environment variable, with a sensible default (e.g., `4` or `8`). The CI configuration should set this variable to an optimal value determined through experimentation on the actual CI runners. ## Acceptance Criteria - The number of test processes is configurable via a `TEST_PROCESSES` environment variable. - `noxfile.py` reads `TEST_PROCESSES` and uses it to set the number of processes for both `unit_tests` and `integration_tests` sessions. - The default value of `TEST_PROCESSES` is increased to a more reasonable value (e.g., `4` or `8`). - The CI configuration sets `TEST_PROCESSES` to the optimal value found through experimentation. - Test execution time is measurably reduced on CI. ## Supporting Information - Parent Epic: #1678 (CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing) - Related: #1911 (Parallelize unit tests), #1968 (Split integration tests for better parallelism) ## Subtasks - [ ] Make the number of test processes configurable via a `TEST_PROCESSES` environment variable. - [ ] Update `noxfile.py` to read `TEST_PROCESSES` and use it to set the number of processes for `unit_tests` and `integration_tests` sessions. - [ ] Experiment with different values for `TEST_PROCESSES` on the CI runners to find the optimal value. - [ ] Update the CI configuration (`.forgejo/workflows/ci.yml`) to set the `TEST_PROCESSES` environment variable to the optimal value. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly: `perf(ci): increase default test parallelism via configurable TEST_PROCESSES env var` - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly: `task/ci-execution-time-increase-test-parallelism` - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - The number of test processes is configurable via a `TEST_PROCESSES` environment variable. - The default number of processes is increased to a more reasonable value (e.g., `4` or `8`). - The test execution time is measurably reduced on CI. - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 09:32:08 +00:00

freemo added a new dependency

2026-04-03 09:32:43 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo referenced this issue

2026-04-03 10:55:21 +00:00

TEST-INFRA: [ci-pipeline-design] Explore Parallel Execution of Nox Sessions #2245

freemo referenced this issue

2026-04-03 11:15:41 +00:00

TEST-INFRA: [ci-execution-time] Parallelize Nox sessions in CI pipeline #2251

freemo referenced this issue