feat(ci): tune test parallelism — configure optimal worker count and parallel execution strategy #2862

Open
opened 2026-04-04 21:03:37 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: feature/m9-tune-test-parallelism
  • Commit Message: feat(ci): tune test parallelism — configure optimal worker count and parallel execution strategy
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The current test suite runs sequentially (or with untuned parallelism), which contributes to unnecessarily long CI wall-clock times. As the test suite grows, the cost of unoptimised parallelism compounds: too few workers leave CPU capacity idle, while too many workers cause resource contention, flaky tests, and unpredictable coverage collection.

This issue tracks the work to profile the test suite, determine the optimal worker count for pytest-xdist (or equivalent), and configure the parallelism settings in both the local nox sessions and the CI workflow so that all test stages run as fast as possible without introducing flakiness.

Expected Behavior

  • nox -s unit_tests and nox -s integration_tests run with an explicitly configured, empirically validated number of parallel workers.
  • The CI workflow passes the same worker-count configuration so local and CI runs are consistent.
  • No test isolation regressions are introduced (no shared-state failures, no coverage data races).
  • Overall CI test-stage wall-clock time is measurably reduced compared to the baseline.

Subtasks

  • Profile the current test suite to establish a baseline wall-clock time for unit_tests and integration_tests
  • Evaluate pytest-xdist worker strategies (auto, fixed count, logical) against the suite
  • Identify and fix any tests that are not isolation-safe for parallel execution (shared fixtures, global state, file I/O collisions)
  • Set the optimal -n <workers> flag in noxfile.py for unit_tests and integration_tests sessions
  • Propagate the same worker-count setting into .forgejo/workflows/ci.yml test job steps
  • Update features/ci_workflow_validation.feature to assert the parallelism configuration is present
  • Run nox (all default sessions) and fix any errors
  • Verify coverage >= 97% via nox -s coverage_report

Definition of Done

  • All subtasks above are completed and checked off
  • noxfile.py specifies an explicit, documented parallel worker count for all test sessions
  • .forgejo/workflows/ci.yml test steps use the same worker-count configuration
  • No new test isolation failures are introduced (all tests pass reliably across multiple runs)
  • features/ci_workflow_validation.feature covers the parallelism configuration
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `feature/m9-tune-test-parallelism` - **Commit Message**: `feat(ci): tune test parallelism — configure optimal worker count and parallel execution strategy` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current test suite runs sequentially (or with untuned parallelism), which contributes to unnecessarily long CI wall-clock times. As the test suite grows, the cost of unoptimised parallelism compounds: too few workers leave CPU capacity idle, while too many workers cause resource contention, flaky tests, and unpredictable coverage collection. This issue tracks the work to profile the test suite, determine the optimal worker count for `pytest-xdist` (or equivalent), and configure the parallelism settings in both the local `nox` sessions and the CI workflow so that all test stages run as fast as possible without introducing flakiness. ## Expected Behavior - `nox -s unit_tests` and `nox -s integration_tests` run with an explicitly configured, empirically validated number of parallel workers. - The CI workflow passes the same worker-count configuration so local and CI runs are consistent. - No test isolation regressions are introduced (no shared-state failures, no coverage data races). - Overall CI test-stage wall-clock time is measurably reduced compared to the baseline. ## Subtasks - [ ] Profile the current test suite to establish a baseline wall-clock time for `unit_tests` and `integration_tests` - [ ] Evaluate `pytest-xdist` worker strategies (`auto`, fixed count, `logical`) against the suite - [ ] Identify and fix any tests that are not isolation-safe for parallel execution (shared fixtures, global state, file I/O collisions) - [ ] Set the optimal `-n <workers>` flag in `noxfile.py` for `unit_tests` and `integration_tests` sessions - [ ] Propagate the same worker-count setting into `.forgejo/workflows/ci.yml` test job steps - [ ] Update `features/ci_workflow_validation.feature` to assert the parallelism configuration is present - [ ] Run `nox` (all default sessions) and fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] `noxfile.py` specifies an explicit, documented parallel worker count for all test sessions - [ ] `.forgejo/workflows/ci.yml` test steps use the same worker-count configuration - [ ] No new test isolation failures are introduced (all tests pass reliably across multiple runs) - [ ] `features/ci_workflow_validation.feature` covers the parallelism configuration - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-04 21:03:44 +00:00
Author
Owner

Label compliance fix applied:

  • Added missing labels: Priority/Backlog (862), State/Unverified (1320), Type/Task (1324)
  • Reason: Issue had no labels at all. Per CONTRIBUTING.md, every issue must have State/*, Priority/*, and Type/* labels. Labels inferred from issue title (feat(ci): → Type/Task, new issue → State/Unverified, Priority/Backlog as default).

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

Label compliance fix applied: - Added missing labels: `Priority/Backlog` (862), `State/Unverified` (1320), `Type/Task` (1324) - Reason: Issue had no labels at all. Per CONTRIBUTING.md, every issue must have `State/*`, `Priority/*`, and `Type/*` labels. Labels inferred from issue title (`feat(ci):` → Type/Task, new issue → State/Unverified, Priority/Backlog as default). --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2862
No description provided.