TEST-INFRA: [ci-pipeline-design] Parallelize unit tests #1911

New issue

Open

opened 2026-04-03 00:11:53 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 00:11:53 +00:00

Owner

Metadata

Branch: task/ci-parallelize-unit-tests
Commit Message: chore(ci): parallelize unit tests using matrix strategy in CI pipeline
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The unit_tests job in the CI pipeline runs all unit tests sequentially in a single nox session. As the test suite grows, this becomes a bottleneck that increases overall CI wall-clock time. Parallelizing the unit tests will reduce feedback latency for contributors and speed up the overall pipeline.

Current Behavior

All Behave BDD unit test scenarios are executed sequentially within a single nox -s unit_tests invocation. There is no parallelism at the CI job level for unit tests.

Expected Behavior

Unit tests are split across multiple parallel nox sessions (or CI matrix jobs), reducing the total time for the unit_tests job. The CI pipeline continues to function correctly and all tests still pass.

Acceptance Criteria

The unit tests are parallelized across multiple CI jobs or nox sessions using a matrix strategy.
The unit_tests job completes faster as a result of the parallelization.
The CI pipeline continues to function as expected — all tests pass, coverage is maintained, and the status-check gate is not broken.
The parallelization strategy is documented in the CI workflow file via comments.

Supporting Information

Related: #1888 (Parallelize nox sessions — overlapping scope)
Related: #1835 (Split tests for parallel execution)
Related: #1873 (Parallelize default nox sessions)
Parent Epic: #1678 (CI Execution Time Optimization)

Subtasks

Audit the current unit_tests nox session to identify how tests can be split (e.g., by feature file, module, or tag)
Define a matrix strategy in .forgejo/workflows/ci.yml for the unit_tests job
Update noxfile.py to accept a shard/partition parameter if needed
Ensure the coverage job correctly aggregates results from all parallel shards
Update features/ci_workflow_validation.feature to assert the matrix strategy is present on the unit_tests job
Tests (Behave): Add/update scenarios for CI workflow validation
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): parallelize unit tests using matrix strategy in CI pipeline), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-parallelize-unit-tests).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-parallelize-unit-tests` - **Commit Message**: `chore(ci): parallelize unit tests using matrix strategy in CI pipeline` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `unit_tests` job in the CI pipeline runs all unit tests sequentially in a single `nox` session. As the test suite grows, this becomes a bottleneck that increases overall CI wall-clock time. Parallelizing the unit tests will reduce feedback latency for contributors and speed up the overall pipeline. ## Current Behavior All Behave BDD unit test scenarios are executed sequentially within a single `nox -s unit_tests` invocation. There is no parallelism at the CI job level for unit tests. ## Expected Behavior Unit tests are split across multiple parallel `nox` sessions (or CI matrix jobs), reducing the total time for the `unit_tests` job. The CI pipeline continues to function correctly and all tests still pass. ## Acceptance Criteria - The unit tests are parallelized across multiple CI jobs or nox sessions using a matrix strategy. - The `unit_tests` job completes faster as a result of the parallelization. - The CI pipeline continues to function as expected — all tests pass, coverage is maintained, and the `status-check` gate is not broken. - The parallelization strategy is documented in the CI workflow file via comments. ## Supporting Information - Related: #1888 (Parallelize nox sessions — overlapping scope) - Related: #1835 (Split tests for parallel execution) - Related: #1873 (Parallelize default nox sessions) - Parent Epic: #1678 (CI Execution Time Optimization) ## Subtasks - [ ] Audit the current `unit_tests` nox session to identify how tests can be split (e.g., by feature file, module, or tag) - [ ] Define a matrix strategy in `.forgejo/workflows/ci.yml` for the `unit_tests` job - [ ] Update `noxfile.py` to accept a shard/partition parameter if needed - [ ] Ensure the `coverage` job correctly aggregates results from all parallel shards - [ ] Update `features/ci_workflow_validation.feature` to assert the matrix strategy is present on the `unit_tests` job - [ ] Tests (Behave): Add/update scenarios for CI workflow validation - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): parallelize unit tests using matrix strategy in CI pipeline`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-parallelize-unit-tests`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 00:12:17 +00:00

freemo added a new dependency

2026-04-03 00:15:26 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 00:19:30 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Could have

label

2026-04-03 00:19:31 +00:00

freemo referenced this issue

2026-04-03 00:26:24 +00:00

TEST-INFRA: [ci-pipeline-design] Split integration tests for better parallelism #1968

freemo referenced this issue

2026-04-03 09:31:27 +00:00

TEST-INFRA: [ci-execution-time] Increase default parallelism for tests #2209