TEST-INFRA: [ci-execution-time] Parallelize test execution using a matrix strategy #2339

New issue

Closed

opened 2026-04-03 14:51:45 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 14:51:45 +00:00

Owner

Metadata

Branch: task/ci-execution-time-matrix-test-parallelism
Commit Message: chore(ci): parallelize unit, integration, and e2e test jobs using a matrix strategy
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The unit_tests, integration_tests, and e2e_tests jobs in .forgejo/workflows/ci.yml currently run all tests within a single sequential session. As the test suites grow, this sequential execution becomes a major bottleneck in the CI pipeline, directly contributing to the wall-clock time problem tracked in the parent Epic (#1678).

By introducing a Forgejo Actions matrix strategy for these jobs, the test suites can be split into multiple parallel shards, significantly reducing the overall time required to validate a pull request.

Expected Behavior

Each of the unit_tests, integration_tests, and e2e_tests CI jobs should use a matrix strategy to distribute test execution across multiple parallel runners. Test results from all matrix shards must be collected and aggregated so that the overall job status correctly reflects the combined outcome.

Acceptance Criteria

The unit_tests job uses a matrix strategy to split and run tests in parallel shards.
The integration_tests job uses a matrix strategy to split and run tests in parallel shards.
The e2e_tests job uses a matrix strategy to split and run tests in parallel shards.
Test results from all matrix shards are collected and the overall CI status is correctly reported.
The total wall-clock time for the test-related CI jobs is demonstrably reduced compared to the sequential baseline.

Subtasks

Modify .forgejo/workflows/ci.yml to implement a test-splitting and matrix execution strategy for the unit_tests job.
Extend the matrix strategy to the integration_tests job.
Apply a similar parallelization strategy to the e2e_tests job.
Ensure test results from all matrix jobs are properly collected and reported.
Update features/ci_workflow_validation.feature to assert the matrix strategy is present for the relevant jobs.
Run nox (all default sessions) and fix any errors.
Verify coverage ≥ 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): parallelize unit, integration, and e2e test jobs using a matrix strategy), followed by a blank line, then additional lines providing relevant implementation details.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-execution-time-matrix-test-parallelism).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-matrix-test-parallelism` - **Commit Message**: `chore(ci): parallelize unit, integration, and e2e test jobs using a matrix strategy` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `unit_tests`, `integration_tests`, and `e2e_tests` jobs in `.forgejo/workflows/ci.yml` currently run all tests within a single sequential session. As the test suites grow, this sequential execution becomes a major bottleneck in the CI pipeline, directly contributing to the wall-clock time problem tracked in the parent Epic (#1678). By introducing a Forgejo Actions matrix strategy for these jobs, the test suites can be split into multiple parallel shards, significantly reducing the overall time required to validate a pull request. ## Expected Behavior Each of the `unit_tests`, `integration_tests`, and `e2e_tests` CI jobs should use a matrix strategy to distribute test execution across multiple parallel runners. Test results from all matrix shards must be collected and aggregated so that the overall job status correctly reflects the combined outcome. ## Acceptance Criteria - The `unit_tests` job uses a matrix strategy to split and run tests in parallel shards. - The `integration_tests` job uses a matrix strategy to split and run tests in parallel shards. - The `e2e_tests` job uses a matrix strategy to split and run tests in parallel shards. - Test results from all matrix shards are collected and the overall CI status is correctly reported. - The total wall-clock time for the test-related CI jobs is demonstrably reduced compared to the sequential baseline. ## Subtasks - [ ] Modify `.forgejo/workflows/ci.yml` to implement a test-splitting and matrix execution strategy for the `unit_tests` job. - [ ] Extend the matrix strategy to the `integration_tests` job. - [ ] Apply a similar parallelization strategy to the `e2e_tests` job. - [ ] Ensure test results from all matrix jobs are properly collected and reported. - [ ] Update `features/ci_workflow_validation.feature` to assert the matrix strategy is present for the relevant jobs. - [ ] Run `nox` (all default sessions) and fix any errors. - [ ] Verify coverage ≥ 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): parallelize unit, integration, and e2e test jobs using a matrix strategy`), followed by a blank line, then additional lines providing relevant implementation details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-execution-time-matrix-test-parallelism`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 14:51:52 +00:00

freemo added a new dependency

2026-04-03 14:54:13 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 17:23:49 +00:00

Author

Owner

Closing as duplicate of #2293.

This issue ("Parallelize test execution using a matrix strategy") covers the same scope as #2293 ("Parallelize Nox Sessions in CI"), which already includes matrix/parallel job strategies. Consolidating to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2293. This issue ("Parallelize test execution using a matrix strategy") covers the same scope as #2293 ("Parallelize Nox Sessions in CI"), which already includes matrix/parallel job strategies. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 17:24:03 +00:00

freemo referenced this issue

2026-04-03 17:24:30 +00:00

TEST-INFRA: [ci-execution-time] Parallelize Nox Sessions in CI #2293