TEST-INFRA: [ci-execution-time] Parallelize integration tests #1705

New issue

Open

opened 2026-04-02 23:31:32 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:31:32 +00:00

Owner

Metadata

Branch: chore/ci-parallelize-integration-tests
Commit Message: chore(ci): parallelize integration test suite to reduce CI execution time
Milestone: v3.8.0
Parent Epic: #1678

Summary

The integration test suite is taking a significant amount of time to run, contributing to a slow CI/CD pipeline. This slows down the development feedback loop and increases the time to merge pull requests.

Problem

The integration tests are currently running sequentially, which takes approximately 21 minutes to complete. This is a major bottleneck in the CI pipeline.

Proposal

To address this, we propose to parallelize the execution of the integration tests. By running the tests in parallel, we can significantly reduce the overall execution time.

We can achieve this by:

Splitting the test suite into multiple smaller chunks.
Running each chunk on a separate CI runner.

Acceptance Criteria

The integration test suite is parallelized.
The execution time of the integration test suite is reduced by at least 50%.
The CI pipeline remains reliable and stable.

Subtasks

Audit the integration test suite to identify parallelization boundaries and any shared state / ordering dependencies
Define a sharding/chunking strategy (e.g., by feature file, by tag, or by directory) and document it
Update .forgejo/workflows/ci.yml to use a matrix strategy that fans out integration test chunks across parallel runners
Ensure each shard is independently runnable (no cross-shard state leakage)
Update features/ci_workflow_validation.feature to assert the matrix/parallel integration test steps are present
Verify total wall-clock time for integration tests is reduced by ≥ 50%
Verify coverage ≥ 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): parallelize integration test suite to reduce CI execution time), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (chore/ci-parallelize-integration-tests).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-parallelize-integration-tests` - **Commit Message**: `chore(ci): parallelize integration test suite to reduce CI execution time` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Summary The integration test suite is taking a significant amount of time to run, contributing to a slow CI/CD pipeline. This slows down the development feedback loop and increases the time to merge pull requests. ## Problem The integration tests are currently running sequentially, which takes approximately 21 minutes to complete. This is a major bottleneck in the CI pipeline. ## Proposal To address this, we propose to parallelize the execution of the integration tests. By running the tests in parallel, we can significantly reduce the overall execution time. We can achieve this by: - Splitting the test suite into multiple smaller chunks. - Running each chunk on a separate CI runner. ## Acceptance Criteria - The integration test suite is parallelized. - The execution time of the integration test suite is reduced by at least 50%. - The CI pipeline remains reliable and stable. ## Subtasks - [ ] Audit the integration test suite to identify parallelization boundaries and any shared state / ordering dependencies - [ ] Define a sharding/chunking strategy (e.g., by feature file, by tag, or by directory) and document it - [ ] Update `.forgejo/workflows/ci.yml` to use a matrix strategy that fans out integration test chunks across parallel runners - [ ] Ensure each shard is independently runnable (no cross-shard state leakage) - [ ] Update `features/ci_workflow_validation.feature` to assert the matrix/parallel integration test steps are present - [ ] Verify total wall-clock time for integration tests is reduced by ≥ 50% - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): parallelize integration test suite to reduce CI execution time`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`chore/ci-parallelize-integration-tests`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-02 23:31:55 +00:00

freemo commented

2026-04-02 23:41:34 +00:00

Author

Owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests).

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests). --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo

2026-04-02 23:41:34 +00:00