TEST-INFRA: [ci-pipeline-design] Split Integration Tests for Better Parallelization #1792

Open
opened 2026-04-02 23:51:10 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-split-integration-tests-parallelization
  • Commit Message: chore(ci): split integration tests into parallel jobs for faster feedback
  • Milestone: v3.7.0
  • Parent Epic: #1678

Description

The current CI pipeline runs all integration tests in a single integration_tests job. To improve performance and provide faster feedback, we should split this job into smaller, more focused jobs that can run in parallel.

Background and Context

As the test suite grows, a monolithic integration_tests job becomes a bottleneck in the CI pipeline. Splitting it into logical groups that run in parallel will reduce total pipeline duration and surface failures faster, improving developer feedback loops. This work is part of the broader CI execution time optimization effort tracked in Epic #1678.

Expected Behavior

Integration tests are organized into logical groups (by feature, component, or service) and run as parallel jobs in the CI pipeline. All test reports are collected and archived regardless of which parallel job produces them.

Acceptance Criteria

  • The integration_tests job is replaced by multiple smaller, focused jobs that run in parallel.
  • Each new job covers a distinct logical group of integration tests.
  • All new integration test jobs are configured to run concurrently in .forgejo/workflows/ci.yml.
  • Test reports from all parallel integration test jobs are collected and archived as CI artifacts.
  • The changes are documented in the project's development documentation.

Subtasks

  • Analyze the existing integration tests to identify logical groups for splitting (e.g., by feature, component, or service).
  • Create new nox sessions for each group of integration tests.
  • Modify the .forgejo/workflows/ci.yml file to run the new integration test jobs in parallel.
  • Ensure that the test reports from all integration test jobs are collected and archived.
  • Update development documentation to reflect the new CI structure.
  • Run nox (all default sessions) and fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

  • The integration_tests job is split into smaller, more focused jobs.
  • The new integration test jobs run in parallel in the CI pipeline.
  • The test results from all integration test jobs are collected and archived.
  • The changes are documented in the project's development documentation.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-split-integration-tests-parallelization` - **Commit Message**: `chore(ci): split integration tests into parallel jobs for faster feedback` - **Milestone**: v3.7.0 - **Parent Epic**: #1678 ## Description The current CI pipeline runs all integration tests in a single `integration_tests` job. To improve performance and provide faster feedback, we should split this job into smaller, more focused jobs that can run in parallel. ## Background and Context As the test suite grows, a monolithic `integration_tests` job becomes a bottleneck in the CI pipeline. Splitting it into logical groups that run in parallel will reduce total pipeline duration and surface failures faster, improving developer feedback loops. This work is part of the broader CI execution time optimization effort tracked in Epic #1678. ## Expected Behavior Integration tests are organized into logical groups (by feature, component, or service) and run as parallel jobs in the CI pipeline. All test reports are collected and archived regardless of which parallel job produces them. ## Acceptance Criteria - The `integration_tests` job is replaced by multiple smaller, focused jobs that run in parallel. - Each new job covers a distinct logical group of integration tests. - All new integration test jobs are configured to run concurrently in `.forgejo/workflows/ci.yml`. - Test reports from all parallel integration test jobs are collected and archived as CI artifacts. - The changes are documented in the project's development documentation. ## Subtasks - [ ] Analyze the existing integration tests to identify logical groups for splitting (e.g., by feature, component, or service). - [ ] Create new `nox` sessions for each group of integration tests. - [ ] Modify the `.forgejo/workflows/ci.yml` file to run the new integration test jobs in parallel. - [ ] Ensure that the test reports from all integration test jobs are collected and archived. - [ ] Update development documentation to reflect the new CI structure. - [ ] Run `nox` (all default sessions) and fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done - [ ] The `integration_tests` job is split into smaller, more focused jobs. - [ ] The new integration test jobs run in parallel in the CI pipeline. - [ ] The test results from all integration test jobs are collected and archived. - [ ] The changes are documented in the project's development documentation. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-02 23:51:25 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1792
No description provided.