TEST-INFRA: [ci-pipeline-design] Parallelize CI jobs for faster execution #7063

Open
opened 2026-04-10 07:27:55 +00:00 by HAL9000 · 1 comment
Owner

Metadata

  • Branch: feature/ci-parallelize-jobs
  • Commit Message: ci(pipeline): parallelize independent CI jobs for faster execution
  • Milestone: (none — backlog, see note below)
  • Parent Epic: #5407

Summary

The current CI pipeline in .forgejo/workflows/ci.yml executes most jobs sequentially, which significantly increases the overall execution time. This can be optimized by running independent jobs in parallel.

Background and Context

The CI pipeline defines multiple jobs (lint, typecheck, security, quality, unit_tests, integration_tests, e2e_tests, coverage, benchmark-regression, benchmark-publish, build, docker, status-check). Many of these jobs have no strict dependency on each other and can safely run concurrently. Currently, unnecessary sequential ordering inflates the total wall-clock time of every CI run, slowing developer feedback loops.

Current Behavior

Most CI jobs run sequentially due to overly conservative needs chains, even when there is no actual data or artifact dependency between them.

Expected Behavior

Independent jobs run in parallel. Only jobs with genuine dependencies (e.g., status-check must wait for all others) use the needs keyword. This reduces total CI wall-clock time significantly.

Acceptance Criteria

  • Independent jobs run concurrently in the CI pipeline.
  • The status-check job runs only after all other jobs have completed.
  • All CI checks continue to pass correctly.
  • No regressions introduced in the workflow logic.

Subtasks

  • Audit .forgejo/workflows/ci.yml to map all job dependencies.
  • Modify the workflow to run independent jobs in parallel (remove unnecessary needs chains).
  • Use the needs keyword only when there is a strict dependency between jobs.
  • Ensure that the status-check job runs only after all other jobs have completed.
  • Validate the new workflow by running it on a test branch.
  • Update documentation if any CI usage instructions are affected.

Definition of Done

  • The CI pipeline execution time is measurably reduced.
  • All CI checks are still running correctly and passing.
  • The status-check job correctly aggregates results from all parallel jobs.
  • The new workflow is merged into the master branch.
  • All nox stages pass.
  • Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.7.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `feature/ci-parallelize-jobs` - **Commit Message**: `ci(pipeline): parallelize independent CI jobs for faster execution` - **Milestone**: *(none — backlog, see note below)* - **Parent Epic**: #5407 ## Summary The current CI pipeline in `.forgejo/workflows/ci.yml` executes most jobs sequentially, which significantly increases the overall execution time. This can be optimized by running independent jobs in parallel. ## Background and Context The CI pipeline defines multiple jobs (`lint`, `typecheck`, `security`, `quality`, `unit_tests`, `integration_tests`, `e2e_tests`, `coverage`, `benchmark-regression`, `benchmark-publish`, `build`, `docker`, `status-check`). Many of these jobs have no strict dependency on each other and can safely run concurrently. Currently, unnecessary sequential ordering inflates the total wall-clock time of every CI run, slowing developer feedback loops. ## Current Behavior Most CI jobs run sequentially due to overly conservative `needs` chains, even when there is no actual data or artifact dependency between them. ## Expected Behavior Independent jobs run in parallel. Only jobs with genuine dependencies (e.g., `status-check` must wait for all others) use the `needs` keyword. This reduces total CI wall-clock time significantly. ## Acceptance Criteria - Independent jobs run concurrently in the CI pipeline. - The `status-check` job runs only after all other jobs have completed. - All CI checks continue to pass correctly. - No regressions introduced in the workflow logic. ## Subtasks - [ ] Audit `.forgejo/workflows/ci.yml` to map all job dependencies. - [ ] Modify the workflow to run independent jobs in parallel (remove unnecessary `needs` chains). - [ ] Use the `needs` keyword only when there is a strict dependency between jobs. - [ ] Ensure that the `status-check` job runs only after all other jobs have completed. - [ ] Validate the new workflow by running it on a test branch. - [ ] Update documentation if any CI usage instructions are affected. ## Definition of Done - [ ] The CI pipeline execution time is measurably reduced. - [ ] All CI checks are still running correctly and passing. - [ ] The `status-check` job correctly aggregates results from all parallel jobs. - [ ] The new workflow is merged into the master branch. - [ ] All nox stages pass. - [ ] Coverage >= 97%. > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.7.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
Author
Owner

Verified — CI optimization: parallelize CI jobs. MoSCoW: Could-have. Priority: Low.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI optimization: parallelize CI jobs. MoSCoW: Could-have. Priority: Low. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#7063
No description provided.