TEST-INFRA: [ci-execution-time] Parallelize CI jobs to reduce execution time #1703

Closed
opened 2026-04-02 23:31:30 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/ci-parallelize-workflow-jobs
  • Commit Message: ci(workflow): parallelize independent CI jobs to reduce overall pipeline execution time
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The current CI pipeline in .forgejo/workflows/ci.yml runs most jobs sequentially, which increases the overall wall-clock execution time. Many of the jobs — lint, typecheck, security, quality, and the various test jobs — are fully independent of one another and have no inter-dependencies. Running them in parallel would significantly reduce the time contributors wait for CI feedback.

Current Behavior

Jobs in the CI pipeline execute sequentially (or with minimal parallelism), causing the total pipeline duration to be the sum of all individual job durations rather than the duration of the longest parallel path.

Expected Behavior

Independent CI jobs run concurrently. The coverage job depends only on the test jobs that produce coverage data. The status-check job depends on all parallelized jobs, acting as a single gate for merge readiness.

Acceptance Criteria

  • The following jobs run in parallel with no sequential dependency between them:
    • lint
    • typecheck
    • security
    • quality
    • unit_tests
    • integration_tests
    • e2e_tests
  • The coverage job declares needs on the relevant test jobs (those that produce coverage artifacts).
  • The status-check job declares needs on all parallelized jobs, ensuring it only passes when every job succeeds.
  • All CI jobs continue to pass successfully after the restructuring.
  • Total CI pipeline wall-clock time is measurably reduced.

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization
  • Related: #1679 (parallelizing nox sessions within a job — complementary but distinct scope)
  • File to modify: .forgejo/workflows/ci.yml

Subtasks

  • Audit .forgejo/workflows/ci.yml to identify all jobs and their current dependency graph
  • Identify which jobs are truly independent (no shared mutable state or ordering requirements)
  • Restructure needs: declarations so that lint, typecheck, security, quality, unit_tests, integration_tests, and e2e_tests run in parallel
  • Update coverage job needs: to depend only on the test jobs that emit coverage data
  • Update status-check job needs: to depend on all parallelized jobs
  • Verify the updated workflow is valid YAML and passes Forgejo/Gitea workflow linting
  • Run the updated pipeline and confirm all jobs pass
  • Measure and document the before/after wall-clock time improvement
  • Run nox (all default sessions), fix any errors
  • Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (ci(workflow): parallelize independent CI jobs to reduce overall pipeline execution time), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-parallelize-workflow-jobs).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-parallelize-workflow-jobs` - **Commit Message**: `ci(workflow): parallelize independent CI jobs to reduce overall pipeline execution time` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline in `.forgejo/workflows/ci.yml` runs most jobs sequentially, which increases the overall wall-clock execution time. Many of the jobs — `lint`, `typecheck`, `security`, `quality`, and the various test jobs — are fully independent of one another and have no inter-dependencies. Running them in parallel would significantly reduce the time contributors wait for CI feedback. ## Current Behavior Jobs in the CI pipeline execute sequentially (or with minimal parallelism), causing the total pipeline duration to be the sum of all individual job durations rather than the duration of the longest parallel path. ## Expected Behavior Independent CI jobs run concurrently. The `coverage` job depends only on the test jobs that produce coverage data. The `status-check` job depends on all parallelized jobs, acting as a single gate for merge readiness. ## Acceptance Criteria - The following jobs run in parallel with no sequential dependency between them: - `lint` - `typecheck` - `security` - `quality` - `unit_tests` - `integration_tests` - `e2e_tests` - The `coverage` job declares `needs` on the relevant test jobs (those that produce coverage artifacts). - The `status-check` job declares `needs` on all parallelized jobs, ensuring it only passes when every job succeeds. - All CI jobs continue to pass successfully after the restructuring. - Total CI pipeline wall-clock time is measurably reduced. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization - Related: #1679 (parallelizing nox sessions within a job — complementary but distinct scope) - File to modify: `.forgejo/workflows/ci.yml` ## Subtasks - [ ] Audit `.forgejo/workflows/ci.yml` to identify all jobs and their current dependency graph - [ ] Identify which jobs are truly independent (no shared mutable state or ordering requirements) - [ ] Restructure `needs:` declarations so that `lint`, `typecheck`, `security`, `quality`, `unit_tests`, `integration_tests`, and `e2e_tests` run in parallel - [ ] Update `coverage` job `needs:` to depend only on the test jobs that emit coverage data - [ ] Update `status-check` job `needs:` to depend on all parallelized jobs - [ ] Verify the updated workflow is valid YAML and passes Forgejo/Gitea workflow linting - [ ] Run the updated pipeline and confirm all jobs pass - [ ] Measure and document the before/after wall-clock time improvement - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`ci(workflow): parallelize independent CI jobs to reduce overall pipeline execution time`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-parallelize-workflow-jobs`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:33:33 +00:00
Author
Owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests).


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests). --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo 2026-04-02 23:41:35 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1703
No description provided.