TEST-INFRA: [ci-execution-time] Optimize CI job parallelism #1741

New issue

Closed

opened 2026-04-02 23:39:23 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:39:23 +00:00

Owner

Metadata

Branch: chore/ci-optimize-job-parallelism
Commit Message: chore(ci): optimize job parallelism to reduce overall pipeline execution time
Milestone: v3.2.0
Parent Epic: #1678

Background and Context

The CI pipeline defined in .forgejo/workflows/ci.yml can be optimized to run more jobs in parallel, which will reduce the overall execution time. Currently, many jobs run sequentially even when they are fully independent of one another, causing unnecessary delays in feedback cycles.

Current Behavior

The CI pipeline runs many jobs sequentially, even when they are independent. For example, the lint, typecheck, security, and quality jobs are all independent and could be run in parallel. Similarly, the unit_tests, integration_tests, and e2e_tests jobs are also independent and could be run in parallel.

Expected Behavior

The following jobs should run in parallel with no inter-dependencies between them:

lint
typecheck
security
quality
unit_tests
integration_tests
e2e_tests

The coverage and benchmark-regression jobs should still depend on lint and typecheck. The docker job should still depend on lint, typecheck, unit_tests, and security.

Acceptance Criteria

The lint, typecheck, security, quality, unit_tests, integration_tests, and e2e_tests jobs run in parallel (no needs: dependency between them).
The coverage and benchmark-regression jobs retain their needs: [lint, typecheck] dependency.
The docker job retains its needs: [lint, typecheck, unit_tests, security] dependency.
The overall CI wall-clock execution time is measurably reduced compared to the baseline.
All CI jobs continue to pass after the restructuring.

Supporting Information

Relevant file: .forgejo/workflows/ci.yml
Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

Subtasks

Audit .forgejo/workflows/ci.yml and map current job dependency graph
Remove sequential needs: constraints between lint, typecheck, security, quality, unit_tests, integration_tests, and e2e_tests
Verify coverage and benchmark-regression still declare needs: [lint, typecheck]
Verify docker still declares needs: [lint, typecheck, unit_tests, security]
Trigger a CI run and confirm all jobs execute correctly in the new parallel configuration
Measure and document the reduction in overall pipeline execution time
Run nox (all default sessions), fix any errors
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): optimize job parallelism to reduce overall pipeline execution time), followed by a blank line, then additional lines providing relevant implementation details.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (chore/ci-optimize-job-parallelism).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
The CI pipeline runs the specified jobs in parallel and the overall CI execution time is reduced.
All nox stages pass.
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-optimize-job-parallelism` - **Commit Message**: `chore(ci): optimize job parallelism to reduce overall pipeline execution time` - **Milestone**: v3.2.0 - **Parent Epic**: #1678 ## Background and Context The CI pipeline defined in `.forgejo/workflows/ci.yml` can be optimized to run more jobs in parallel, which will reduce the overall execution time. Currently, many jobs run sequentially even when they are fully independent of one another, causing unnecessary delays in feedback cycles. ## Current Behavior The CI pipeline runs many jobs sequentially, even when they are independent. For example, the `lint`, `typecheck`, `security`, and `quality` jobs are all independent and could be run in parallel. Similarly, the `unit_tests`, `integration_tests`, and `e2e_tests` jobs are also independent and could be run in parallel. ## Expected Behavior The following jobs should run in parallel with no inter-dependencies between them: - `lint` - `typecheck` - `security` - `quality` - `unit_tests` - `integration_tests` - `e2e_tests` The `coverage` and `benchmark-regression` jobs should still depend on `lint` and `typecheck`. The `docker` job should still depend on `lint`, `typecheck`, `unit_tests`, and `security`. ## Acceptance Criteria - The `lint`, `typecheck`, `security`, `quality`, `unit_tests`, `integration_tests`, and `e2e_tests` jobs run in parallel (no `needs:` dependency between them). - The `coverage` and `benchmark-regression` jobs retain their `needs: [lint, typecheck]` dependency. - The `docker` job retains its `needs: [lint, typecheck, unit_tests, security]` dependency. - The overall CI wall-clock execution time is measurably reduced compared to the baseline. - All CI jobs continue to pass after the restructuring. ## Supporting Information - Relevant file: `.forgejo/workflows/ci.yml` - Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing ## Subtasks - [ ] Audit `.forgejo/workflows/ci.yml` and map current job dependency graph - [ ] Remove sequential `needs:` constraints between `lint`, `typecheck`, `security`, `quality`, `unit_tests`, `integration_tests`, and `e2e_tests` - [ ] Verify `coverage` and `benchmark-regression` still declare `needs: [lint, typecheck]` - [ ] Verify `docker` still declares `needs: [lint, typecheck, unit_tests, security]` - [ ] Trigger a CI run and confirm all jobs execute correctly in the new parallel configuration - [ ] Measure and document the reduction in overall pipeline execution time - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): optimize job parallelism to reduce overall pipeline execution time`), followed by a blank line, then additional lines providing relevant implementation details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`chore/ci-optimize-job-parallelism`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - The CI pipeline runs the specified jobs in parallel and the overall CI execution time is reduced. - All nox stages pass. - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.2.0 milestone

2026-04-02 23:40:28 +00:00

freemo commented

2026-04-02 23:41:31 +00:00

Author

Owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests).

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate. CI parallelization and execution time improvements are tracked in #1604 (setup consolidation), #1536 (parallelize static analysis), and #1632 (split integration tests). --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo

2026-04-02 23:41:31 +00:00