TEST-INFRA: Investigate and optimize the 'fix(test): address review findings in M3 smoke tests' CI workflow execution time #1746

Open
opened 2026-04-02 23:40:36 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-optimize-m3-smoke-tests-workflow
  • Commit Message: perf(ci): reduce execution time of M3 smoke tests workflow by 50%
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The fix(test): address review findings in M3 smoke tests workflow has been identified as one of the longest-running workflows in the CI pipeline, with an average execution time exceeding 2 hours. This excessive duration slows down developer feedback loops, consumes shared runner capacity, and delays the detection of regressions.

This issue is part of the broader CI Execution Time Optimization Epic (#1678), which addresses structural gaps in the pipeline including missing concurrency groups, absent job-level timeouts, and redundant test re-execution in the coverage job.

Current Behavior

The fix(test): address review findings in M3 smoke tests workflow currently runs for over 2 hours on average. No investigation has been performed to identify the root cause of this duration, and no optimizations (parallelization, caching, test splitting, or timeout enforcement) have been applied.

Expected Behavior

The workflow execution time is reduced by at least 50% (i.e., to under 1 hour on average) through targeted optimizations identified during investigation. The workflow continues to provide full test coverage and reliable signal.

Acceptance Criteria

  • The average execution time of the fix(test): address review findings in M3 smoke tests workflow is reduced by at least 50% (from >2 hours to <1 hour).
  • All existing tests continue to pass after optimizations are applied.
  • Optimizations are documented (parallelization strategy, caching keys, test splitting rationale, or timeout values chosen).
  • No test coverage is lost as a result of the optimizations.

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization
  • Related work: concurrency groups (#1678 child-1), job-level timeouts (#1678 child-2), coverage artifact sharing (#1678 child-3)
  • Potential optimization vectors: test parallelization, dependency/fixture caching, test suite splitting across parallel jobs, removal of redundant setup steps, enforcement of a job-level timeout-minutes

Subtasks

  • Profile the current workflow to identify the slowest steps/jobs (use CI run timing data)
  • Identify root cause(s) of the >2 hour execution time (e.g., sequential test execution, missing caching, redundant installs)
  • Evaluate and select optimization strategies (parallelization, caching, test splitting, timeout enforcement)
  • Implement selected optimizations in .forgejo/workflows/ci.yml and/or test configuration
  • Add or update features/ci_workflow_validation.feature to assert the optimized workflow structure
  • Measure post-optimization execution time and confirm ≥50% reduction
  • Verify coverage ≥ 97% via nox -s coverage_report
  • Run nox (all default sessions) and fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • The average execution time of the fix(test): address review findings in M3 smoke tests workflow is demonstrably reduced by at least 50%.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (perf(ci): reduce execution time of M3 smoke tests workflow by 50%), followed by a blank line, then additional lines providing relevant implementation details.
  • The commit is pushed to the remote on the branch chore/ci-optimize-m3-smoke-tests-workflow.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-optimize-m3-smoke-tests-workflow` - **Commit Message**: `perf(ci): reduce execution time of M3 smoke tests workflow by 50%` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `fix(test): address review findings in M3 smoke tests` workflow has been identified as one of the longest-running workflows in the CI pipeline, with an average execution time exceeding 2 hours. This excessive duration slows down developer feedback loops, consumes shared runner capacity, and delays the detection of regressions. This issue is part of the broader CI Execution Time Optimization Epic (#1678), which addresses structural gaps in the pipeline including missing concurrency groups, absent job-level timeouts, and redundant test re-execution in the coverage job. ## Current Behavior The `fix(test): address review findings in M3 smoke tests` workflow currently runs for over 2 hours on average. No investigation has been performed to identify the root cause of this duration, and no optimizations (parallelization, caching, test splitting, or timeout enforcement) have been applied. ## Expected Behavior The workflow execution time is reduced by at least 50% (i.e., to under 1 hour on average) through targeted optimizations identified during investigation. The workflow continues to provide full test coverage and reliable signal. ## Acceptance Criteria - [ ] The average execution time of the `fix(test): address review findings in M3 smoke tests` workflow is reduced by at least 50% (from >2 hours to <1 hour). - [ ] All existing tests continue to pass after optimizations are applied. - [ ] Optimizations are documented (parallelization strategy, caching keys, test splitting rationale, or timeout values chosen). - [ ] No test coverage is lost as a result of the optimizations. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization - Related work: concurrency groups (#1678 child-1), job-level timeouts (#1678 child-2), coverage artifact sharing (#1678 child-3) - Potential optimization vectors: test parallelization, dependency/fixture caching, test suite splitting across parallel jobs, removal of redundant setup steps, enforcement of a job-level `timeout-minutes` ## Subtasks - [ ] Profile the current workflow to identify the slowest steps/jobs (use CI run timing data) - [ ] Identify root cause(s) of the >2 hour execution time (e.g., sequential test execution, missing caching, redundant installs) - [ ] Evaluate and select optimization strategies (parallelization, caching, test splitting, timeout enforcement) - [ ] Implement selected optimizations in `.forgejo/workflows/ci.yml` and/or test configuration - [ ] Add or update `features/ci_workflow_validation.feature` to assert the optimized workflow structure - [ ] Measure post-optimization execution time and confirm ≥50% reduction - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions) and fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - The average execution time of the `fix(test): address review findings in M3 smoke tests` workflow is demonstrably reduced by at least 50%. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`perf(ci): reduce execution time of M3 smoke tests workflow by 50%`), followed by a blank line, then additional lines providing relevant implementation details. - The commit is pushed to the remote on the branch `chore/ci-optimize-m3-smoke-tests-workflow`. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:40:52 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1746
No description provided.