[AUTO-INF-9] Optimize CI Execution Time: coverage job #8335

New issue

Closed

opened 2026-04-13 09:11:30 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-13 09:11:30 +00:00

Owner

Summary

The coverage job re-runs the entire Python suite under Slipcover after unit tests have already executed, adding ~13 minutes to every successful workflow (run 12455: 06:53:37Z → 07:06:42Z).
Most of that time is spent rehydrating a fresh .nox/coverage_report environment, reinstalling dependencies, and executing duplicate tests purely for coverage collection.
Consolidating coverage generation into the unit-test phase or caching the instrumented environment would immediately return double-digit minutes per PR build.

Evidence

/app/ci_logs/pr5448/coverage.log: job runtime 13m05s with nox > Session coverage_report was successful in 12 minutes.
Same log shows ~8 MB of Slipcover JSON output generated after tests finish, demonstrating the duplication of work already done in unit_tests.
Forgejo Actions run https://git.cleverthis.com/cleveragents/cleveragents-core/actions/runs/1742 lists coverage as one of the final gating jobs, extending end-to-end time even when preceding jobs complete quickly.

Root Cause

Coverage is collected in a standalone job that bootstraps a clean environment and replays the complete unit-test suite instead of reusing artifacts from unit_tests. This design incurs full dependency installation and execution costs twice per workflow.

Recommendations

Collect coverage during unit_tests (e.g., coverage run --parallel-mode) and have the coverage job only aggregate (coverage combine, coverage xml/html).
Cache the .nox/coverage_report environment via Forgejo caches so slipcover instrumentation and dependencies aren’t rebuilt each run.
Trim Slipcover output to a summary JSON/HTML and keep historical artifacts in object storage instead of uploading full reports every PR.
Fail-fast on missing data by sharing the build/nox-unit-tests-output.log artifact, avoiding the need to re-run tests when gathering coverage summaries.

Duplicate Check

Searched open issues for coverage_report / coverage job (April 13, 2026) — none found.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - The `coverage` job re-runs the entire Python suite under Slipcover after unit tests have already executed, adding ~13 minutes to every successful workflow (run 12455: 06:53:37Z → 07:06:42Z). - Most of that time is spent rehydrating a fresh `.nox/coverage_report` environment, reinstalling dependencies, and executing duplicate tests purely for coverage collection. - Consolidating coverage generation into the unit-test phase or caching the instrumented environment would immediately return double-digit minutes per PR build. ## Evidence - `/app/ci_logs/pr5448/coverage.log`: job runtime 13m05s with `nox > Session coverage_report was successful in 12 minutes`. - Same log shows ~8 MB of Slipcover JSON output generated after tests finish, demonstrating the duplication of work already done in `unit_tests`. - Forgejo Actions run https://git.cleverthis.com/cleveragents/cleveragents-core/actions/runs/1742 lists coverage as one of the final gating jobs, extending end-to-end time even when preceding jobs complete quickly. ## Root Cause - Coverage is collected in a standalone job that bootstraps a clean environment and replays the complete unit-test suite instead of reusing artifacts from `unit_tests`. This design incurs full dependency installation and execution costs twice per workflow. ## Recommendations 1. **Collect coverage during `unit_tests`** (e.g., `coverage run --parallel-mode`) and have the coverage job only aggregate (`coverage combine`, `coverage xml/html`). 2. **Cache the `.nox/coverage_report` environment** via Forgejo caches so slipcover instrumentation and dependencies aren’t rebuilt each run. 3. **Trim Slipcover output** to a summary JSON/HTML and keep historical artifacts in object storage instead of uploading full reports every PR. 4. **Fail-fast on missing data** by sharing the `build/nox-unit-tests-output.log` artifact, avoiding the need to re-run tests when gathering coverage summaries. ### Duplicate Check - Searched open issues for `coverage_report` / `coverage job` (April 13, 2026) — none found. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

Rows
Columns