[AUTO-INF-1] Parallelize slipcover coverage to eliminate duplicate 12 minute behave pass #8244

New issue

Open

opened 2026-04-13 06:17:35 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-13 06:17:35 +00:00

Owner

Summary

The coverage job reruns the entire behave suite sequentially under Slipcover, adding ~12 minutes to every CI run.
This coverage pass duplicates the unit_tests job work (590 features, 56k steps) but disables parallelism, so the pipeline effectively runs the same scenarios twice.

Evidence

.forgejo/workflows/ci.yml lines 334-395 run nox -s coverage_report after unit tests.
ci_logs/pr5448/coverage.log:123411 — nox > Session coverage_report was successful in 12 minutes.
ci_logs/pr5448/unit_tests.log:103242-103247 — the unit test job already exercises the same 590 features in 4m29s wall time.
scripts/run_behave_parallel.py forces sequential mode whenever BEHAVE_PARALLEL_COVERAGE is set, so coverage always runs single-threaded.

Proposal

Extend scripts/run_behave_parallel.py with a coverage chunk mode (e.g. env BEHAVE_COVERAGE_CHUNKS=4):
- split the discovered feature files into N chunks;
- for each chunk spawn python -m slipcover --json --out build/coverage-chunk-<n>.json -- ... <features> in parallel via ProcessPoolExecutor;
- copy the template DB to build/.template-migrated-<n>.db per chunk to avoid cross-process writes.
After the parallel runs finish, merge the chunk artifacts with slipcover --merge to produce the existing build/coverage.json, coverage.xml, and textual summary (preserving the 97% threshold gate).
Update .forgejo/workflows/ci.yml to set BEHAVE_COVERAGE_CHUNKS=4 (and fall back to 1 when the env is absent) so coverage runs in ~¼ the time while maintaining deterministic coverage data.
Keep uploading the merged coverage artifacts so downstream tooling is unaffected.

Duplicate Check

✅ curl --compressed ...issues?state=open&limit=50 filtered for coverage → only status/epic tickets, no CI runtime work.
✅ curl --compressed ...issues?state=open&limit=50 filtered for slipcover → no results.
✅ curl --compressed ...issues?state=closed&limit=50 filtered for coverage → automation status items only.
✅ curl --compressed ...issues?state=open&labels=TEST-INFRA → no coverage pipeline improvements logged.
✅ Confident this request is new.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - The `coverage` job reruns the entire behave suite sequentially under Slipcover, adding ~12 minutes to every CI run. - This coverage pass duplicates the `unit_tests` job work (590 features, 56k steps) but disables parallelism, so the pipeline effectively runs the same scenarios twice. ## Evidence - `.forgejo/workflows/ci.yml` lines 334-395 run `nox -s coverage_report` after unit tests. - `ci_logs/pr5448/coverage.log:123411` — `nox > Session coverage_report was successful in 12 minutes.` - `ci_logs/pr5448/unit_tests.log:103242-103247` — the unit test job already exercises the same 590 features in 4m29s wall time. - `scripts/run_behave_parallel.py` forces sequential mode whenever `BEHAVE_PARALLEL_COVERAGE` is set, so coverage always runs single-threaded. ## Proposal 1. Extend `scripts/run_behave_parallel.py` with a coverage chunk mode (e.g. env `BEHAVE_COVERAGE_CHUNKS=4`): - split the discovered feature files into N chunks; - for each chunk spawn `python -m slipcover --json --out build/coverage-chunk-<n>.json -- ... <features>` in parallel via `ProcessPoolExecutor`; - copy the template DB to `build/.template-migrated-<n>.db` per chunk to avoid cross-process writes. 2. After the parallel runs finish, merge the chunk artifacts with `slipcover --merge` to produce the existing `build/coverage.json`, `coverage.xml`, and textual summary (preserving the 97% threshold gate). 3. Update `.forgejo/workflows/ci.yml` to set `BEHAVE_COVERAGE_CHUNKS=4` (and fall back to `1` when the env is absent) so coverage runs in ~¼ the time while maintaining deterministic coverage data. 4. Keep uploading the merged coverage artifacts so downstream tooling is unaffected. ### Duplicate Check - ✅ `curl --compressed ...issues?state=open&limit=50` filtered for `coverage` → only status/epic tickets, no CI runtime work. - ✅ `curl --compressed ...issues?state=open&limit=50` filtered for `slipcover` → no results. - ✅ `curl --compressed ...issues?state=closed&limit=50` filtered for `coverage` → automation status items only. - ✅ `curl --compressed ...issues?state=open&labels=TEST-INFRA` → no coverage pipeline improvements logged. - ✅ Confident this request is new. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

Rows
Columns