perf(tests): optimize coverage instrumentation and reporting pipeline #484

Closed
brent.edwards wants to merge 3 commits from perf/optimize-coverage-pipeline into master

3 Commits

Author SHA1 Message Date
brent.edwards fd82a9852c Merge branch 'master' into perf/optimize-coverage-pipeline
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 23s
CI / security (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 55s
CI / integration_tests (pull_request) Successful in 2m55s
CI / benchmark-regression (pull_request) Successful in 21m55s
CI / unit_tests (pull_request) Successful in 24m41s
CI / docker (pull_request) Successful in 52s
CI / coverage (pull_request) Successful in 24m45s
2026-03-01 05:58:21 +00:00
brent.edwards 812a880811 Merge branch 'master' into perf/optimize-coverage-pipeline
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 24s
CI / quality (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 33s
CI / security (pull_request) Successful in 50s
CI / integration_tests (pull_request) Successful in 4m44s
CI / coverage (pull_request) Successful in 12m31s
CI / benchmark-regression (pull_request) Successful in 20m58s
CI / unit_tests (pull_request) Successful in 25m18s
CI / docker (pull_request) Successful in 38s
2026-03-01 03:53:44 +00:00
brent.edwards 4629c44fc0 perf(tests): optimize coverage instrumentation and reporting pipeline
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 17s
CI / security (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 33s
CI / integration_tests (pull_request) Successful in 2m42s
CI / unit_tests (pull_request) Successful in 11m53s
CI / coverage (pull_request) Successful in 11m52s
CI / docker (pull_request) Successful in 1m3s
CI / benchmark-regression (pull_request) Successful in 25m37s
Replace coverage.py (sys.settrace-based) with slipcover (bytecode-based
instrumentation) for significantly faster coverage collection:

- Each behave-parallel worker runs under slipcover, producing per-feature
  JSON coverage files with unique UUIDs to avoid write contention.
- After all workers finish, slipcover --merge combines per-worker data
  into a single build/coverage.json report.
- XML report generated via slipcover --merge --xml for CI tooling.
- Terminal report with --fail-under=97 threshold enforcement.
- Robust JSON key-fallback logic handles both slipcover ('summary') and
  coverage.py ('totals') output formats.

CI workflow fixes:
- ci.yml 'Surface coverage summary' step uses defensive key lookup
  instead of hardcoded ['totals']['percent_covered'].
- nightly-quality.yml applies same fallback pattern.

Documentation updated to reflect slipcover as the coverage tool.

Part of #478. Closes #482.
2026-03-01 02:57:36 +00:00