feat(ci): parallel coverage engine + explicit-gated coverage job; de-razor coverage to 96.62% #11258

Merged

HAL9000 merged 5 commits from coverage-engine-master-port into master 2026-06-02 03:34:31 +00:00

Conversation 3 Commits 5 Files Changed 9

+724 -90

5 Commits

Author	SHA1	Message	Date
drew	11c0fd11d5	perf(ci): raise pabot integration parallelism cap min(2)->min(6) ... integration_tests was throttled to `min(2, _default_processes())` (commit 92e258535, a flaky-stabilization measure). Evidence from a cached CI run shows the suite does ~47min of work serialized through 2 pabot lanes on a 32-core runner -> ~24min wall (exactly 2x == 2 lanes), the chronic ~21-30min seen across recent runs. It is linearly parallelism-bound. Raise the cap to `min(6, _default_processes())`: ~4x more lanes -> projected ~6-8min wall, while staying well short of per-core fan-out (the runner reports 32 cores; behave already uses all of them). The cap is kept (not removed) because the suite makes live LLM calls — ~149 HTTP 429 rate-limit responses were observed even at 2-way, and unbounded fan-out would spike 429/OOM flakes (the exact failure 92e258535 was masking). 6 is a deliberate middle ground; TEST_PROCESSES / --processes still override. Updates the now-stale "<=2 processes" docstrings on integration_tests and slow_integration_tests. Behave/unit_tests parallelism (_default_processes(), uncapped) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-01 22:32:43 -04:00
drew	8119043837	fix(coverage): reconcile robot coverage-threshold test with pyproject delegation (WS5) ... integration_tests (Robot/pabot) failed on robot/coverage_threshold.robot "Noxfile Contains Coverage Threshold Constant": it asserted the literal `COVERAGE_THRESHOLD = 96.5` in noxfile.py, but the coverage_report session now delegates the constant to pyproject's single source (`COVERAGE_THRESHOLD = _read_coverage_fail_under()`). This was the last WS5 reconciliation gap (the plan enumerated behave feature steps + ci.yml + guidelines prose, but not the Robot suite). Assert the delegation plus the pyproject floor (`fail_under = 96.5`) instead of the literal, mirroring the behave step fix in b499834c0. The other assertions in the suite (--fail-under= substring, [tool.coverage.run], branch=true, source=["src"]) are unaffected; verified no other robot/behave/pytest test depends on the literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-01 22:11:54 -04:00
drew	b499834c0f	fix(coverage): resolve fail-under from pyproject in threshold-enforcement step (WS5) ... The coverage_report noxfile session now delegates COVERAGE_THRESHOLD to the pyproject single source (`COVERAGE_THRESHOLD = _read_coverage_fail_under()`), so the AST value is a Call, not a Constant. The coverage_threshold_enforcement.feature step "I parse the COVERAGE_THRESHOLD constant from noxfile.py" raised ValueError (errored scenario) because it only handled a literal constant — this was the missed third WS5 step file (the config + consolidated steps were already reconciled). Add the same pyproject `[tool.coverage.report].fail_under` fallback used by coverage_threshold_config_steps.py, and fix the phantom-97 in the feature description prose. Verified: the four coverage feature files now pass (136 scenarios, 0 errored); previously unit_tests errored on coverage_threshold_enforcement.feature:7. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-01 17:37:24 -04:00
drew	a49f37eb1f	test(coverage): de-razor master coverage 96.465% -> 96.623% ... Master passed the 96.5 floor only by rounding (96.465 -> 96.5). Add genuine behave coverage + omit a structurally-uncoverable module so the floor has real headroom. fail_under stays 96.5 (ratchet rule: raise only after coverage holds). - omit src/cleveragents/application/services/__init__.py (noxfile + pyproject): 100% of its "missing" lines are inside an `if TYPE_CHECKING:` block (never executes at runtime) and slipcover has no per-line pragma to exclude them. - features/coverage_validation_error_paths.feature (+ steps): 16 scenarios exercising the defensive error-path branches in core/validation.py that the existing structural_validation.feature does not reach (non-dict node, dup decision_id, non-list children, invalid child ULID, missing decision fields, wrong-typed confidence/parent/sequence, malformed structured-output, unknown dispatcher target). Routed through the public validate_structured_component_output dispatcher; pure functions, no DB/CLI. Full engine run: NOX_EXIT=0, no dead chunks, 96.623% (validation.py now fully covered). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-01 17:07:39 -04:00
drew	4dec646e2f	feat(ci): parallel coverage engine + explicit nox-gated coverage job (Phase 2) ... Ports the WS1 fast/parallel coverage engine onto master and reconciles the coverage gate to a single source. Local full-suite run on master's suite: NOX_EXIT=0, all 12 chunks alive (no dead chunks), peak 658MB/chunk (ceiling ~2.6GB vs the old ~1.4GB single-process that OOMs/reaper-kills), 96.465% -> 96.5 (rounded) >= 96.5 floor. noxfile.py — coverage_report full-suite path now fans out K concurrent slipcover processes (K=COVERAGE_PROCESSES, default 4) over N bin-packed chunks (by scenario count), failing loud on any dead chunk (never merges survivors) and merging per-chunk JSON. Bounds per-process peak RSS, killing the single-process OOM/reaper collapse. Targeted (.feature posargs) runs keep the single-process path. COVERAGE_THRESHOLD now reads pyproject [tool.coverage.report].fail_under. Features are enumerated by direct glob (NOT by importing run_behave_parallel, whose top level imports behave/behave_parallel and is unavailable in the nox orchestrator process). pyproject.toml — adds [tool.coverage.report].fail_under = 96.5 as the single source of truth, with the evidence-gated ratchet rule (objective 97%). .forgejo/workflows/ci.yml — coverage job: carries the skip_coverage operator valve; propagates nox's exit EXPLICITLY (set -uo pipefail + PIPESTATUS + exit $rc) instead of relying on the runner's implicit bash -eo pipefail; adds timeout-minutes: 30 so a hang fails cleanly with diagnostics; deletes the dead threshold=50 "Surface coverage summary" step; fixes the phantom-97 step label. Gating remains nox's --fail-under (sourced from pyproject). coverage_threshold_config_steps.py — the fail-under feature step resolves the floor from pyproject when COVERAGE_THRESHOLD delegates to the reader. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-01 16:32:53 -04:00