5 Commits

Author SHA1 Message Date
drew 4dec646e2f feat(ci): parallel coverage engine + explicit nox-gated coverage job (Phase 2)
Ports the WS1 fast/parallel coverage engine onto master and reconciles the
coverage gate to a single source. Local full-suite run on master's suite:
NOX_EXIT=0, all 12 chunks alive (no dead chunks), peak 658MB/chunk
(ceiling ~2.6GB vs the old ~1.4GB single-process that OOMs/reaper-kills),
96.465% -> 96.5 (rounded) >= 96.5 floor.

noxfile.py — coverage_report full-suite path now fans out K concurrent
slipcover processes (K=COVERAGE_PROCESSES, default 4) over N bin-packed
chunks (by scenario count), failing loud on any dead chunk (never merges
survivors) and merging per-chunk JSON. Bounds per-process peak RSS, killing
the single-process OOM/reaper collapse. Targeted (.feature posargs) runs keep
the single-process path. COVERAGE_THRESHOLD now reads
pyproject [tool.coverage.report].fail_under. Features are enumerated by direct
glob (NOT by importing run_behave_parallel, whose top level imports
behave/behave_parallel and is unavailable in the nox orchestrator process).

pyproject.toml — adds [tool.coverage.report].fail_under = 96.5 as the single
source of truth, with the evidence-gated ratchet rule (objective 97%).

.forgejo/workflows/ci.yml — coverage job: carries the skip_coverage operator
valve; propagates nox's exit EXPLICITLY (set -uo pipefail + PIPESTATUS +
exit $rc) instead of relying on the runner's implicit bash -eo pipefail;
adds timeout-minutes: 30 so a hang fails cleanly with diagnostics; deletes
the dead threshold=50 "Surface coverage summary" step; fixes the phantom-97
step label. Gating remains nox's --fail-under (sourced from pyproject).

coverage_threshold_config_steps.py — the fail-under feature step resolves the
floor from pyproject when COVERAGE_THRESHOLD delegates to the reader.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 16:32:53 -04:00
CoreRasurae cecca72b8e test(behave): Reduce the coverage level to 96.5%
CI / benchmark-publish (push) Has started running
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Failing after 14s
CI / helm (push) Failing after 8s
CI / unit_tests (push) Failing after 21s
CI / typecheck (push) Failing after 32s
CI / build (push) Failing after 9s
CI / quality (push) Failing after 24s
CI / integration_tests (push) Failing after 15s
CI / security (push) Failing after 24s
CI / lint (push) Failing after 34s
CI / coverage (push) Has been skipped
CI / docker (push) Has been skipped
CI / push-validation (push) Successful in 20s
CI / status-check (push) Failing after 6s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Failing after 1m11s
CI / coverage (pull_request) Successful in 14m34s
CI / push-validation (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 1m14s
CI / integration_tests (pull_request) Successful in 3m42s
CI / e2e_tests (pull_request) Successful in 4m26s
CI / unit_tests (pull_request) Successful in 6m29s
CI / build (pull_request) Successful in 59s
CI / lint (pull_request) Successful in 53s
CI / quality (pull_request) Successful in 56s
CI / security (pull_request) Successful in 1m19s
CI / helm (pull_request) Successful in 53s
CI / docker (pull_request) Successful in 1m46s
CI / status-check (pull_request) Successful in 3s
2026-05-05 02:01:49 +00:00
freemo 02250473ad fix(ci): restore all CI quality gates to passing on master
Fix all failing CI quality gates (lint, unit_tests, format) without
suppressing any quality enforcement.

Root causes and fixes:

1. Format: features/steps/plan_namespaced_name_tdd_steps.py had trailing
   whitespace; fixed by running ruff format.

2. Unit tests - A2A JSON-RPC 2.0 migration (commit 9c6d6915) renamed
   A2aRequest fields (operation→method, request_id→id, a2a_version→jsonrpc)
   and A2aResponse fields (status+data→result, request_id→id) but did not
   update all step files and feature files:
   - a2a_jsonrpc_wire_format_steps.py: added use_step_matcher('re') and
     reset to 'parse' at end to prevent parallel test interference
   - a2a_facade_wiring_steps.py: updated operation= to method=, .status/.data
     to .result
   - a2a_facade_steps.py: updated request_id→id, a2a_version→jsonrpc,
     A2aResponse(request_id=..., status=...) to new API
   - m6_facade_steps.py: updated all old API usage
   - devcontainer_cleanup_steps.py: updated A2aRequest(operation=...)
   - plan_prompt_command_steps.py: updated A2aRequest(operation=...)
   - wf03_plan_prompt_confidence_steps.py: updated A2aRequest(operation=...)
   - consolidated_misc.feature: updated old A2aRequest/A2aResponse scenarios

3. Unit tests - Session CLI output changed (commit 0d5d9cf0 and others):
   - 'Session Created' → 'Session created' (lowercase)
   - 'Session Details' → 'Session Summary'
   - 'Sessions (N total)' → 'Sessions'
   - session list JSON: top-level 'total' → nested 'summary.total'
   - Fixed in: session_cli.feature, session_cli_coverage_boost.feature,
     session_cli_uncovered_branches.feature, session_list_error.feature,
     tdd_session_create_persist_steps.py

4. Unit tests - Plan list output changed (commit 1a07a891):
   - 'V3 Lifecycle Plans' → 'Plans'
   - 'Lifecycle Plans' → 'Plans'
   - Name column removed (restored in source)
   - Invariants column removed (restored in source)
   - Project truncation removed (restored in source)
   - Fixed in: plan_cli_cancel_revert_coverage.feature,
     plan_lifecycle_cli_coverage.feature, plan_cli_coverage_boost_steps.py,
     plan.py (source code restored)

5. Unit tests - Plan apply command now requires ULID (commit 300a5d6d):
   - plan_cli_coverage_r3.feature: updated 'PLAN-001' to valid ULID
   - plan_cli_coverage_r3_steps.py: added --yes flag, added new step for
     no-eligible-plans path

6. Unit tests - Various source code bugs:
   - ThoughtBlock: converted from @dataclass to Pydantic BaseModel
     (architecture test requires all dataclasses to use Pydantic)
   - session.py: added DatabaseError handling to export, import, tell commands
   - database.py: fixed rollback_to() to reuse checkpoint connection for writes
   - database.py: added _get_checkpoint_conn() helper
   - check-tls-cert.py: fixed SSLCertVerificationError.reason AttributeError

7. Unit tests - Test step bugs:
   - error_recovery_coverage_boost_steps.py: fixed invalid ULID _PLAN_ID
   - session_service_coverage_steps.py: fixed 'sha256:' prefix bug in checksum
   - database_models_new_coverage_steps.py: added 'name' field to session mock
   - async_audit_recording_steps.py: fixed Settings(audit_async=False) via env var
   - coverage_threshold_config_steps.py: added --coverage-min pattern support
   - m5_acms_smoke_steps.py: updated usage hint text
   - actor_cli_yaml_steps.py: updated 'Removed actor' → 'Actor removed'
   - aimodelscredentials_steps.py: set context.imported_class in import step
   - domain_base_model.feature: added missing 'When I examine model_config' step
   - tui_first_run_steps.py: fixed module reload to restore cleveragents.tui.*
     modules after test (prevented patch interference in subsequent tests)
   - tui_first_run_steps.py: added set_search('') step for empty string
   - resource_handler_base_coverage_r3_steps.py: use _MinimalHandler instead
     of DatabaseResourceHandler for NotImplementedError tests
   - resource_handler_crud.feature: updated to test new DatabaseHandler behavior
   - resource_handler_sandbox.feature: updated to test new DatabaseHandler behavior
   - tdd_json_decode_crash_persistence.feature: fixed @tdd_bug → @tdd_issue tags

8. Parallel test interference:
   - All step files using use_step_matcher('re') now reset to 'parse' at end
     to prevent global matcher state leaking to subsequent step files
2026-04-04 20:38:16 +00:00
brent.edwards bb3c4ffaef fix(qa): resolve merge conflicts with master and fix coverage threshold test
Merged master into feature/q0-min-coverage. Resolved conflicts in:
- benchmarks/coverage_report_bench.py: combined AST and regex benchmarks
- docs/development/testing.md: merged both versions into comprehensive guide
- robot/coverage_threshold.robot: merged test cases from both branches
- implementation_plan.md: kept both Brent and Luis notes

Fixed coverage_threshold_config test that failed because it expected
a literal --fail-under=97 but our noxfile uses the COVERAGE_THRESHOLD
constant via f-string. Updated step to resolve the constant via AST.
2026-02-13 22:25:16 +00:00
freemo 9b30421b34 feat(db): add spec-aligned action and plan tables with migrations, ORM models, and benchmarks
CI / lint (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 27s
CI / security (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 16s
CI / integration_tests (pull_request) Successful in 4m10s
CI / build (pull_request) Successful in 15s
CI / unit_tests (pull_request) Successful in 8m0s
CI / docker (pull_request) Successful in 40s
CI / coverage (pull_request) Successful in 6m22s
CI / lint (push) Successful in 13s
CI / typecheck (push) Successful in 27s
CI / security (push) Successful in 22s
CI / quality (push) Successful in 15s
CI / integration_tests (push) Successful in 4m16s
CI / build (push) Successful in 15s
CI / unit_tests (push) Successful in 8m2s
CI / docker (push) Successful in 39s
CI / coverage (push) Successful in 6m23s
2026-02-13 13:05:20 -05:00