cleveragents/cleveragents-core

Fork 3

feat(qa): enforce coverage >=97% in CI #214

New issue

Closed

opened 2026-02-22 23:40:11 +00:00 by freemo · 1 comment

freemo commented

2026-02-22 23:40:11 +00:00

Owner

Metadata

Commit: feat(qa): enforce coverage >=97% in CI
Branch: feature/q0-min-coverage

Subtasks

Ensure nox -s coverage_report fails below 97% and emits a single-line error message suitable for CI parsing. (COVERAGE_THRESHOLD constant + session.error with CI-parseable summary)
Update CI summary output (or job annotations) to surface the 97% threshold failure line clearly. (added "Surface coverage summary" step in CI that reads build/coverage.json)
Update docs/development/testing.md with the 97% coverage requirement and a sample failure output. (created docs/development/testing.md with full testing guide)
Tests (Behave): Add a scenario that parses coverage config and asserts threshold >=97%. (features/coverage_threshold_enforcement.feature, 11 scenarios)
Tests (Robot): Add a Robot test that runs nox -s coverage_report and asserts pass/fail behavior. (robot/coverage_threshold.robot, 6 test cases)
Tests (ASV): Add benchmarks/coverage_report_bench.py for coverage report runtime baseline. (benchmarks/coverage_report_bench.py, 4 benchmarks)
Run nox (all default sessions, including benchmark). (lint 0 findings, typecheck 0 errors, 2235 unit scenarios passed, 211 integration tests passed)
Verify coverage >=97% via nox -s coverage_report. (97.5% coverage, threshold 97%)

Section: ### Section 0: Quality Automation Setup [WORKSTREAM Q - Brent Lead]
Status: Completed

## Metadata - **Commit**: `feat(qa): enforce coverage >=97% in CI` - **Branch**: `feature/q0-min-coverage` ## Subtasks - [x] Ensure `nox -s coverage_report` fails below 97% and emits a single-line error message suitable for CI parsing. (COVERAGE_THRESHOLD constant + session.error with CI-parseable summary) - [x] Update CI summary output (or job annotations) to surface the 97% threshold failure line clearly. (added "Surface coverage summary" step in CI that reads build/coverage.json) - [x] Update `docs/development/testing.md` with the 97% coverage requirement and a sample failure output. (created docs/development/testing.md with full testing guide) - [x] Tests (Behave): Add a scenario that parses coverage config and asserts threshold >=97%. (features/coverage_threshold_enforcement.feature, 11 scenarios) - [x] Tests (Robot): Add a Robot test that runs `nox -s coverage_report` and asserts pass/fail behavior. (robot/coverage_threshold.robot, 6 test cases) - [x] Tests (ASV): Add `benchmarks/coverage_report_bench.py` for coverage report runtime baseline. (benchmarks/coverage_report_bench.py, 4 benchmarks) - [x] Run `nox` (all default sessions, including benchmark). (lint 0 findings, typecheck 0 errors, 2235 unit scenarios passed, 211 integration tests passed) - [x] Verify coverage >=97% via `nox -s coverage_report`. (97.5% coverage, threshold 97%) **Section**: ### Section 0: Quality Automation Setup [WORKSTREAM Q - Brent Lead] **Status**: Completed

freemo added this to the (deleted) milestone

2026-02-22 23:40:11 +00:00

freemo added the

labels

2026-02-22 23:40:11 +00:00

brent.edwards was assigned by freemo

2026-02-22 23:40:11 +00:00

freemo closed this issue

2026-02-22 23:40:11 +00:00

freemo added a new dependency

2026-02-22 23:54:07 +00:00

#349 Epic: Quality Automation Setup (Q0)

freemo modified the milestone from (deleted) to v3.0.0

2026-02-23 00:07:09 +00:00

freemo added a new dependency

2026-02-23 01:11:11 +00:00

#50 feat(qa): enforce coverage >=97% in CI

freemo referenced this issue from a pull request that will close it,

2026-02-23 17:25:57 +00:00

feat(qa): enforce coverage >=97% in CI #50

freemo commented

2026-02-24 02:29:19 +00:00

Author

Owner

Implementation Notes — Coverage Threshold Enforcement

2026-02-13: Task Q0-min-coverage In Progress - Coverage Threshold Enforcement

Enhanced coverage_report nox session: COVERAGE_THRESHOLD = 97 constant, generates HTML/XML/JSON reports before checking threshold, emits CI-parseable summary line.
Created docs/development/testing.md (180+ lines): full testing guide covering unit (Behave), integration (Robot), benchmarks (ASV), and coverage.
Created features/coverage_threshold_enforcement.feature (11 scenarios).
Verification: 2235 unit scenarios passed, 211 integration tests passed, 97.5% coverage.

(Migrated from docs/implementation-notes.md)

## Implementation Notes — Coverage Threshold Enforcement **2026-02-13**: Task Q0-min-coverage In Progress - Coverage Threshold Enforcement - Enhanced `coverage_report` nox session: `COVERAGE_THRESHOLD = 97` constant, generates HTML/XML/JSON reports before checking threshold, emits CI-parseable summary line. - Created `docs/development/testing.md` (180+ lines): full testing guide covering unit (Behave), integration (Robot), benchmarks (ASV), and coverage. - Created `features/coverage_threshold_enforcement.feature` (11 scenarios). - Verification: 2235 unit scenarios passed, 211 integration tests passed, 97.5% coverage. *(Migrated from `docs/implementation-notes.md`)*

freemo referenced this issue

2026-02-24 03:27:03 +00:00

Create Unit Test Suite for Core Bug Fixes #13

freemo referenced this issue

2026-02-24 03:27:06 +00:00

Add comprehensive unit tests for core application logic and all agent types #17

freemo removed a dependency

2026-03-03 22:54:42 +00:00

#50 feat(qa): enforce coverage >=97% in CI

freemo added a new dependency

2026-03-03 22:54:42 +00:00

#50 feat(qa): enforce coverage >=97% in CI