cleveragents/cleveragents-core

Fork 3

UAT: `nox -e coverage_report` fails — test coverage is 37%, far below the required 97% threshold #1782

New issue

Open

opened 2026-04-02 23:49:00 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:49:00 +00:00

Owner

Metadata

Branch: fix/uat-coverage-report-37pct-below-threshold
Commit Message: fix(coverage): resolve coverage_report failure caused by AmbiguousStep crash dropping coverage to 37%
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The nox -e coverage_report quality gate fails because test coverage is only 37%, which is massively below the required 97% threshold enforced by the spec and CONTRIBUTING.md.

This was discovered during UAT testing by running nox -e coverage_report in a clean clone of the repository.

Current Behavior

The session fails with coverage at 37%:

(summary)   84260   53054   37

Total lines: 84,260
Covered lines: 31,206
Missing lines: 53,054
Coverage: 37.0% (threshold: 97%)

The nox -e coverage_report session exits with code 1 (COVERAGE FAILED).

Root cause analysis:
The coverage failure is directly caused by the nox -e unit_tests crash (AmbiguousStep error in tui_thought_block_steps.py vs tui_first_run_steps.py). Because the Behave test suite crashes immediately during step loading, only a fraction of the codebase is exercised before the crash. The 37% coverage reflects code that is imported/loaded but not actually tested.

Expected Behavior

nox -e coverage_report must pass with coverage >= 97%. The spec states: "Unit test coverage must be maintained at or above 97% at all times."

Steps to Reproduce

Clone the repository
Run nox -e coverage_report
Observe COVERAGE FAILED: 37.0% < 97% threshold

Impact

nox -e coverage_report is a mandatory quality gate that is completely broken
The 60-percentage-point gap (37% vs 97%) indicates the vast majority of the codebase has no test coverage
This is a Critical blocking issue for any PR merge or release

The AmbiguousStep crash (tui_thought_block_steps.py vs tui_first_run_steps.py) is the immediate cause
Issue #1609 tracks the broader ambiguous step fix

Subtasks

Confirm root cause: verify that fixing the AmbiguousStep crash in tui_thought_block_steps.py / tui_first_run_steps.py (see #1609) restores the full Behave test suite execution
Run nox -e unit_tests after the AmbiguousStep fix and confirm the suite completes without crashing
Run nox -e coverage_report and confirm coverage rises to >= 97%
If coverage remains below 97% after the AmbiguousStep fix, identify and add missing test scenarios for uncovered code paths
Ensure all new/updated step definitions follow BDD Test Organization Guidelines (no duplicate step files, feature-specific naming)
Run full nox (all default sessions) and fix any errors
Verify nox -e coverage_report passes with coverage >= 97%

Definition of Done

All subtasks above are completed and checked off
nox -e unit_tests completes without crashing (no AmbiguousStep errors)
nox -e coverage_report passes with coverage >= 97%
All nox stages pass
Coverage >= 97%
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/uat-coverage-report-37pct-below-threshold` - **Commit Message**: `fix(coverage): resolve coverage_report failure caused by AmbiguousStep crash dropping coverage to 37%` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `nox -e coverage_report` quality gate fails because test coverage is only **37%**, which is massively below the required **97%** threshold enforced by the spec and CONTRIBUTING.md. This was discovered during UAT testing by running `nox -e coverage_report` in a clean clone of the repository. ## Current Behavior The session fails with coverage at 37%: ``` (summary) 84260 53054 37 ``` - Total lines: 84,260 - Covered lines: 31,206 - Missing lines: 53,054 - Coverage: **37.0%** (threshold: 97%) The `nox -e coverage_report` session exits with code 1 (COVERAGE FAILED). **Root cause analysis:** The coverage failure is directly caused by the `nox -e unit_tests` crash (AmbiguousStep error in `tui_thought_block_steps.py` vs `tui_first_run_steps.py`). Because the Behave test suite crashes immediately during step loading, only a fraction of the codebase is exercised before the crash. The 37% coverage reflects code that is imported/loaded but not actually tested. ## Expected Behavior `nox -e coverage_report` must pass with coverage >= 97%. The spec states: "Unit test coverage must be maintained at or above 97% at all times." ## Steps to Reproduce 1. Clone the repository 2. Run `nox -e coverage_report` 3. Observe COVERAGE FAILED: 37.0% < 97% threshold ## Impact - `nox -e coverage_report` is a mandatory quality gate that is completely broken - The 60-percentage-point gap (37% vs 97%) indicates the vast majority of the codebase has no test coverage - This is a Critical blocking issue for any PR merge or release ## Related Issues - The AmbiguousStep crash (`tui_thought_block_steps.py` vs `tui_first_run_steps.py`) is the immediate cause - Issue #1609 tracks the broader ambiguous step fix ## Subtasks - [ ] Confirm root cause: verify that fixing the AmbiguousStep crash in `tui_thought_block_steps.py` / `tui_first_run_steps.py` (see #1609) restores the full Behave test suite execution - [ ] Run `nox -e unit_tests` after the AmbiguousStep fix and confirm the suite completes without crashing - [ ] Run `nox -e coverage_report` and confirm coverage rises to >= 97% - [ ] If coverage remains below 97% after the AmbiguousStep fix, identify and add missing test scenarios for uncovered code paths - [ ] Ensure all new/updated step definitions follow BDD Test Organization Guidelines (no duplicate step files, feature-specific naming) - [ ] Run full `nox` (all default sessions) and fix any errors - [ ] Verify `nox -e coverage_report` passes with coverage >= 97% ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] `nox -e unit_tests` completes without crashing (no AmbiguousStep errors) - [ ] `nox -e coverage_report` passes with coverage >= 97% - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.8.0 milestone

2026-04-02 23:49:16 +00:00

freemo added a new dependency

2026-04-02 23:52:18 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo referenced this issue

2026-04-02 23:53:28 +00:00

TEST-INFRA: [coverage-gaps] Add test coverage for src/cleveragents/a2a/asgi.py #1807

freemo referenced this issue from a commit

2026-04-02 23:57:42 +00:00

docs(timeline): update schedule adherence Day 53 (2026-04-02) night refresh

freemo referenced this issue

2026-04-02 23:58:08 +00:00

[Automated] Product Build Session State #1576

freemo referenced this issue

2026-04-03 00:05:55 +00:00

TEST-INFRA: [coverage-gaps] Add test coverage for src/cleveragents/a2a/cli_bootstrap.py #1884

freemo added the

MoSCoW

Must have

label

2026-04-05 09:21:18 +00:00

freemo commented

2026-04-05 09:23:56 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Critical
MoSCoW: Must Have — test coverage at 37% is catastrophically below the 97% requirement. This is a project-wide quality gate failure that must be addressed immediately. Every milestone's acceptance criteria include "Test coverage >= 97%"

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Critical - **MoSCoW**: Must Have — test coverage at 37% is catastrophically below the 97% requirement. This is a project-wide quality gate failure that must be addressed immediately. Every milestone's acceptance criteria include "Test coverage >= 97%" --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner