[AUTO-INF-2] Close coverage gaps in CLI envelopes, A2A events, and session DI #9893

New issue

Open

opened 2026-04-15 23:42:26 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-15 23:42:26 +00:00

Owner

Summary

nox -s coverage_report currently replays all 14,406 Behave scenarios sequentially; the first run hit the 30 minute tool timeout (tool_d9356efef001iz7LMo2QCusfPX shows bash tool terminated command after exceeding timeout 1800000 ms) and the follow-up single-process retry is still in progress. Coverage data never lands, so we cannot track regressions.
analysis_results.txt flags 11 "Category B" scenarios that still lack green coverage. The failing stories map to the CLI formatting/envelope helpers, A2A event queue/bridge, and the session DI wiring path.
The uncovered areas are concentrated in:
- src/cleveragents/cli/formatting.py (features/cli_output_formats.feature, features/cli_json_envelope.feature) — YAML envelope paths and direct format_output helpers remain @tdd_expected_fail (@tdd_issue_4364, @tdd_issue_4369, @tdd_issue_4370). JSON-only coverage is tracked in #9854, but the YAML/table/plain branches and _build_envelope validation still lack tests.
- src/cleveragents/a2a/events.py (features/tdd_event_bus_exception_swallow.feature) — subscriber error logging, queue closure guards, and remote subscribe edge cases are uncovered; slipcover output shows exceptions from A2aEventQueue.publish during coverage runs without assertions.
- src/cleveragents/cli/commands/session.py and its DI providers (features/tdd_session_create_di.feature) — the CLI path depends on container.session_service(), yet there is no green test proving the container supplies a working DB provider or that JSON output stays envelope compliant.

Details

Coverage execution

nox -s coverage_report waits on lint/typecheck/security/quality/unit/integration, then reruns every feature sequentially under slipcover. With 624 feature files and 14,406 scenarios this step alone consumes 30–45 minutes and currently times out before writing build/coverage.json/coverage.xml (find build -type f shows nothing after timeout).
Re-running with --processes=1 (tool call toolu_014zBdyt85mFtzzdKB6oLB8a) still streams the entire behave suite; without gating or scenario filtering the job remains a bottleneck and hides real coverage deltas.

CLI envelope and formatting gaps

features/cli_output_formats.feature → @tdd_issue_4364 scenario fails because _serialize_value/_format_plain/_format_table output is not exercised by any passing test. The module now delegates to materialisers (cleveragents.cli.output.materializers) but only JSON happy-path coverage exists (#9854).
features/cli_json_envelope.feature → @tdd_issue_4369 & @tdd_issue_4370 show YAML envelopes never assert presence of command/status/exit_code/data/timing/messages. _build_envelope enforces validation yet there is no unit coverage for bad statuses or negative exit codes.

A2A event queue and bridge gaps

features/tdd_event_bus_exception_swallow.feature remains @tdd_expected_fail: the queue should log and continue when subscriber callbacks raise, validate subscription IDs, and enforce remote endpoint checks. None of these branches are covered by unit tests in src/cleveragents/a2a/events.py.
Coverage logs from the first slipcover run show a2a.event.callback_error stack traces without assertions, confirming this code path runs only during manual coverage attempts.

Session CLI DI path gaps

features/tdd_session_create_di.feature documents the missing db provider in the DI container; the CLI command currently relies on _get_session_service() pulling from get_container(), but there is no integration test proving the container wiring works end-to-end or that structured JSON output stays spec compliant.
The CLI’s facade notification (_facade_dispatch) is also unverified, so regressions in A2A facade routing go unnoticed.

Recommendations

Collapse the slipcover run into the parallel behave session (or shard by feature tag) so coverage artifacts are produced. Until then, reduce coverage_report to reuse the behaviour results instead of replaying the suite sequentially.
Add targeted unit tests for _build_envelope, _format_plain/_format_table, and _serialize_value covering YAML/plain/table output, invalid status/exit codes, and envelope validation. Pair these with Behave scenarios that flip @tdd_issue_4364/@tdd_issue_4369/@tdd_issue_4370 to green. Coordinate with #9854, which focuses on the JSON envelope guards.
Introduce unit tests for A2aEventQueue and EventBusBridge that explicitly exercise subscriber exceptions, remote subscribe validation, queue closure, and event type translation. Mark the existing @tdd_event_bus_exception_swallow scenarios as expected-pass once the queue emits/asserts the correct log entries.
Add an integration test for agents session create that boots the real DI container (or a fixture container) to ensure the session_service provider exists, returns structured envelope output for --format json/yaml, and notifies the facade without raising. This should convert @tdd_issue_570 into a regression test instead of a placeholder.
Track the coverage artifacts in CI to confirm the new tests raise overall coverage back above the 97% threshold before re-enabling the job gate.

Duplicate Check

Query	Result
`format_output`	Found #9854 "Add BDD coverage for format_output envelope validation guards" (JSON-only); current gaps cover YAML/plain/table branches and `_build_envelope` validation.
`session create`	No open issues mention DI coverage for the session CLI (`GET /issues?state=open&limit=50`).
`a2a event`	No open issues reference A2A event queue/bridge coverage.
`yaml envelope`	No matches found in open issues.
`coverage gap`	Only infrastructure/CI status tickets (#9883, #9886, etc.) mention coverage in passing; no task covers the CLI/A2A/session test gaps described here.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

## Summary - `nox -s coverage_report` currently replays all 14,406 Behave scenarios sequentially; the first run hit the 30 minute tool timeout (`tool_d9356efef001iz7LMo2QCusfPX` shows `bash tool terminated command after exceeding timeout 1800000 ms`) and the follow-up single-process retry is still in progress. Coverage data never lands, so we cannot track regressions. - `analysis_results.txt` flags 11 "Category B" scenarios that still lack green coverage. The failing stories map to the CLI formatting/envelope helpers, A2A event queue/bridge, and the session DI wiring path. - The uncovered areas are concentrated in: * `src/cleveragents/cli/formatting.py` (`features/cli_output_formats.feature`, `features/cli_json_envelope.feature`) — YAML envelope paths and direct `format_output` helpers remain @tdd_expected_fail (`@tdd_issue_4364`, `@tdd_issue_4369`, `@tdd_issue_4370`). JSON-only coverage is tracked in #9854, but the YAML/table/plain branches and `_build_envelope` validation still lack tests. * `src/cleveragents/a2a/events.py` (`features/tdd_event_bus_exception_swallow.feature`) — subscriber error logging, queue closure guards, and remote subscribe edge cases are uncovered; slipcover output shows exceptions from `A2aEventQueue.publish` during coverage runs without assertions. * `src/cleveragents/cli/commands/session.py` and its DI providers (`features/tdd_session_create_di.feature`) — the CLI path depends on `container.session_service()`, yet there is no green test proving the container supplies a working DB provider or that JSON output stays envelope compliant. ## Details ### Coverage execution - `nox -s coverage_report` waits on lint/typecheck/security/quality/unit/integration, then reruns every feature sequentially under slipcover. With 624 feature files and 14,406 scenarios this step alone consumes 30–45 minutes and currently times out before writing `build/coverage.json`/`coverage.xml` (`find build -type f` shows nothing after timeout). - Re-running with `--processes=1` (tool call `toolu_014zBdyt85mFtzzdKB6oLB8a`) still streams the entire behave suite; without gating or scenario filtering the job remains a bottleneck and hides real coverage deltas. ### CLI envelope and formatting gaps - `features/cli_output_formats.feature` → `@tdd_issue_4364` scenario fails because `_serialize_value`/`_format_plain`/`_format_table` output is not exercised by any passing test. The module now delegates to materialisers (`cleveragents.cli.output.materializers`) but only JSON happy-path coverage exists (#9854). - `features/cli_json_envelope.feature` → `@tdd_issue_4369` & `@tdd_issue_4370` show YAML envelopes never assert presence of `command/status/exit_code/data/timing/messages`. `_build_envelope` enforces validation yet there is no unit coverage for bad statuses or negative exit codes. ### A2A event queue and bridge gaps - `features/tdd_event_bus_exception_swallow.feature` remains @tdd_expected_fail: the queue should log and continue when subscriber callbacks raise, validate subscription IDs, and enforce remote endpoint checks. None of these branches are covered by unit tests in `src/cleveragents/a2a/events.py`. - Coverage logs from the first slipcover run show `a2a.event.callback_error` stack traces without assertions, confirming this code path runs only during manual coverage attempts. ### Session CLI DI path gaps - `features/tdd_session_create_di.feature` documents the missing `db` provider in the DI container; the CLI command currently relies on `_get_session_service()` pulling from `get_container()`, but there is no integration test proving the container wiring works end-to-end or that structured JSON output stays spec compliant. - The CLI’s facade notification (`_facade_dispatch`) is also unverified, so regressions in A2A facade routing go unnoticed. ## Recommendations 1. Collapse the slipcover run into the parallel behave session (or shard by feature tag) so coverage artifacts are produced. Until then, reduce `coverage_report` to reuse the behaviour results instead of replaying the suite sequentially. 2. Add targeted unit tests for `_build_envelope`, `_format_plain/_format_table`, and `_serialize_value` covering YAML/plain/table output, invalid status/exit codes, and envelope validation. Pair these with Behave scenarios that flip `@tdd_issue_4364/@tdd_issue_4369/@tdd_issue_4370` to green. Coordinate with #9854, which focuses on the JSON envelope guards. 3. Introduce unit tests for `A2aEventQueue` and `EventBusBridge` that explicitly exercise subscriber exceptions, remote subscribe validation, queue closure, and event type translation. Mark the existing `@tdd_event_bus_exception_swallow` scenarios as expected-pass once the queue emits/asserts the correct log entries. 4. Add an integration test for `agents session create` that boots the real DI container (or a fixture container) to ensure the `session_service` provider exists, returns structured envelope output for `--format json/yaml`, and notifies the facade without raising. This should convert `@tdd_issue_570` into a regression test instead of a placeholder. 5. Track the coverage artifacts in CI to confirm the new tests raise overall coverage back above the 97% threshold before re-enabling the job gate. ### Duplicate Check | Query | Result | | --- | --- | | `format_output` | Found #9854 "Add BDD coverage for format_output envelope validation guards" (JSON-only); current gaps cover YAML/plain/table branches and `_build_envelope` validation. | | `session create` | No open issues mention DI coverage for the session CLI (`GET /issues?state=open&limit=50`). | | `a2a event` | No open issues reference A2A event queue/bridge coverage. | | `yaml envelope` | No matches found in open issues. | | `coverage gap` | Only infrastructure/CI status tickets (#9883, #9886, etc.) mention coverage in passing; no task covers the CLI/A2A/session test gaps described here. | --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

Rows
Columns