[AUTO-INF-12] Missing Test Levels Identified #8333

New issue

Closed

opened 2026-04-13 09:04:44 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-13 09:04:44 +00:00

Owner

Summary

Identified modules with uneven coverage across Behave (unit), Robot (integration), and ASV (benchmark) suites.
Reactive routing, LangGraph orchestration, TUI UI pipeline, and platform bootstrap helpers lack at least one testing tier, reducing regression detection.
Recommend targeted Robot scenarios and ASV benchmarks to close coverage gaps.

Findings

Module	Unit Tests	Integration Tests	Benchmarks	Notes
`reactive`	✅ `features/reactive_application_coverage.feature`, `reactive_route_coverage_boost.feature`	❌ No `robot/reactive.robot` suites (confirmed via glob search)	❌ No `benchmarks/reactive` modules	Stream router and graph executor exercised only at unit level; lacks CLI-driven coverage & performance baselines.
`langgraph`	✅ `features/langgraph_nodes_coverage_boost.feature`, `plan_generation_langgraph_coverage.feature`	⚠️ Only incidental assertions in `robot/rxpy_route_validation.robot` & `robot/plan_generation_graph.robot`; no dedicated `langgraph` suite	❌ No `benchmarks/langgraph` modules	Complex state machine lacks sustained integration/performance regression coverage.
`tui`	✅ Multiple Behave specs (e.g. `tui_app_coverage.feature`, `tui_shell_exec_coverage.feature`)	✅ `robot/tui_smoke.robot`, `robot/tui_help_command.robot`	❌ No ASV benchmarks targeting TUI rendering	Interactive UI rendering performance untracked; regressions can slip in.
`platform`	✅ Covered by consolidated Behave steps (e.g. `module_coverage.feature` scenario)	❌ No Robot coverage exercising packaged CLI bootstrap	❌ No benchmarks (low risk)	`ensure_cli_importable()` only unit-tested; missing runtime validation in packaged contexts.

Recommendations

Author Robot suites that launch the CLI with reactive routing enabled (exercise reactive/application.py, route_bridge.py, and stream hand-offs) and add ASV benchmarks that measure stream_router.StreamRouter.dispatch latency.
Introduce LangGraph-focused Robot scenarios that run PlanExecutor with real checkpoint/state transitions, plus ASV benchmarks around langgraph.graph_executor.GraphExecutor throughput.
Add ASV cases for the TUI layer (pane layout/rendering and slash-command catalog hydration) to prevent regressions in interactive performance.
Add a lightweight Robot smoke test that packages the CLI (sdist/wheel) and invokes python -m cleveragents to ensure platform.ensure_cli_importable() works post-packaging; document benchmark omission given the helper’s low complexity.

Duplicate Check

Queried open issues via Forgejo REST (issues?state=open&limit=100) on 2026-04-13 and found no existing reports covering these missing test levels.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - Identified modules with uneven coverage across Behave (unit), Robot (integration), and ASV (benchmark) suites. - Reactive routing, LangGraph orchestration, TUI UI pipeline, and platform bootstrap helpers lack at least one testing tier, reducing regression detection. - Recommend targeted Robot scenarios and ASV benchmarks to close coverage gaps. ### Findings | Module | Unit Tests | Integration Tests | Benchmarks | Notes | | --- | --- | --- | --- | --- | | `reactive` | ✅ `features/reactive_application_coverage.feature`, `reactive_route_coverage_boost.feature` | ❌ No `robot/*reactive*.robot` suites (confirmed via glob search) | ❌ No `benchmarks/*reactive*` modules | Stream router and graph executor exercised only at unit level; lacks CLI-driven coverage & performance baselines. | | `langgraph` | ✅ `features/langgraph_nodes_coverage_boost.feature`, `plan_generation_langgraph_coverage.feature` | ⚠️ Only incidental assertions in `robot/rxpy_route_validation.robot` & `robot/plan_generation_graph.robot`; no dedicated `langgraph` suite | ❌ No `benchmarks/*langgraph*` modules | Complex state machine lacks sustained integration/performance regression coverage. | | `tui` | ✅ Multiple Behave specs (e.g. `tui_app_coverage.feature`, `tui_shell_exec_coverage.feature`) | ✅ `robot/tui_smoke.robot`, `robot/tui_help_command.robot` | ❌ No ASV benchmarks targeting TUI rendering | Interactive UI rendering performance untracked; regressions can slip in. | | `platform` | ✅ Covered by consolidated Behave steps (e.g. `module_coverage.feature` scenario) | ❌ No Robot coverage exercising packaged CLI bootstrap | ❌ No benchmarks (low risk) | `ensure_cli_importable()` only unit-tested; missing runtime validation in packaged contexts. | ### Recommendations 1. Author Robot suites that launch the CLI with reactive routing enabled (exercise `reactive/application.py`, `route_bridge.py`, and stream hand-offs) and add ASV benchmarks that measure `stream_router.StreamRouter.dispatch` latency. 2. Introduce LangGraph-focused Robot scenarios that run `PlanExecutor` with real checkpoint/state transitions, plus ASV benchmarks around `langgraph.graph_executor.GraphExecutor` throughput. 3. Add ASV cases for the TUI layer (pane layout/rendering and slash-command catalog hydration) to prevent regressions in interactive performance. 4. Add a lightweight Robot smoke test that packages the CLI (sdist/wheel) and invokes `python -m cleveragents` to ensure `platform.ensure_cli_importable()` works post-packaging; document benchmark omission given the helper’s low complexity. ### Duplicate Check - Queried open issues via Forgejo REST (`issues?state=open&limit=100`) on 2026-04-13 and found no existing reports covering these missing test levels. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

Rows
Columns