[AUTO-INF-3] Introduce shared CLI test harness for Behave and Robot suites #8263

New issue

Open

opened 2026-04-13 06:55:07 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-13 06:55:07 +00:00

Owner

Summary

Behave CLI scenarios repeatedly instantiate CliRunner() inline (for example features/steps/plan_lifecycle_commands_coverage_steps.py lines 82-135) and patch _get_* services manually for every step.
Robot helpers such as robot/helper_actor_cli_show.py lines 40-84 contain the same pattern; grep 'CliRunner(' reports 275 occurrences across step modules.

Problem

Without a shared CLI harness, patch/cleanup logic drifts between files, increasing the risk of inconsistent CLI behaviour assertions and lingering side effects when patches are not unwound consistently.
The duplicated scaffolding slows down adding new CLI behaviours because changes must be replicated across hundreds of call sites, making the test architecture brittle.

Proposal

Add a shared CLI test harness (e.g., tests/support/cli.py) exposing helpers such as invoke_cli(app, args, services=...) that create the runner, patch service factories, and register cleanup handlers against the Behave context.
Provide Behave context/Robot keyword utilities that wrap the harness so step definitions can focus on scenario-specific setup instead of boilerplate.
Replace direct CliRunner() usage in Behave and Robot modules with the shared helpers and document the new pattern in the test contributor guide.

Acceptance Criteria

Shared harness centralises runner creation, service patch injection, and cleanup registration.
Behave and Robot CLI test modules adopt the harness; ad-hoc CliRunner() instantiations are removed outside the harness itself.
Test suites remain green with coverage ≥ 97%, and the harness usage is documented for future contributors.

Duplicate Check

Open issues (q=CliRunner): #6061 standardises benchmark CliRunner usage only; no cross-framework harness exists.
Open issues (q="CLI harness"): search returned no overlapping test-architecture work.
Cross-area (q=[AUTO-INF-): existing [AUTO-INF-*] items such as #8168 focus on CI/nox parallelism, not CLI harness reuse.
Closed issues (q=CliRunner): only CI runtime and watchdog items; nothing about shared test harnesses.
Closed issues (q="test harness"): results are status/epic updates with no actionable overlap.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

### Summary - Behave CLI scenarios repeatedly instantiate `CliRunner()` inline (for example `features/steps/plan_lifecycle_commands_coverage_steps.py` lines 82-135) and patch `_get_*` services manually for every step. - Robot helpers such as `robot/helper_actor_cli_show.py` lines 40-84 contain the same pattern; `grep 'CliRunner('` reports 275 occurrences across step modules. ### Problem - Without a shared CLI harness, patch/cleanup logic drifts between files, increasing the risk of inconsistent CLI behaviour assertions and lingering side effects when patches are not unwound consistently. - The duplicated scaffolding slows down adding new CLI behaviours because changes must be replicated across hundreds of call sites, making the test architecture brittle. ### Proposal 1. Add a shared CLI test harness (e.g., `tests/support/cli.py`) exposing helpers such as `invoke_cli(app, args, services=...)` that create the runner, patch service factories, and register cleanup handlers against the Behave context. 2. Provide Behave context/Robot keyword utilities that wrap the harness so step definitions can focus on scenario-specific setup instead of boilerplate. 3. Replace direct `CliRunner()` usage in Behave and Robot modules with the shared helpers and document the new pattern in the test contributor guide. ### Acceptance Criteria - Shared harness centralises runner creation, service patch injection, and cleanup registration. - Behave and Robot CLI test modules adopt the harness; ad-hoc `CliRunner()` instantiations are removed outside the harness itself. - Test suites remain green with coverage ≥ 97%, and the harness usage is documented for future contributors. ### Duplicate Check - Open issues (`q=CliRunner`): #6061 standardises benchmark CliRunner usage only; no cross-framework harness exists. - Open issues (`q="CLI harness"`): search returned no overlapping test-architecture work. - Cross-area (`q=[AUTO-INF-`): existing `[AUTO-INF-*]` items such as #8168 focus on CI/nox parallelism, not CLI harness reuse. - Closed issues (`q=CliRunner`): only CI runtime and watchdog items; nothing about shared test harnesses. - Closed issues (`q="test harness"`): results are status/epic updates with no actionable overlap. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

HAL9000 commented

2026-04-28 07:39:31 +00:00

Author

Owner

Re-Review Attempt — Stale Clean Group — PR Not Found

⚠️ PR #8263 does not exist as an open (or closed) pull request.

All API attempts to access this PR return HTTP 404:

GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263 → 404
GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263?state=closed → 404
GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263/merge → 404

Note: Issue #8263 does exist ("[AUTO-INF-3] Introduce shared CLI test harness for Behave and Robot suites"), but there is no pull request with this number.

The task metadata provided was:

Base SHA: ca050538fb6bea7e1a1165bf71f6e4c6da5bf959
Merge Base: ecf9710369abbff5135e627757a795d66ce94a67
Stale State: stale_no_conflicts
CI Status: failing
Approvals: 1

This PR may have been merged, closed, or the PR number in the task was incorrect. No review action was taken as the target PR cannot be found.

Automated by CleverAgents Bot
Supervisor: Stale PR Re-Review | Agent: task-implementor

**Re-Review Attempt** — Stale Clean Group — PR Not Found ⚠️ **PR #8263 does not exist as an open (or closed) pull request.** All API attempts to access this PR return HTTP 404: - `GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263` → 404 - `GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263?state=closed` → 404 - `GET /api/v1/repos/cleveragents/cleveragents-core/pulls/8263/merge` → 404 Note: Issue #8263 does exist ("[AUTO-INF-3] Introduce shared CLI test harness for Behave and Robot suites"), but there is no pull request with this number. The task metadata provided was: - Base SHA: `ca050538fb6bea7e1a1165bf71f6e4c6da5bf959` - Merge Base: `ecf9710369abbff5135e627757a795d66ce94a67` - Stale State: `stale_no_conflicts` - CI Status: `failing` - Approvals: 1 This PR may have been merged, closed, or the PR number in the task was incorrect. No review action was taken as the target PR cannot be found. --- Automated by CleverAgents Bot Supervisor: Stale PR Re-Review | Agent: task-implementor