Files
cleveragents-core/features/steps/cli_lifecycle_coverage_steps.py
T
hurui200320 414abb1396
CI / benchmark-publish (push) Waiting to run
CI / lint (push) Successful in 19s
CI / build (push) Successful in 20s
CI / quality (push) Successful in 3m49s
CI / typecheck (push) Successful in 3m58s
CI / benchmark-regression (push) Waiting to run
CI / security (push) Successful in 4m7s
CI / integration_tests (push) Successful in 6m48s
CI / unit_tests (push) Successful in 7m30s
CI / docker (push) Successful in 1m12s
CI / e2e_tests (push) Successful in 10m1s
CI / coverage (push) Successful in 11m48s
CI / status-check (push) Successful in 1s
fix(cli): add missing --yes flag to plan apply command (#1127)
## Summary

Adds the `--yes`/`-y` flag to the `lifecycle-apply` CLI command as required by the specification (`agents plan apply [--yes|-y] <PLAN_ID>`). Without `--yes`, a confirmation prompt now displays before proceeding with the destructive Apply phase. With `--yes`, the apply proceeds immediately without prompting.

Closes #932

## Changes

### Source Code
- **`src/cleveragents/cli/commands/plan.py`**: Added `yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip confirmation prompt")] = False` parameter to `lifecycle_apply_plan`. Added `typer.confirm()` prompt before the apply operation, consistent with the pattern used by `rollback_plan`, `correct_plan`, and other destructive commands.
  - Confirmation prompt text matches spec exactly: `"Apply changes for plan {plan_id}?"` producing `Apply changes for plan <ID>? [y/N]:`.
  - Fixed redundant plan ID display when `pre_plan` is `None` — now shows `"Apply changes for plan X?"` instead of `"Apply plan X (X)?"`.
  - Added `except ValueError` handler consistent with sibling commands `lifecycle_execute_plan` and `_lifecycle_apply_with_id`.
  - Added `except Exception` catch-all handler with `isinstance(e, (typer.Abort, typer.Exit))` re-raise guard, consistent with `lifecycle_execute_plan`.
  - Moved `PlanPhase` and `ProcessingState` imports to module level per CONTRIBUTING.md §Import Guidelines.

### TDD Tag Removal (Bug Fix Workflow)
- **`features/tdd_plan_apply_yes_flag.feature`**: Removed `@tdd_expected_fail` tag (leaving `@tdd_bug` and `@tdd_bug_932` as permanent regression guards).
- **`robot/tdd_plan_apply_yes_flag.robot`**: Removed `tdd_expected_fail` tag (leaving `tdd_bug` and `tdd_bug_932`).

### Test Updates
Updated all existing `lifecycle-apply` invocations across 17 test/benchmark files to pass `--yes`, since the new confirmation prompt would otherwise abort in non-interactive test environments:
- 9 Behave step definition files
- 3 Robot Framework helper scripts
- 2 Robot Framework e2e acceptance tests
- 3 ASV benchmark files (4 invocations: `cli_robot_flow_bench.py` ×2, `m1_sourcecode_smoke_bench.py` ×1, `plan_cli_smoke_bench.py` ×1)

### Confirmation Prompt Tests (New + Strengthened)
- **`features/tdd_plan_apply_yes_flag.feature`**: 5 scenarios total:
  - `lifecycle-apply recognises the --yes long flag` — verifies flag acceptance, prompt suppression, exit code 0, and `apply_plan` was called
  - `lifecycle-apply recognises the -y short flag` — same as above for short flag
  - `lifecycle-apply without --yes prompts for confirmation and user declines` — verifies `"Apply cancelled."` message, `exit_code == 0`, and `apply_plan` was NOT called
  - `lifecycle-apply without --yes prompts for confirmation and user accepts` — verifies prompt appears, `exit_code == 0`, and `apply_plan` was called
  - `lifecycle-apply catches unexpected exceptions cleanly` — verifies `"Unexpected error"` output, no traceback leak, non-zero exit code (exercises the `except Exception` catch-all)
- **`features/steps/tdd_plan_apply_yes_flag_steps.py`**: Refactored step definitions:
  - `_make_mock_plan` uses `PlanPhase` and `ProcessingState` enum types instead of raw strings
  - `_make_mock_plan` uses `datetime.now(tz=UTC)` instead of timezone-naive `datetime.now()`
  - Unified prompt suppression step handles both `--yes` and `-y` via parameterised step pattern
  - Added `When` step for unexpected error scenario with `RuntimeError` side_effect
  - Added `Then` step for non-zero exit code assertion
- **Feature/Robot documentation**: Updated stale descriptions that said "implementation does not accept --yes" to reflect the flag is now implemented.

### Documentation
- **`docs/reference/plan_cli.md`**: Updated `lifecycle-apply` section with:
  - `### Synopsis` heading with code block
  - `### Options` table listing `--yes/-y` and `--format/-f` flags
  - `### Arguments` table listing `PLAN_ID`
  - Matches the style used by other command sections in the same file

## Review Fixes (Cycle 3 — Luis's review)

| ID | Severity | Issue | Resolution |
|----|----------|-------|------------|
| M1 | Medium | `typer.Abort()` on user decline produces exit code 1 and redundant "Aborted." | Changed to `raise typer.Exit(0)` — consistent with `correct_decision` and legacy `apply` |
| M2 | Medium | Missing exit code assertion on decline scenario | Added `And the lifecycle-apply exit code should be 0` to the decline scenario |
| M3 | Medium | Spec compliance: "summary of pending changes" not implemented | Deferred — spec example shows summary *after* confirmation, not before; implementation matches spec. Ticket-vs-spec ambiguity noted. |
| L1 | Low | Missing `except Exception` catch-all handler | Added catch-all matching `lifecycle_execute_plan` pattern; re-raises `typer.Abort`/`typer.Exit` |
| L2 | Low | Documentation description not updated | Expanded description in `plan_cli.md` to explain confirmation prompt and `--yes` |
| L3 | Low | Dead code `is not None` guards | Removed both guards — `get_plan()` raises `NotFoundError`, never returns `None` |
| I1 | Info | Duplicate `PlanPhase` import | Hoisted import to top of `try` block, eliminating duplicate at old line 2087 |
| I2 | Info | `typer.confirm` without explicit `default=False` | Added `default=False` for consistency with sibling commands |
| L4 | Low | No test for `--yes` after positional arg | Not addressed — Typer/Click handles both orderings; low risk |
| L5 | Low | No test for auto-select + interactive prompt | Not addressed — separate concern outside ticket scope |
| I3 | Info | Robot helper only tests flag recognition | By design — noted as informational |

## Review Fixes (Cycle 4 — Self-QA)

| ID | Severity | Issue | Resolution |
|----|----------|-------|------------|
| Major-1 | Major | No test for `except Exception` catch-all handler | Added new scenario `"lifecycle-apply catches unexpected exceptions cleanly"` with `RuntimeError` side_effect; asserts `"Unexpected error"` output, no traceback, non-zero exit |
| Minor-2 | Minor | Missing `ValueError` handler inconsistent with siblings | Added `except ValueError as e:` with `"[red]Execution Error:[/red]"` before catch-all, matching `lifecycle_execute_plan` and `_lifecycle_apply_with_id` |
| Minor-3 | Minor | Flag scenarios don't verify `apply_plan` called | Added `And the lifecycle-apply should have called apply` to both `--yes` and `-y` scenarios |
| Minor-4 | Minor | Stale docstring in Robot helper references `tdd_expected_fail` inversion | Updated to reflect bug is fixed and tests serve as regression guards |
| Minor-5 | Minor | `plan_cli.md` lacks Options table for `lifecycle-apply` | Added Synopsis, Options, and Arguments sections matching sibling command style |
| Nit-6 | Nit | Duplicated step defs for `--yes` vs `-y` prompt suppression | Unified into single parameterised step `"the lifecycle-apply {flag} output should not contain the confirmation prompt"` |
| Nit-7 | Nit | `datetime.now()` timezone-naive | Changed to `datetime.now(tz=UTC)` |
| Nit-8 | Nit | `_make_mock_plan` params use `str` instead of enum types | Changed to `PlanPhase` and `ProcessingState` enum types |

## Review Fixes (Cycle 5 — Jeff's approval note)

| ID | Severity | Issue | Resolution |
|----|----------|-------|------------|
| Import-1 | Minor | `PlanPhase`/`ProcessingState` imports inside function body instead of module level | Moved to module-level import per CONTRIBUTING.md §Import Guidelines |

## Known Limitations / Deferred Items

- **M3: Ticket AC mentions "summary of pending changes"** but the spec example only shows `"Apply changes for plan <ID>? [y/N]: y"` without a change summary. The implementation shows plan ID only (matching the spec), not a change summary. This is a ticket-vs-spec ambiguity; recommend discussing with ticket author.
- **Legacy `apply` command** accepts `--yes` but does not pass it to `_lifecycle_apply_with_id()`. This is a pre-existing issue outside the scope of this ticket.
- **`pre_plan is None` branch** has no explicit test. Pre-existing architectural issue; no action taken.

## Quality Gates

| Gate | Result |
|------|--------|
| `nox -s lint` |  passed |
| `nox -s typecheck` |  passed (0 errors) |
| `nox -s unit_tests` |  passed (471 features, 12,424 scenarios, 0 failures) |
| `nox -s integration_tests` |  passed (1,727 tests, 0 failures) |
| `nox -s e2e_tests` |  passed (41 tests, 0 failures) |
| `nox -s coverage_report` |  passed (≥97% coverage) |

Reviewed-on: #1127
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
2026-03-26 07:50:09 +00:00

1099 lines
39 KiB
Python

"""Step definitions for CLI lifecycle command coverage feature.
All step names are prefixed with ``lifecycle coverage`` to avoid
collisions with existing steps (Behave loads all steps globally).
"""
from __future__ import annotations
import contextlib
import os
import tempfile
from datetime import datetime
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.application.services.plan_lifecycle_service import (
ActionNotAvailableError,
InvalidPhaseTransitionError,
PlanNotReadyError,
)
from cleveragents.cli.commands.action import app as action_app
from cleveragents.cli.commands.plan import app as plan_app
from cleveragents.core.exceptions import (
CleverAgentsError,
NotFoundError,
PlanError,
ValidationError,
)
from cleveragents.domain.models.core.action import (
Action,
ActionState,
)
from cleveragents.domain.models.core.plan import (
AutomationProfileProvenance,
AutomationProfileRef,
NamespacedName,
Plan,
PlanIdentity,
PlanPhase,
PlanTimestamps,
ProcessingState,
ProjectLink,
)
_PLAN_ULID = "01KHDE6WWS2171PWW3GJEBXZ8S"
_VALID_YAML = """\
name: local/lc-action
description: Lifecycle coverage action
strategy_actor: openai/gpt-4
execution_actor: openai/gpt-4
definition_of_done: All lifecycle tests pass
"""
_INVALID_SCHEMA_YAML = """\
name: 123-bad
"""
_INVALID_VALUE_YAML = """\
description: missing name
strategy_actor: openai/gpt-4
execution_actor: openai/gpt-4
definition_of_done: passes
"""
def _register_cleanup(context: Context, func) -> None:
"""Register cleanup callback with Behave scenario context."""
if hasattr(context, "add_cleanup"):
context.add_cleanup(func)
return
if not hasattr(context, "_cleanup_handlers"):
context._cleanup_handlers = []
context._cleanup_handlers.append(func)
def _safe_stop_patcher(patcher: object) -> None:
"""Best-effort patcher stop used in scenario cleanup."""
stop = getattr(patcher, "stop", None)
if not callable(stop):
return
with contextlib.suppress(RuntimeError):
stop()
def _make_lc_action(
name: str = "local/lc-action",
state: ActionState = ActionState.AVAILABLE,
) -> Action:
"""Create a mock Action for lifecycle coverage tests."""
return Action(
namespaced_name=NamespacedName.parse(name),
description="Lifecycle coverage action",
long_description="A lifecycle coverage action",
definition_of_done="All lifecycle tests pass",
strategy_actor="openai/gpt-4",
execution_actor="openai/gpt-4",
reusable=True,
read_only=False,
state=state,
created_at=datetime.now(),
updated_at=datetime.now(),
created_by=None,
)
def _make_lc_plan(
name: str = "local/lc-plan",
phase: PlanPhase = PlanPhase.STRATEGIZE,
state: ProcessingState = ProcessingState.QUEUED,
project_links: list[ProjectLink] | None = None,
error_message: str | None = None,
automation_profile: AutomationProfileRef | None = None,
strategy_actor: str | None = "openai/gpt-4",
execution_actor: str | None = "openai/gpt-4",
plan_id: str = _PLAN_ULID,
) -> Plan:
"""Create a mock Plan for lifecycle coverage tests."""
now = datetime.now()
return Plan(
identity=PlanIdentity(plan_id=plan_id),
namespaced_name=NamespacedName.parse(name),
description="Lifecycle coverage plan",
definition_of_done="Tests pass",
action_name="local/lc-action",
phase=phase,
processing_state=state,
project_links=project_links or [],
arguments={"target_coverage": 80},
arguments_order=["target_coverage"],
automation_profile=automation_profile,
strategy_actor=strategy_actor,
execution_actor=execution_actor,
reusable=True,
read_only=False,
created_by=None,
timestamps=PlanTimestamps(created_at=now, updated_at=now),
error_message=error_message,
)
def _write_temp_yaml(context: Context, content: str) -> str:
"""Write YAML content to a temporary file."""
fd, path = tempfile.mkstemp(suffix=".yaml")
with os.fdopen(fd, "w") as fh:
fh.write(content)
_register_cleanup(
context,
lambda p=path: os.unlink(p) if os.path.exists(p) else None,
)
return path
# ---------------------------------------------------------------------------
# Background
# ---------------------------------------------------------------------------
@given("a lifecycle coverage CLI runner")
def step_lc_cli_runner(context: Context) -> None:
"""Set up the CLI runner for lifecycle coverage tests."""
context.lc_runner = CliRunner()
@given("a lifecycle coverage mocked lifecycle service")
def step_lc_mocked_service(context: Context) -> None:
"""Set up a mock PlanLifecycleService for both action and plan apps."""
context.lc_mock = MagicMock()
context.lc_action_patcher = patch(
"cleveragents.cli.commands.action._get_lifecycle_service",
return_value=context.lc_mock,
)
context.lc_plan_patcher = patch(
"cleveragents.cli.commands.plan._get_lifecycle_service",
return_value=context.lc_mock,
)
context.lc_executor_patcher = patch(
"cleveragents.cli.commands.plan._get_plan_executor",
return_value=MagicMock(),
)
context.lc_action_patcher.start()
context.lc_plan_patcher.start()
context.lc_executor_patcher.start()
_register_cleanup(context, lambda: _safe_stop_patcher(context.lc_action_patcher))
_register_cleanup(context, lambda: _safe_stop_patcher(context.lc_plan_patcher))
_register_cleanup(context, lambda: _safe_stop_patcher(context.lc_executor_patcher))
# ---------------------------------------------------------------------------
# Action create
# ---------------------------------------------------------------------------
@given("a lifecycle coverage valid action config file")
def step_lc_valid_config(context: Context) -> None:
context.lc_config_path = _write_temp_yaml(context, _VALID_YAML)
@given("a lifecycle coverage invalid schema config file")
def step_lc_invalid_schema_config(context: Context) -> None:
context.lc_bad_schema_path = _write_temp_yaml(context, _INVALID_SCHEMA_YAML)
@given("a lifecycle coverage invalid value config file")
def step_lc_invalid_value_config(context: Context) -> None:
context.lc_bad_value_path = _write_temp_yaml(context, _INVALID_VALUE_YAML)
@given("a lifecycle coverage empty config file")
def step_lc_empty_config(context: Context) -> None:
context.lc_empty_config_path = _write_temp_yaml(context, "")
@when("I run lifecycle coverage action create with config")
def step_lc_action_create(context: Context) -> None:
created = _make_lc_action()
context.lc_mock.create_action.return_value = created
context.lc_result = context.lc_runner.invoke(
action_app, ["create", "--config", context.lc_config_path]
)
@when("I run lifecycle coverage action create with missing config")
def step_lc_action_create_missing(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["create", "--config", "/tmp/nonexistent_lc_config.yaml"]
)
@when("I run lifecycle coverage action create with invalid schema config")
def step_lc_action_create_invalid_schema(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["create", "--config", context.lc_bad_schema_path]
)
@when("I run lifecycle coverage action create with value error config")
def step_lc_action_create_value_error(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["create", "--config", context.lc_bad_value_path]
)
@when("I run lifecycle coverage action create with empty config")
def step_lc_action_create_empty(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["create", "--config", context.lc_empty_config_path]
)
@then("lifecycle coverage action create should succeed")
def step_lc_action_create_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0, f"CLI failed: {context.lc_result.output}"
context.lc_mock.create_action.assert_called_once()
@then('lifecycle coverage created action name should be "{name}"')
def step_lc_created_action_name(context: Context, name: str) -> None:
call_kwargs = context.lc_mock.create_action.call_args[1]
assert call_kwargs["name"] == name
@then("lifecycle coverage action CLI should abort")
def step_lc_action_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
# ---------------------------------------------------------------------------
# Action list
# ---------------------------------------------------------------------------
@given("lifecycle coverage mocked actions exist")
def step_lc_mocked_actions(context: Context) -> None:
context.lc_actions = [
_make_lc_action(name="local/alpha-action"),
_make_lc_action(name="local/beta-action"),
_make_lc_action(name="myorg/gamma-action"),
]
context.lc_mock.list_actions.return_value = context.lc_actions
@when("I run lifecycle coverage action list")
def step_lc_action_list(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["list"])
@when('I run lifecycle coverage action list with format "{fmt}"')
def step_lc_action_list_fmt(context: Context, fmt: str) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["list", "--format", fmt])
@when('I run lifecycle coverage action list with regex "{pattern}"')
def step_lc_action_list_regex(context: Context, pattern: str) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["list", pattern])
@when('I run lifecycle coverage action list with invalid regex "{pattern}"')
def step_lc_action_list_invalid_regex(context: Context, pattern: str) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["list", pattern])
@then("lifecycle coverage action list should show table with 3 actions")
def step_lc_action_list_table(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "(3 total)" in context.lc_result.output
@then("lifecycle coverage action list should succeed")
def step_lc_action_list_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage action list filtered should succeed")
def step_lc_action_list_filtered_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
# ---------------------------------------------------------------------------
# Action show
# ---------------------------------------------------------------------------
@given('lifecycle coverage a specific action "{name}" exists')
def step_lc_specific_action(context: Context, name: str) -> None:
action = _make_lc_action(name=name)
context.lc_mock.get_action_by_name.return_value = action
context.lc_existing_action = action
@given("lifecycle coverage action not found for show")
def step_lc_action_not_found_show(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = NotFoundError(
resource_type="action", resource_id="local/no-such-action"
)
@given("lifecycle coverage action not found for archive")
def step_lc_action_not_found_archive(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = NotFoundError(
resource_type="action", resource_id="local/no-such-action"
)
@when('I run lifecycle coverage action show "{name}" with format "{fmt}"')
def step_lc_action_show_fmt(context: Context, name: str, fmt: str) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["show", name, "--format", fmt]
)
@when('I run lifecycle coverage action show "{name}" in default format')
def step_lc_action_show(context: Context, name: str) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["show", name])
@then("lifecycle coverage action show should display details")
def step_lc_action_show_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "Action" in context.lc_result.output
@then("lifecycle coverage action show should succeed")
def step_lc_action_show_succeed(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage action not found should abort")
def step_lc_action_not_found_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
assert "not found" in context.lc_result.output.lower()
# ---------------------------------------------------------------------------
# Action archive
# ---------------------------------------------------------------------------
@given("lifecycle coverage an archivable action exists")
def step_lc_archivable_action(context: Context) -> None:
action = _make_lc_action()
context.lc_mock.get_action_by_name.return_value = action
archived = _make_lc_action(state=ActionState.ARCHIVED)
context.lc_mock.archive_action.return_value = archived
@when('I run lifecycle coverage action archive "{name}" using format "{fmt}"')
def step_lc_action_archive_fmt(context: Context, name: str, fmt: str) -> None:
context.lc_result = context.lc_runner.invoke(
action_app, ["archive", name, "--format", fmt]
)
@when('I run lifecycle coverage action archive "{name}" in default format')
def step_lc_action_archive(context: Context, name: str) -> None:
context.lc_result = context.lc_runner.invoke(action_app, ["archive", name])
@then("lifecycle coverage action archive should succeed")
def step_lc_action_archive_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
context.lc_mock.archive_action.assert_called_once()
@then("lifecycle coverage action archive json should succeed")
def step_lc_action_archive_json_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
# ---------------------------------------------------------------------------
# Plan use
# ---------------------------------------------------------------------------
@given("lifecycle coverage an action for plan use exists")
def step_lc_action_for_plan_use(context: Context) -> None:
action = _make_lc_action()
context.lc_mock.get_action_by_name.return_value = action
context.lc_plan = _make_lc_plan(
project_links=[ProjectLink(project_name="proj-a")],
automation_profile=AutomationProfileRef(
profile_name="trusted",
provenance=AutomationProfileProvenance.PLAN,
),
)
context.lc_mock.use_action.return_value = context.lc_plan
@given("lifecycle coverage an action not available for plan use")
def step_lc_action_not_available(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = None
context.lc_mock.get_action_by_name.return_value = _make_lc_action()
context.lc_mock.use_action.side_effect = ActionNotAvailableError(
"local/archived-action", ActionState.ARCHIVED
)
@given("lifecycle coverage an action with validation error for plan use")
def step_lc_action_validation_error(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = None
context.lc_mock.get_action_by_name.return_value = _make_lc_action()
context.lc_mock.use_action.side_effect = ValidationError("Missing required args")
@given("lifecycle coverage an action with general error for plan use")
def step_lc_action_general_error(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = None
context.lc_mock.get_action_by_name.return_value = _make_lc_action()
context.lc_mock.use_action.side_effect = CleverAgentsError("Something broke")
@given("lifecycle coverage unknown action for plan use")
def step_lc_unknown_action(context: Context) -> None:
context.lc_mock.get_action_by_name.side_effect = NotFoundError(
resource_type="action", resource_id="local/does-not-exist"
)
@when('I run lifecycle coverage plan use "{action}" with projects "{p1}" "{p2}" "{p3}"')
def step_lc_plan_use_multi(
context: Context, action: str, p1: str, p2: str, p3: str
) -> None:
plan = _make_lc_plan(
project_links=[
ProjectLink(project_name=p1),
ProjectLink(project_name=p2),
ProjectLink(project_name=p3),
]
)
context.lc_mock.use_action.return_value = plan
context.lc_plan = plan
context.lc_result = context.lc_runner.invoke(plan_app, ["use", action, p1, p2, p3])
@when(
'I run lifecycle coverage plan use "{action}" on project "{project}" with format "{fmt}"'
)
def step_lc_plan_use_fmt(context: Context, action: str, project: str, fmt: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", action, project, "--format", fmt]
)
@when('I run lifecycle coverage plan use "{action}" targeting project "{project}"')
def step_lc_plan_use(context: Context, action: str, project: str) -> None:
context.lc_result = context.lc_runner.invoke(plan_app, ["use", action, project])
@when('I run lifecycle coverage plan use with automation profile "{profile}"')
def step_lc_plan_use_profile(context: Context, profile: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--automation-profile", profile]
)
@when('I run lifecycle coverage plan use with invariants "{inv1}" and "{inv2}"')
def step_lc_plan_use_invariants(context: Context, inv1: str, inv2: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app,
[
"use",
"local/lc-action",
"proj-a",
"--invariant",
inv1,
"--invariant",
inv2,
],
)
@when('I run lifecycle coverage plan use with strategy actor "{actor}"')
def step_lc_plan_use_strategy_actor(context: Context, actor: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--strategy-actor", actor]
)
@when('I run lifecycle coverage plan use with execution actor "{actor}"')
def step_lc_plan_use_exec_actor(context: Context, actor: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--execution-actor", actor]
)
@when('I run lifecycle coverage plan use with estimation actor "{actor}"')
def step_lc_plan_use_estimation_actor(context: Context, actor: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--estimation-actor", actor]
)
@when('I run lifecycle coverage plan use with invariant actor "{actor}"')
def step_lc_plan_use_invariant_actor(context: Context, actor: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--invariant-actor", actor]
)
@when('I run lifecycle coverage plan use with arg "{arg_str}"')
def step_lc_plan_use_arg(context: Context, arg_str: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--arg", arg_str]
)
@when('I run lifecycle coverage plan use with automation level "{level}"')
def step_lc_plan_use_auto_level(context: Context, level: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--automation-level", level]
)
@when('I run lifecycle coverage plan use with invalid automation level "{level}"')
def step_lc_plan_use_invalid_auto(context: Context, level: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--automation-level", level]
)
@when('I run lifecycle coverage plan use with invalid arg "{arg_str}"')
def step_lc_plan_use_invalid_arg(context: Context, arg_str: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["use", "local/lc-action", "proj-a", "--arg", arg_str]
)
@then("lifecycle coverage plan use should succeed")
def step_lc_plan_use_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0, f"CLI failed: {context.lc_result.output}"
@then("lifecycle coverage plan should be in strategize phase")
def step_lc_plan_strategize(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan should link 3 projects")
def step_lc_plan_3_projects(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert len(context.lc_plan.project_links) == 3
@then('lifecycle coverage plan automation profile should be "{profile}"')
def step_lc_plan_auto_profile(context: Context, profile: str) -> None:
# The profile is applied post-creation in the CLI
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan should have 2 invariants")
def step_lc_plan_invariants(context: Context) -> None:
assert context.lc_result.exit_code == 0
# Verify invariants were passed to use_action
call_kwargs = context.lc_mock.use_action.call_args[1]
invariants = call_kwargs.get("invariants", [])
assert len(invariants) == 2
@then('lifecycle coverage plan strategy actor should be "{actor}"')
def step_lc_plan_strategy_actor(context: Context, actor: str) -> None:
assert context.lc_result.exit_code == 0
@then('lifecycle coverage plan execution actor should be "{actor}"')
def step_lc_plan_exec_actor(context: Context, actor: str) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan use should abort")
def step_lc_plan_use_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
# ---------------------------------------------------------------------------
# Plan lifecycle-list
# ---------------------------------------------------------------------------
@given("lifecycle coverage plans exist for listing")
def step_lc_plans_exist(context: Context) -> None:
context.lc_plans = [
_make_lc_plan(name="local/plan-one", plan_id="01KHDE6WWS2171PWW3GJEBXZ8A"),
_make_lc_plan(
name="local/plan-two",
plan_id="01KHDE6WWS2171PWW3GJEBXZ8B",
phase=PlanPhase.EXECUTE,
state=ProcessingState.PROCESSING,
),
]
context.lc_mock.list_plans.return_value = context.lc_plans
@given("lifecycle coverage no plans exist for listing")
def step_lc_no_plans(context: Context) -> None:
context.lc_mock.list_plans.return_value = []
@when("I run lifecycle coverage plan lifecycle-list")
def step_lc_plan_list(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(plan_app, ["lifecycle-list"])
@when('I run lifecycle coverage plan lifecycle-list with format "{fmt}"')
def step_lc_plan_list_fmt(context: Context, fmt: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["lifecycle-list", "--format", fmt]
)
@when('I run lifecycle coverage plan lifecycle-list with invalid phase "{phase}"')
def step_lc_plan_list_invalid_phase(context: Context, phase: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["lifecycle-list", "--phase", phase]
)
@when('I run lifecycle coverage plan lifecycle-list with invalid state "{state}"')
def step_lc_plan_list_invalid_state(context: Context, state: str) -> None:
context.lc_result = context.lc_runner.invoke(
plan_app, ["lifecycle-list", "--state", state]
)
@then("lifecycle coverage plan list should show table")
def step_lc_plan_list_table(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "Plans" in context.lc_result.output
@then("lifecycle coverage plan list should succeed")
def step_lc_plan_list_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan list should show no plans message")
def step_lc_plan_list_empty(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "No plans found" in context.lc_result.output
@then("lifecycle coverage plan list should abort")
def step_lc_plan_list_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
# ---------------------------------------------------------------------------
# Plan status
# ---------------------------------------------------------------------------
@given("lifecycle coverage a specific plan exists for status")
def step_lc_specific_plan_status(context: Context) -> None:
plan = _make_lc_plan(
project_links=[ProjectLink(project_name="proj-a", alias="api")],
automation_profile=AutomationProfileRef(
profile_name="trusted",
provenance=AutomationProfileProvenance.PLAN,
),
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan in strategize phase exists")
def step_lc_plan_strategize_phase(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.STRATEGIZE, state=ProcessingState.PROCESSING)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan in execute phase exists")
def step_lc_plan_execute_phase(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.EXECUTE, state=ProcessingState.PROCESSING)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan in apply phase exists")
def step_lc_plan_apply_phase(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.APPLY, state=ProcessingState.PROCESSING)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan with applied outcome exists")
def step_lc_plan_applied(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.APPLY, state=ProcessingState.APPLIED)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan with constrained outcome exists")
def step_lc_plan_constrained(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.APPLY,
state=ProcessingState.CONSTRAINED,
error_message="Cannot proceed within constraints",
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan with errored outcome exists")
def step_lc_plan_errored(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.APPLY,
state=ProcessingState.ERRORED,
error_message="Apply failed",
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan with cancelled outcome exists")
def step_lc_plan_cancelled(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.CANCELLED,
error_message="User cancelled",
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage plan status service error")
def step_lc_plan_status_error(context: Context) -> None:
context.lc_mock.get_plan.side_effect = CleverAgentsError("Status fetch failed")
context.lc_plan = _make_lc_plan()
@when("I run lifecycle coverage plan status with plan ID")
def step_lc_plan_status_by_id(context: Context) -> None:
plan_id = getattr(context, "lc_plan", None)
pid = plan_id.identity.plan_id if plan_id else _PLAN_ULID
context.lc_result = context.lc_runner.invoke(plan_app, ["status", pid])
@when("I run lifecycle coverage plan status without ID")
def step_lc_plan_status_no_id(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(plan_app, ["status"])
@when('I run lifecycle coverage plan status with plan ID and format "{fmt}"')
def step_lc_plan_status_fmt(context: Context, fmt: str) -> None:
plan_id = context.lc_plan.identity.plan_id
context.lc_result = context.lc_runner.invoke(
plan_app, ["status", plan_id, "--format", fmt]
)
@then("lifecycle coverage plan status should show details")
def step_lc_plan_status_details(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "Plan" in context.lc_result.output
@then("lifecycle coverage plan status should show table")
def step_lc_plan_status_table(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan status should show no plans")
def step_lc_plan_status_no_plans(context: Context) -> None:
assert context.lc_result.exit_code == 0
assert "No v3 lifecycle plans found" in context.lc_result.output
@then("lifecycle coverage plan status should succeed")
def step_lc_plan_status_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0
@then("lifecycle coverage plan status should abort")
def step_lc_plan_status_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
@then('lifecycle coverage plan status should show phase "{phase}"')
def step_lc_plan_status_phase(context: Context, phase: str) -> None:
assert context.lc_result.exit_code == 0
output_lower = context.lc_result.output.lower()
assert phase.lower() in output_lower
@then('lifecycle coverage plan status should show state "{state}"')
def step_lc_plan_status_state(context: Context, state: str) -> None:
assert context.lc_result.exit_code == 0
output_lower = context.lc_result.output.lower()
assert state.lower() in output_lower
@then("lifecycle coverage plan should be terminal")
def step_lc_plan_terminal(context: Context) -> None:
assert context.lc_plan.is_terminal
# ---------------------------------------------------------------------------
# Plan execute
# ---------------------------------------------------------------------------
@given("lifecycle coverage a plan ready for execute exists")
def step_lc_plan_ready_execute(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.EXECUTE, state=ProcessingState.QUEUED)
context.lc_mock.execute_plan.return_value = plan
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a single plan ready for auto-execute exists")
def step_lc_single_execute(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.COMPLETE,
)
context.lc_mock.list_plans.return_value = [plan]
context.lc_mock.get_plan.return_value = plan
executed = _make_lc_plan(phase=PlanPhase.EXECUTE, state=ProcessingState.QUEUED)
context.lc_mock.execute_plan.return_value = executed
context.lc_plan = plan
@given("lifecycle coverage no plans ready for execute")
def step_lc_no_execute(context: Context) -> None:
context.lc_mock.list_plans.return_value = []
@given("lifecycle coverage multiple plans ready for execute")
def step_lc_multi_execute(context: Context) -> None:
plans = [
_make_lc_plan(
name="local/plan-x",
plan_id="01KHDE6WWS2171PWW3GJEBXZ8C",
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.COMPLETE,
),
_make_lc_plan(
name="local/plan-y",
plan_id="01KHDE6WWS2171PWW3GJEBXZ8D",
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.COMPLETE,
),
]
context.lc_mock.list_plans.return_value = plans
@given("lifecycle coverage a plan with invalid transition for execute")
def step_lc_invalid_transition(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.STRATEGIZE, state=ProcessingState.COMPLETE)
context.lc_mock.execute_plan.side_effect = InvalidPhaseTransitionError(
PlanPhase.APPLY, PlanPhase.EXECUTE
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan not ready for execute")
def step_lc_plan_not_ready(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.STRATEGIZE, state=ProcessingState.COMPLETE)
context.lc_mock.execute_plan.side_effect = PlanNotReadyError(
_PLAN_ULID, PlanPhase.STRATEGIZE, ProcessingState.QUEUED
)
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a plan execute with general error")
def step_lc_plan_exec_general_error(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.STRATEGIZE, state=ProcessingState.COMPLETE)
context.lc_mock.execute_plan.side_effect = CleverAgentsError("Execute failed")
context.lc_mock.get_plan.return_value = plan
context.lc_plan = plan
@when("I run lifecycle coverage plan execute with plan ID")
def step_lc_plan_execute(context: Context) -> None:
pid = getattr(context, "lc_plan", None)
plan_id = pid.identity.plan_id if pid else _PLAN_ULID
context.lc_result = context.lc_runner.invoke(plan_app, ["execute", plan_id])
@when("I run lifecycle coverage plan execute without ID")
def step_lc_plan_execute_no_id(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(plan_app, ["execute"])
@when('I run lifecycle coverage plan execute with plan ID and format "{fmt}"')
def step_lc_plan_execute_fmt(context: Context, fmt: str) -> None:
pid = context.lc_plan.identity.plan_id
context.lc_result = context.lc_runner.invoke(
plan_app, ["execute", pid, "--format", fmt]
)
@then("lifecycle coverage plan execute should succeed")
def step_lc_plan_execute_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0, f"CLI failed: {context.lc_result.output}"
@then("lifecycle coverage plan execute should abort")
def step_lc_plan_execute_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
# ---------------------------------------------------------------------------
# Plan lifecycle-apply
# ---------------------------------------------------------------------------
@given("lifecycle coverage a plan ready for apply exists")
def step_lc_plan_ready_apply(context: Context) -> None:
plan = _make_lc_plan(phase=PlanPhase.APPLY, state=ProcessingState.QUEUED)
context.lc_mock.apply_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a single plan ready for auto-apply exists")
def step_lc_single_apply(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.EXECUTE,
state=ProcessingState.COMPLETE,
)
context.lc_mock.list_plans.return_value = [plan]
applied = _make_lc_plan(phase=PlanPhase.APPLY, state=ProcessingState.QUEUED)
context.lc_mock.apply_plan.return_value = applied
context.lc_plan = plan
@given("lifecycle coverage no plans ready for apply")
def step_lc_no_apply(context: Context) -> None:
context.lc_mock.list_plans.return_value = []
@given("lifecycle coverage multiple plans ready for apply")
def step_lc_multi_apply(context: Context) -> None:
plans = [
_make_lc_plan(
name="local/plan-m",
plan_id="01KHDE6WWS2171PWW3GJEBXZ8E",
phase=PlanPhase.EXECUTE,
state=ProcessingState.COMPLETE,
),
_make_lc_plan(
name="local/plan-n",
plan_id="01KHDE6WWS2171PWW3GJEBXZ8F",
phase=PlanPhase.EXECUTE,
state=ProcessingState.COMPLETE,
),
]
context.lc_mock.list_plans.return_value = plans
@given("lifecycle coverage a plan apply with general error")
def step_lc_plan_apply_general_error(context: Context) -> None:
context.lc_mock.apply_plan.side_effect = CleverAgentsError("Apply failed")
context.lc_plan = _make_lc_plan()
@when("I run lifecycle coverage plan lifecycle-apply with plan ID")
def step_lc_plan_apply(context: Context) -> None:
pid = getattr(context, "lc_plan", None)
plan_id = pid.identity.plan_id if pid else _PLAN_ULID
context.lc_result = context.lc_runner.invoke(
plan_app, ["lifecycle-apply", "--yes", plan_id]
)
@when("I run lifecycle coverage plan lifecycle-apply without ID")
def step_lc_plan_apply_no_id(context: Context) -> None:
context.lc_result = context.lc_runner.invoke(plan_app, ["lifecycle-apply", "--yes"])
@when('I run lifecycle coverage plan lifecycle-apply with plan ID and format "{fmt}"')
def step_lc_plan_apply_fmt(context: Context, fmt: str) -> None:
pid = context.lc_plan.identity.plan_id
context.lc_result = context.lc_runner.invoke(
plan_app, ["lifecycle-apply", "--yes", pid, "--format", fmt]
)
@then("lifecycle coverage plan apply should succeed")
def step_lc_plan_apply_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0, f"CLI failed: {context.lc_result.output}"
@then("lifecycle coverage plan apply should abort")
def step_lc_plan_apply_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0
# ---------------------------------------------------------------------------
# Plan cancel
# ---------------------------------------------------------------------------
@given("lifecycle coverage a cancellable plan exists")
def step_lc_cancellable_plan(context: Context) -> None:
plan = _make_lc_plan(
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.CANCELLED,
)
context.lc_mock.cancel_plan.return_value = plan
context.lc_plan = plan
@given("lifecycle coverage a terminal plan exists for cancel")
def step_lc_terminal_cancel(context: Context) -> None:
context.lc_mock.cancel_plan.side_effect = PlanError(
"Plan is already in terminal state"
)
context.lc_plan = _make_lc_plan()
@given("lifecycle coverage a plan cancel with general error")
def step_lc_cancel_general_error(context: Context) -> None:
context.lc_mock.cancel_plan.side_effect = CleverAgentsError("Cancel failed")
context.lc_plan = _make_lc_plan()
@when('I run lifecycle coverage plan cancel with reason "{reason}" and format "{fmt}"')
def step_lc_plan_cancel_fmt(context: Context, reason: str, fmt: str) -> None:
pid = context.lc_plan.identity.plan_id
context.lc_result = context.lc_runner.invoke(
plan_app, ["cancel", pid, "--reason", reason, "--format", fmt]
)
@when('I run lifecycle coverage plan cancel providing reason "{reason}"')
def step_lc_plan_cancel_reason(context: Context, reason: str) -> None:
pid = getattr(context, "lc_plan", None)
plan_id = pid.identity.plan_id if pid else _PLAN_ULID
context.lc_result = context.lc_runner.invoke(
plan_app, ["cancel", plan_id, "--reason", reason]
)
@when("I run lifecycle coverage plan cancel without reason")
def step_lc_plan_cancel_no_reason(context: Context) -> None:
pid = context.lc_plan.identity.plan_id
context.lc_result = context.lc_runner.invoke(plan_app, ["cancel", pid])
@then("lifecycle coverage plan cancel should succeed")
def step_lc_plan_cancel_ok(context: Context) -> None:
assert context.lc_result.exit_code == 0, f"CLI failed: {context.lc_result.output}"
@then('lifecycle coverage cancel output should contain "{text}"')
def step_lc_cancel_output(context: Context, text: str) -> None:
assert text in context.lc_result.output
@then("lifecycle coverage plan cancel should abort")
def step_lc_plan_cancel_abort(context: Context) -> None:
assert context.lc_result.exit_code != 0