Files
cleveragents-core/features/steps/m1_sourcecode_smoke_steps.py
T
freemo 5f07316641
CI / lint (pull_request) Successful in 16s
CI / typecheck (pull_request) Successful in 42s
CI / security (pull_request) Successful in 46s
CI / quality (pull_request) Successful in 27s
CI / unit_tests (pull_request) Successful in 3m16s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 15s
CI / e2e_tests (pull_request) Successful in 1m12s
CI / integration_tests (pull_request) Successful in 4m20s
CI / docker (pull_request) Successful in 59s
CI / coverage (pull_request) Successful in 6m34s
CI / lint (push) Successful in 20s
CI / typecheck (push) Successful in 45s
CI / security (push) Successful in 46s
CI / quality (push) Successful in 37s
CI / build (push) Successful in 17s
CI / e2e_tests (push) Successful in 52s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 5m9s
CI / integration_tests (push) Successful in 5m32s
CI / docker (push) Successful in 57s
CI / coverage (push) Successful in 6m10s
CI / benchmark-publish (push) Successful in 20m11s
CI / benchmark-regression (pull_request) Successful in 38m29s
fix: wire DI persistence and plan execute/apply for M1 lifecycle
Fixed 5 bugs preventing the M1 E2E acceptance test from passing:

1. _get_lifecycle_service() in action.py and plan.py bypassed the DI
   container, creating PlanLifecycleService without UnitOfWork. All
   plan/action data was in-memory only and lost between subprocess
   calls. Now uses container.plan_lifecycle_service() for DB persistence.

2. `plan execute` CLI only called service.execute_plan() (a pure state
   transition) without running PlanExecutor phase processing. Rewrote
   to detect the plan's current phase/state and dispatch synchronously:
   Strategize/queued → run_strategize(), Strategize/complete → transition
   + run_execute(), Execute/queued → run_execute().

3. `plan apply` CLI had no plan_id argument. Added optional positional
   plan_id with _lifecycle_apply_with_id() that drives the plan through
   Apply/queued → Apply/processing → Apply/applied.

4. Preflight guardrail in start_strategize() built action_registry from
   the in-memory _actions dict only. Added get_action(plan.action_name)
   call to load the action from DB into cache before the guardrail check.

5. Robot Framework Create File syntax used continuation lines producing
   9 arguments instead of 1. Fixed to use Catenate SEPARATOR=\n then
   pass single variable to Create File. Also fixed --branch main to
   --branch master (git init default).

update mocks for execute_plan CLI changes across unit and integration tests

The new execute_plan() command calls _get_plan_executor() and
service.get_plan(plan_id) for phase/state detection. Existing tests
only mocked _get_lifecycle_service, so MagicMock defaults caused
phase/state comparisons to fail.

Changes across 14 files:
- Patch _get_plan_executor in all test setups that invoke the CLI
  execute command (Behave step files + Robot helper scripts)
- Set service.get_plan.return_value to real Plan objects with correct
  phase/state so the execute_plan dispatch logic works
- Fix error-path tests to use STRATEGIZE/COMPLETE plans so the error
  side_effects are actually reached
- Fix "Multiple plans eligible" → "Multiple plans ready" message text
  to match existing test expectations

increase Robot Framework subprocess timeouts for CI resource contention

Three integration tests were timing out in CI due to resource contention
when pabot runs multiple test suites in parallel. All three pass locally
and the timeouts were simply too tight for constrained CI environments.

- tdd_session_create_di.robot: 30s → 90s (DI container init + DB setup)
- database_integration.robot: 60s → 120s (Run Python Script keyword)
- m3_e2e_verification.robot: 60s → 120s (correction-live-revert spawns
  3 sequential CLI subprocesses with full container initialization)

ISSUES CLOSED: #789
2026-03-15 20:50:02 +00:00

602 lines
22 KiB
Python

"""Step definitions for M1 source-code plan lifecycle smoke tests.
All step names are prefixed with ``m1 smoke`` to avoid ``AmbiguousStep``
conflicts with existing steps.
"""
from __future__ import annotations
import contextlib
import json
from datetime import datetime
from pathlib import Path
from unittest.mock import MagicMock, patch
import yaml
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.cli.commands.action import app as action_app
from cleveragents.cli.commands.plan import app as plan_app
from cleveragents.cli.commands.project import app as project_app
from cleveragents.domain.models.core.action import Action, ActionState
from cleveragents.domain.models.core.plan import (
NamespacedName,
Plan,
PlanIdentity,
PlanPhase,
PlanTimestamps,
ProcessingState,
ProjectLink,
)
_FIXTURES_DIR = Path(__file__).resolve().parents[1] / "fixtures" / "m1"
_PLAN_ULID = "01M1SM0KE00000000000000001"
def _make_m1_plan(
*,
name: str = "local/m1-smoke-plan",
action_name: str = "local/m1-source-review",
phase: PlanPhase = PlanPhase.STRATEGIZE,
state: ProcessingState = ProcessingState.QUEUED,
project_links: list[ProjectLink] | None = None,
arguments: dict[str, object] | None = None,
is_terminal: bool = False,
) -> Plan:
"""Create a Plan instance for M1 smoke tests."""
now = datetime.now()
return Plan(
identity=PlanIdentity(plan_id=_PLAN_ULID),
namespaced_name=NamespacedName.parse(name),
description="M1 smoke test plan",
definition_of_done="Source code reviewed",
action_name=action_name,
phase=phase,
processing_state=state,
project_links=project_links or [],
arguments=dict(arguments) if arguments else {},
arguments_order=[],
strategy_actor="openai/gpt-4",
execution_actor="openai/gpt-4",
reusable=True,
read_only=False,
created_by=None,
timestamps=PlanTimestamps(created_at=now, updated_at=now),
)
def _make_m1_action(
name: str = "local/m1-source-review",
) -> Action:
"""Create an Action for M1 smoke tests."""
return Action(
namespaced_name=NamespacedName.parse(name),
description="Minimal source-code review action",
long_description=None,
definition_of_done="Source code reviewed",
strategy_actor="openai/gpt-4",
execution_actor="openai/gpt-4",
reusable=True,
read_only=True,
state=ActionState.AVAILABLE,
created_by=None,
created_at=datetime.now(),
updated_at=datetime.now(),
)
# ---------------------------------------------------------------------------
# Background
# ---------------------------------------------------------------------------
@given("a m1 smoke test runner")
def step_m1_smoke_runner(context: Context) -> None:
"""Set up the CLI runner for M1 smoke tests."""
context.runner = CliRunner()
@given("a m1 smoke mocked lifecycle service")
def step_m1_smoke_mock_service(context: Context) -> None:
"""Set up the mocked lifecycle service for M1 smoke tests."""
from cleveragents.application.services.plan_lifecycle_service import (
ActionNotAvailableError,
)
context.mock_service = MagicMock()
# Default: get_action_by_name raises for any action not explicitly set up
context._m1_known_actions = {} # type-checked in step helpers
def _get_action_by_name_side_effect(name: str) -> Action:
if name in context._m1_known_actions:
return context._m1_known_actions[name]
raise ActionNotAvailableError(action_name=name, state=ActionState.ARCHIVED)
context.mock_service.get_action_by_name.side_effect = (
_get_action_by_name_side_effect
)
context.plan_patcher = patch(
"cleveragents.cli.commands.plan._get_lifecycle_service",
return_value=context.mock_service,
)
context.action_patcher = patch(
"cleveragents.cli.commands.action._get_lifecycle_service",
return_value=context.mock_service,
)
context.project_patcher = patch(
"cleveragents.cli.commands.project._get_namespaced_project_repo",
)
context.resource_patcher = patch(
"cleveragents.cli.commands.project._get_resource_registry_service",
)
context.link_patcher = patch(
"cleveragents.cli.commands.project._get_resource_link_repo",
)
context.plan_patcher.start()
context.action_patcher.start()
context.mock_project_repo = context.project_patcher.start()
context.mock_resource_svc = context.resource_patcher.start()
context.mock_link_repo = context.link_patcher.start()
context.last_result = None
context.captured_plan_id = None
# ---------------------------------------------------------------------------
# Fixture loading steps
# ---------------------------------------------------------------------------
@when("I m1 smoke load the git repo fixture")
def step_m1_load_git_repo(context: Context) -> None:
"""Load the git repo fixture JSON."""
fixture_path = _FIXTURES_DIR / "git_repo.json"
with open(fixture_path) as f:
context.git_repo_fixtures = json.load(f)
@then("the m1 smoke git repo fixture should have a minimal repo entry")
def step_m1_git_repo_has_minimal(context: Context) -> None:
"""Verify fixture has a minimal_git_repo entry."""
fixtures = context.git_repo_fixtures["fixtures"]
names = [f["name"] for f in fixtures]
assert "minimal_git_repo" in names, f"Expected 'minimal_git_repo' in {names}"
@then("the m1 smoke minimal repo should contain expected files")
def step_m1_git_repo_has_files(context: Context) -> None:
"""Verify the minimal repo defines expected files."""
fixtures = context.git_repo_fixtures["fixtures"]
minimal = next(f for f in fixtures if f["name"] == "minimal_git_repo")
files = minimal["files"]
assert "README.md" in files, "Expected README.md in fixture files"
assert "src/main.py" in files, "Expected src/main.py in fixture files"
@when("I m1 smoke load the git checkout resource fixture")
def step_m1_load_git_checkout(context: Context) -> None:
"""Load the git-checkout resource fixture JSON."""
fixture_path = _FIXTURES_DIR / "git_checkout_resource.json"
with open(fixture_path) as f:
context.checkout_fixtures = json.load(f)
@then("the m1 smoke git checkout fixture should have a minimal entry")
def step_m1_checkout_has_minimal(context: Context) -> None:
"""Verify fixture has a minimal_git_checkout entry."""
fixtures = context.checkout_fixtures["fixtures"]
names = [f["name"] for f in fixtures]
assert "minimal_git_checkout" in names, (
f"Expected 'minimal_git_checkout' in {names}"
)
@then('the m1 smoke minimal checkout should have type "{rtype}"')
def step_m1_checkout_type(context: Context, rtype: str) -> None:
"""Verify the minimal checkout has the expected resource type."""
fixtures = context.checkout_fixtures["fixtures"]
minimal = next(f for f in fixtures if f["name"] == "minimal_git_checkout")
assert minimal["resource_type"] == rtype, (
f"Expected type '{rtype}', got '{minimal['resource_type']}'"
)
@when("I m1 smoke load the action YAML fixture")
def step_m1_load_action_yaml(context: Context) -> None:
"""Load the action YAML fixture."""
fixture_path = _FIXTURES_DIR / "action_sourcecode.yaml"
with open(fixture_path) as f:
context.action_fixture = yaml.safe_load(f)
@then('the m1 smoke action fixture should define strategy actor "{actor}"')
def step_m1_action_strategy_actor(context: Context, actor: str) -> None:
"""Verify the action fixture defines the expected strategy actor."""
assert context.action_fixture["strategy_actor"] == actor
@then('the m1 smoke action fixture should define execution actor "{actor}"')
def step_m1_action_execution_actor(context: Context, actor: str) -> None:
"""Verify the action fixture defines the expected execution actor."""
assert context.action_fixture["execution_actor"] == actor
# ---------------------------------------------------------------------------
# Action create steps
# ---------------------------------------------------------------------------
@given("a m1 smoke temporary action config file")
def step_m1_temp_action_config(context: Context) -> None:
"""Create a temp YAML config file for action create."""
fixture_path = _FIXTURES_DIR / "action_sourcecode.yaml"
context.m1_action_config = str(fixture_path)
context.mock_service.create_action.return_value = _make_m1_action()
@when("I m1 smoke invoke action create with the config")
def step_m1_invoke_action_create(context: Context) -> None:
"""Invoke action create CLI with config file."""
result = context.runner.invoke(
action_app,
["create", "--config", context.m1_action_config],
)
context.last_result = result
@then("the m1 smoke action create should succeed")
def step_m1_action_create_ok(context: Context) -> None:
"""Verify action create succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
@then('the m1 smoke action output should contain "{text}"')
def step_m1_action_output_contains(context: Context, text: str) -> None:
"""Verify action output contains expected text."""
output = context.last_result.output
assert text in output, f"Expected '{text}' in: {output}"
# ---------------------------------------------------------------------------
# Project + resource link steps
# ---------------------------------------------------------------------------
@when('I m1 smoke create a temp project "{name}"')
def step_m1_create_project(context: Context, name: str) -> None:
"""Create a temp project via the mock."""
mock_repo = context.mock_project_repo.return_value
mock_project = MagicMock()
mock_project.namespaced_name = name
mock_project.namespace = name.split("/")[0]
mock_project.name = name.split("/")[1]
mock_project.description = "M1 smoke test project"
mock_project.linked_resources = []
mock_project.created_at = datetime.now()
mock_project.updated_at = datetime.now()
mock_repo.create.return_value = None
mock_repo.get.return_value = mock_project
result = context.runner.invoke(
project_app,
["create", name, "--format", "plain"],
)
context.last_result = result
@then("the m1 smoke project creation should succeed")
def step_m1_project_create_ok(context: Context) -> None:
"""Verify project creation succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
@then('the m1 smoke project output should contain "{text}"')
def step_m1_project_output_contains(context: Context, text: str) -> None:
"""Verify project output contains expected text."""
output = context.last_result.output
assert text in output, f"Expected '{text}' in: {output}"
@given('a m1 smoke project "{name}" exists')
def step_m1_project_exists(context: Context, name: str) -> None:
"""Set up existing project mock."""
mock_repo = context.mock_project_repo.return_value
mock_project = MagicMock()
mock_project.namespaced_name = name
mock_project.namespace = name.split("/")[0]
mock_project.name = name.split("/")[1]
mock_project.linked_resources = []
mock_repo.get.return_value = mock_project
@given('a m1 smoke resource "{name}" exists')
def step_m1_resource_exists(context: Context, name: str) -> None:
"""Set up existing resource mock."""
mock_registry = context.mock_resource_svc.return_value
mock_resource = MagicMock()
mock_resource.resource_id = "01M1RESOURCE000000000000001"
mock_resource.name = name
mock_registry.show_resource.return_value = mock_resource
@when('I m1 smoke link resource "{resource}" to project "{project}"')
def step_m1_link_resource(context: Context, resource: str, project: str) -> None:
"""Link a resource to a project via CLI."""
mock_link_repo = context.mock_link_repo.return_value
mock_link_repo.create_link.return_value = None
result = context.runner.invoke(
project_app,
["link-resource", project, resource, "--format", "plain"],
)
context.last_result = result
@then("the m1 smoke link should succeed")
def step_m1_link_ok(context: Context) -> None:
"""Verify resource link succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
# ---------------------------------------------------------------------------
# Plan use steps
# ---------------------------------------------------------------------------
@given('a m1 smoke action "{name}" exists')
def step_m1_action_exists(context: Context, name: str) -> None:
"""Set up an existing action for plan use."""
action = _make_m1_action(name)
context._m1_known_actions[name] = action
context.mock_service.use_action.return_value = _make_m1_plan(action_name=name)
@when('I m1 smoke invoke plan use with action "{name}"')
def step_m1_plan_use(context: Context, name: str) -> None:
"""Invoke plan use with the given action."""
result = context.runner.invoke(plan_app, ["use", name])
context.last_result = result
if result.exit_code == 0:
context.captured_plan_id = _PLAN_ULID
@when('I m1 smoke invoke plan use linking project "{project}" to action "{name}"')
def step_m1_plan_use_with_project(context: Context, project: str, name: str) -> None:
"""Invoke plan use with action and project."""
result = context.runner.invoke(plan_app, ["use", name, project])
context.last_result = result
@when('I m1 smoke invoke plan use passing arg "{arg_str}" to action "{name}"')
def step_m1_plan_use_with_arg(context: Context, arg_str: str, name: str) -> None:
"""Invoke plan use with action and --arg."""
result = context.runner.invoke(plan_app, ["use", name, "--arg", arg_str])
context.last_result = result
if result.exit_code == 0:
context.captured_plan_id = _PLAN_ULID
context.last_result = result
@when('I m1 smoke invoke plan use with invalid strategy actor "{actor}"')
def step_m1_plan_use_invalid_actor(context: Context, actor: str) -> None:
"""Invoke plan use with an invalid actor format."""
result = context.runner.invoke(
plan_app, ["use", "local/m1-source-review", "--strategy-actor", actor]
)
context.last_result = result
@then("the m1 smoke plan use should succeed")
def step_m1_plan_use_ok(context: Context) -> None:
"""Verify plan use succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
@then("the m1 smoke plan use should fail")
def step_m1_plan_use_fail(context: Context) -> None:
"""Verify plan use failed."""
assert context.last_result is not None
assert context.last_result.exit_code != 0
@then('the m1 smoke plan should be in phase "{phase}"')
def step_m1_plan_phase(context: Context, phase: str) -> None:
"""Verify plan is in the expected phase."""
output = context.last_result.output
assert phase in output.lower(), f"Expected phase '{phase}' in output: {output}"
@then("the m1 smoke captured plan id should not be empty")
def step_m1_plan_id_captured(context: Context) -> None:
"""Verify we captured a plan ID."""
assert context.captured_plan_id is not None
assert len(context.captured_plan_id) > 0
# ---------------------------------------------------------------------------
# Plan execute steps
# ---------------------------------------------------------------------------
@given("a m1 smoke plan exists in strategize phase")
def step_m1_plan_in_strategize(context: Context) -> None:
"""Set up a plan in strategize/complete phase for execute."""
plan = _make_m1_plan(
phase=PlanPhase.STRATEGIZE,
state=ProcessingState.COMPLETE,
)
context.mock_service.list_plans.return_value = [plan]
context.mock_service.get_plan.return_value = plan
context.mock_service.execute_plan.return_value = _make_m1_plan(
phase=PlanPhase.EXECUTE,
state=ProcessingState.QUEUED,
)
# Patch the plan executor so the execute phase runs with a mock
context._m1_executor_patcher = patch(
"cleveragents.cli.commands.plan._get_plan_executor",
return_value=MagicMock(),
)
context._m1_executor_patcher.start()
@when("I m1 smoke invoke plan execute")
def step_m1_plan_execute(context: Context) -> None:
"""Invoke plan execute."""
result = context.runner.invoke(plan_app, ["execute"])
context.last_result = result
# Clean up executor patcher if it was set
with contextlib.suppress(AttributeError):
context._m1_executor_patcher.stop()
@then("the m1 smoke plan execute should succeed")
def step_m1_plan_execute_ok(context: Context) -> None:
"""Verify plan execute succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
@then("the m1 smoke plan execute should fail")
def step_m1_plan_execute_fail(context: Context) -> None:
"""Verify plan execute failed."""
assert context.last_result is not None
assert context.last_result.exit_code != 0
# ---------------------------------------------------------------------------
# Plan diff steps
# ---------------------------------------------------------------------------
@given("a m1 smoke plan exists in execute phase with changeset")
def step_m1_plan_with_changeset(context: Context) -> None:
"""Set up a plan in execute phase with a changeset mock."""
plan = _make_m1_plan(
phase=PlanPhase.EXECUTE,
state=ProcessingState.COMPLETE,
)
context.mock_service.get_plan.return_value = plan
context.mock_service.list_plans.return_value = [plan]
# Mock the apply service since plan diff uses _get_apply_service()
mock_apply_svc = MagicMock()
mock_apply_svc.diff.return_value = "No changes detected."
context.apply_patcher = patch(
"cleveragents.cli.commands.plan._get_apply_service",
return_value=mock_apply_svc,
)
context.apply_patcher.start()
@when("I m1 smoke invoke plan diff")
def step_m1_plan_diff(context: Context) -> None:
"""Invoke plan diff."""
result = context.runner.invoke(plan_app, ["diff", _PLAN_ULID])
context.last_result = result
@then("the m1 smoke plan diff should succeed")
def step_m1_plan_diff_ok(context: Context) -> None:
"""Verify plan diff succeeded."""
assert context.last_result is not None
# diff may succeed or return 0 even without changes
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
# ---------------------------------------------------------------------------
# Plan apply steps
# ---------------------------------------------------------------------------
@given("a m1 smoke plan exists in apply phase")
def step_m1_plan_in_apply(context: Context) -> None:
"""Set up a plan in apply phase."""
plan = _make_m1_plan(
phase=PlanPhase.APPLY,
state=ProcessingState.COMPLETE,
)
context.mock_service.list_plans.return_value = [plan]
context.mock_service.get_plan.return_value = plan
context.mock_service.apply_plan.return_value = _make_m1_plan(
phase=PlanPhase.APPLY,
state=ProcessingState.APPLIED,
)
# For execute attempts on an apply-phase plan, raise error
from cleveragents.application.services.plan_lifecycle_service import (
InvalidPhaseTransitionError,
)
context.mock_service.execute_plan.side_effect = InvalidPhaseTransitionError(
from_phase=PlanPhase.APPLY,
to_phase=PlanPhase.EXECUTE,
message="Cannot execute a plan in apply phase",
)
@when("I m1 smoke invoke plan lifecycle-apply")
def step_m1_plan_apply(context: Context) -> None:
"""Invoke plan lifecycle-apply."""
result = context.runner.invoke(plan_app, ["lifecycle-apply", _PLAN_ULID])
context.last_result = result
@then("the m1 smoke plan apply should succeed")
def step_m1_plan_apply_ok(context: Context) -> None:
"""Verify plan apply succeeded."""
assert context.last_result is not None
assert context.last_result.exit_code == 0, (
f"Expected exit 0, got {context.last_result.exit_code}. "
f"Output: {context.last_result.output}"
)
@then("the m1 smoke plan should be in terminal state")
def step_m1_plan_terminal(context: Context) -> None:
"""Verify the plan is in terminal state."""
output = context.last_result.output.lower()
assert "appl" in output or "terminal" in output, (
f"Expected terminal state indicator in: {context.last_result.output}"
)
# ---------------------------------------------------------------------------
# Cleanup
# ---------------------------------------------------------------------------
def after_scenario(context: Context, scenario: object) -> None:
"""Clean up patchers after each scenario."""
for name in (
"plan_patcher",
"action_patcher",
"project_patcher",
"resource_patcher",
"link_patcher",
"apply_patcher",
):
patcher = getattr(context, name, None)
if patcher:
patcher.stop()