Merge branch 'master' into test/e2e-m5-acceptance
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 19s
CI / quality (pull_request) Successful in 30s
CI / build (pull_request) Successful in 30s
CI / security (pull_request) Successful in 45s
CI / typecheck (pull_request) Successful in 46s
CI / unit_tests (pull_request) Successful in 3m8s
CI / integration_tests (pull_request) Successful in 3m34s
CI / docker (pull_request) Successful in 57s
CI / e2e_tests (pull_request) Successful in 5m24s
CI / coverage (pull_request) Successful in 6m56s
CI / benchmark-regression (pull_request) Failing after 31m11s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 19s
CI / quality (pull_request) Successful in 30s
CI / build (pull_request) Successful in 30s
CI / security (pull_request) Successful in 45s
CI / typecheck (pull_request) Successful in 46s
CI / unit_tests (pull_request) Successful in 3m8s
CI / integration_tests (pull_request) Successful in 3m34s
CI / docker (pull_request) Successful in 57s
CI / e2e_tests (pull_request) Successful in 5m24s
CI / coverage (pull_request) Successful in 6m56s
CI / benchmark-regression (pull_request) Failing after 31m11s
This commit is contained in:
@@ -2,6 +2,15 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Added TDD bug-capture tests for bug #967 — `plan execute` phase processing.
|
||||
Tests exercise the CLI orchestration layer via CliRunner (Behave) and
|
||||
replicated CLI logic (Robot) to verify that `plan execute` correctly
|
||||
handles plans in Strategize/QUEUED state by running `run_strategize()`
|
||||
before transitioning. Includes four Behave scenarios and four Robot
|
||||
integration test cases covering CLI execute from QUEUED, full lifecycle
|
||||
orchestration, positive control, and auto-discovery of QUEUED plans.
|
||||
(`features/tdd_plan_execute_phase_processing.feature`,
|
||||
`robot/tdd_plan_execute_phase_processing.robot`) (#977)
|
||||
- Added built-in deferred virtual resource types: `remote`, `submodule`, and
|
||||
`symlink` with equivalence metadata rules for cross-repo and cross-layer
|
||||
identity tracking. Registry bootstrap includes deferred virtual types but
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
"""Step definitions for TDD Bug #783 — init --yes should not require user input.
|
||||
|
||||
These steps exercise the *real* ``agents init --yes`` code path while
|
||||
simulating a TTY environment by replacing the migration runner's
|
||||
``_default_prompt_for_migration`` with a function that declines
|
||||
migration (simulating a user on a real terminal providing no input).
|
||||
The ``@tdd_expected_fail`` tag on the scenarios inverts the result:
|
||||
these tests **pass** CI while the bug is present and will **fail** once
|
||||
the bug is fixed (signalling that the tag should be removed).
|
||||
|
||||
Root cause
|
||||
~~~~~~~~~~
|
||||
``init_command()`` in ``project.py`` receives the ``--yes`` flag but
|
||||
does not forward it to the migration runner. The migration runner's
|
||||
``_default_prompt_for_migration()`` calls ``typer.confirm()`` when a
|
||||
TTY is detected, blocking the command for user input even though
|
||||
``--yes`` was specified. In a non-TTY environment (like CliRunner),
|
||||
the prompt is silently skipped — the bug only surfaces in real terminal
|
||||
sessions. We replace the prompt function to reproduce the TTY code
|
||||
path.
|
||||
|
||||
Mock strategy
|
||||
~~~~~~~~~~~~~
|
||||
CliRunner replaces ``sys.stdin`` during ``invoke()``, so patching
|
||||
``sys.stdin.isatty()`` alone is ineffective. Instead we:
|
||||
|
||||
1. Patch ``sys.stdin`` on the real ``sys`` module (documents intent
|
||||
and is the correct target per the project's established pattern in
|
||||
``features/steps/migration_runner_steps.py``).
|
||||
2. Patch ``MigrationRunner._default_prompt_for_migration`` with a
|
||||
replacement that returns ``False`` — simulating a TTY prompt where
|
||||
the user declines migration. This ensures the prompt code path is
|
||||
exercised deterministically regardless of CliRunner's stdin
|
||||
replacement.
|
||||
3. Set ``CLEVERAGENTS_DATABASE_URL`` to a path inside our tmp directory
|
||||
whose filename does **not** match the ``before_scenario`` template-DB
|
||||
prefixes (``cleveragents_*``, ``test_*``, ``db.*``). This forces the
|
||||
migration runner's ``init_or_upgrade`` through the real Alembic path
|
||||
instead of the test fast-path that copies a pre-migrated template.
|
||||
|
||||
With bug #783 present, ``require_confirmation=True`` is hardcoded in
|
||||
``UnitOfWork._ensure_database_initialized()``, so the prompt fires and
|
||||
returns ``False``, causing ``MigrationNotApprovedError``. After the
|
||||
fix, ``--yes`` should bypass the prompt entirely.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.container import reset_container
|
||||
from cleveragents.cli.commands.project import app as project_app
|
||||
from cleveragents.infrastructure.database.migration_runner import MigrationRunner
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def _tty_prompt_declines(_message: str) -> bool:
|
||||
"""Simulate a TTY migration prompt that declines (no user input).
|
||||
|
||||
Replaces ``_default_prompt_for_migration`` to reproduce the TTY
|
||||
code path without depending on ``sys.stdin.isatty()``, which
|
||||
CliRunner overrides during ``invoke()``.
|
||||
"""
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a fresh environment for tdd-init-yes-no-input")
|
||||
def step_fresh_environment(context: Context) -> None:
|
||||
"""Set up a fresh temporary environment with no existing database.
|
||||
|
||||
Creates a temporary directory to serve as the working directory so
|
||||
that ``agents init`` creates a fresh ``.cleveragents`` directory.
|
||||
Disables auto-apply environment variables so the real prompt
|
||||
behaviour is exercised.
|
||||
"""
|
||||
context.tdd_init_tmpdir = tempfile.mkdtemp(prefix="tdd_init_yes_")
|
||||
|
||||
# Save env vars that would bypass the prompt or redirect the
|
||||
# database, and remove them. ``CLEVERAGENTS_DATABASE_URL`` and
|
||||
# ``CLEVERAGENTS_TEST_DATABASE_URL`` are set by ``before_scenario``
|
||||
# to pre-migrated scenario databases — removing them forces the
|
||||
# container to derive a fresh DB path from ``CLEVERAGENTS_HOME``.
|
||||
context.tdd_init_saved_env = {
|
||||
k: os.environ.pop(k, None)
|
||||
for k in (
|
||||
"CLEVERAGENTS_AUTO_APPLY_MIGRATIONS",
|
||||
"CI",
|
||||
"BEHAVE_TESTING",
|
||||
"CLEVERAGENTS_DATABASE_URL",
|
||||
"CLEVERAGENTS_TEST_DATABASE_URL",
|
||||
)
|
||||
}
|
||||
|
||||
# Register cleanup to restore env vars and remove tmpdir even if
|
||||
# subsequent steps are never reached (e.g. hook errors).
|
||||
saved_env = context.tdd_init_saved_env
|
||||
tmpdir = context.tdd_init_tmpdir
|
||||
|
||||
def _cleanup() -> None:
|
||||
for key, val in saved_env.items():
|
||||
if val is not None:
|
||||
os.environ[key] = val
|
||||
else:
|
||||
os.environ.pop(key, None)
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
reset_container()
|
||||
|
||||
context.add_cleanup(_cleanup)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I invoke agents init --yes with no stdin")
|
||||
def step_invoke_init_yes_no_stdin(context: Context) -> None:
|
||||
"""Invoke ``agents init --yes`` simulating a TTY with no real input.
|
||||
|
||||
Uses ``CliRunner`` with ``input=""`` to provide no input. Patches
|
||||
``sys.stdin`` on the real ``sys`` module for intent documentation
|
||||
and replaces ``MigrationRunner._default_prompt_for_migration`` with
|
||||
a function that returns ``False`` (simulating a TTY prompt where the
|
||||
user declines). See module docstring for the full mock strategy.
|
||||
"""
|
||||
# Create a mock stdin that reports as a TTY but has no data
|
||||
mock_stdin = MagicMock(spec=sys.stdin)
|
||||
mock_stdin.isatty.return_value = True
|
||||
|
||||
# Reset the global container to ensure a fresh UoW is created
|
||||
# using our env var overrides (no stale CLEVERAGENTS_DATABASE_URL).
|
||||
reset_container()
|
||||
|
||||
# Use a DB path whose filename (``init_yes_test.sqlite``) does
|
||||
# NOT start with the template-fast-path prefixes
|
||||
# (``cleveragents_*``, ``test_*``, ``db.*``) so the real Alembic
|
||||
# migration path runs instead of copying a pre-migrated template.
|
||||
# Do NOT pre-create directories — ``init_command`` expects a
|
||||
# fresh path without an existing ``.cleveragents`` directory.
|
||||
tdd_db_path = os.path.join(
|
||||
context.tdd_init_tmpdir, ".cleveragents", "init_yes_test.sqlite"
|
||||
)
|
||||
tdd_db_url = f"sqlite:///{tdd_db_path}"
|
||||
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CLEVERAGENTS_HOME": context.tdd_init_tmpdir,
|
||||
"CLEVERAGENTS_DATABASE_URL": tdd_db_url,
|
||||
},
|
||||
):
|
||||
# Remove auto-apply env vars inside the patched environment
|
||||
for key in (
|
||||
"CLEVERAGENTS_AUTO_APPLY_MIGRATIONS",
|
||||
"CI",
|
||||
"BEHAVE_TESTING",
|
||||
):
|
||||
os.environ.pop(key, None)
|
||||
|
||||
# Patch sys.stdin on the real sys module (correct target per
|
||||
# project convention) and replace the prompt function to
|
||||
# simulate a TTY decline. See module docstring for why
|
||||
# both patches are needed.
|
||||
with (
|
||||
patch.object(sys, "stdin", mock_stdin),
|
||||
patch.object(
|
||||
MigrationRunner,
|
||||
"_default_prompt_for_migration",
|
||||
staticmethod(_tty_prompt_declines),
|
||||
),
|
||||
):
|
||||
context.tdd_init_result = runner.invoke(
|
||||
project_app,
|
||||
["init", "--yes", "--path", context.tdd_init_tmpdir],
|
||||
input="",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then("the tdd-init-yes-no-input command should exit successfully")
|
||||
def step_exit_success(context: Context) -> None:
|
||||
"""Assert the init command exited with code 0."""
|
||||
result = context.tdd_init_result
|
||||
assert result is not None, "init --yes was not invoked"
|
||||
actual = result.exit_code
|
||||
assert actual == 0, f"Expected exit code 0, got {actual}.\nOutput:\n{result.output}"
|
||||
|
||||
|
||||
@then('the tdd-init-yes-no-input output should not contain "{text}"')
|
||||
def step_output_not_contains(context: Context, text: str) -> None:
|
||||
"""Assert the given text does NOT appear in the command output."""
|
||||
result = context.tdd_init_result
|
||||
assert result is not None, "init --yes was not invoked"
|
||||
output = result.output
|
||||
assert text.lower() not in output.lower(), (
|
||||
f"Did NOT expect '{text}' in output but found it:\n{output}"
|
||||
)
|
||||
|
||||
|
||||
@then('the tdd-init-yes-no-input output should contain "{text}"')
|
||||
def step_output_contains(context: Context, text: str) -> None:
|
||||
"""Assert the given text appears in the command output."""
|
||||
result = context.tdd_init_result
|
||||
assert result is not None, "init --yes was not invoked"
|
||||
output = result.output
|
||||
assert text.lower() in output.lower(), (
|
||||
f"Expected '{text}' in output but did not find it:\n{output}"
|
||||
)
|
||||
@@ -0,0 +1,391 @@
|
||||
"""Step definitions for TDD Bug #967 — plan execute phase processing.
|
||||
|
||||
These steps exercise the **CLI orchestration layer** (the ``execute_plan``
|
||||
command handler in ``plan.py``) via Typer's ``CliRunner`` to verify that
|
||||
the ``plan execute`` command correctly handles plans in Strategize/QUEUED
|
||||
state.
|
||||
|
||||
Bug #967: The ``plan execute`` CLI command originally only called
|
||||
``service.execute_plan(plan_id)``, which is a state transition only
|
||||
(Strategize/COMPLETE → Execute/QUEUED). It did not construct a
|
||||
``PlanExecutor`` or call ``run_strategize()`` / ``run_execute()``. When a
|
||||
plan was in Strategize/QUEUED state (immediately after ``plan use``), the
|
||||
command failed because ``execute_plan()`` requires Strategize/COMPLETE.
|
||||
|
||||
The fix added phase-aware orchestration to the CLI handler: when a plan is
|
||||
in Strategize/QUEUED, it calls ``PlanExecutor.run_strategize()`` before
|
||||
transitioning. The auto-discovery filter was also updated to include
|
||||
Strategize/QUEUED plans.
|
||||
|
||||
Scenarios 1, 2, and 4 exercise the CLI handler (the code path that was
|
||||
fixed). Since the fix is already in the codebase, the ``@tdd_expected_fail``
|
||||
tag has been removed and these tests serve as permanent regression guards.
|
||||
|
||||
The "positive control" scenario (Scenario 3) demonstrates that the proper
|
||||
service-level orchestration path (``run_strategize`` → ``execute_plan``)
|
||||
works correctly.
|
||||
|
||||
This test was written to capture bug #967 per ticket #977.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from cleveragents.application.services.plan_executor import PlanExecutor
|
||||
from cleveragents.application.services.plan_lifecycle_service import (
|
||||
PlanLifecycleService,
|
||||
PlanNotReadyError,
|
||||
)
|
||||
from cleveragents.cli.commands.plan import app as plan_app
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.core.exceptions import PlanError
|
||||
from cleveragents.domain.models.core.plan import (
|
||||
NamespacedName,
|
||||
Plan,
|
||||
PlanIdentity,
|
||||
PlanPhase,
|
||||
PlanTimestamps,
|
||||
ProcessingState,
|
||||
ProjectLink,
|
||||
)
|
||||
|
||||
|
||||
def _make_cli_plan(
|
||||
*,
|
||||
plan_id: str = "01ARZ3NDEKTSV4RRFFQ69G5967",
|
||||
phase: PlanPhase = PlanPhase.STRATEGIZE,
|
||||
state: ProcessingState = ProcessingState.QUEUED,
|
||||
) -> Plan:
|
||||
"""Build a real ``Plan`` domain object for CLI tests.
|
||||
|
||||
Uses the Plan domain model directly (not a MagicMock) so that the CLI
|
||||
handler's ``isinstance`` checks and attribute accesses work correctly.
|
||||
"""
|
||||
return Plan(
|
||||
identity=PlanIdentity(plan_id=plan_id),
|
||||
namespaced_name=NamespacedName(namespace="local", name="tdd-967-plan"),
|
||||
action_name="local/tdd-967-action",
|
||||
description="TDD plan for bug #967",
|
||||
phase=phase,
|
||||
processing_state=state,
|
||||
project_links=[ProjectLink(project_name="proj-1")],
|
||||
strategy_actor="openai/gpt-4",
|
||||
execution_actor="openai/gpt-4",
|
||||
timestamps=PlanTimestamps(created_at=datetime.now(), updated_at=datetime.now()),
|
||||
)
|
||||
|
||||
|
||||
def _create_service_and_queued_plan() -> tuple[PlanLifecycleService, str]:
|
||||
"""Create a real in-memory PlanLifecycleService and a plan in Strategize/QUEUED.
|
||||
|
||||
Returns a tuple of (service, plan_id). The plan is created via the normal
|
||||
``create_action`` → ``use_action`` flow, which puts the plan in
|
||||
Strategize/QUEUED — the state that triggers bug #967.
|
||||
"""
|
||||
settings: Settings = Settings()
|
||||
service: PlanLifecycleService = PlanLifecycleService(settings=settings)
|
||||
action = service.create_action(
|
||||
name="local/tdd-967-action",
|
||||
description="TDD action for bug #967",
|
||||
definition_of_done="Complete the test plan",
|
||||
strategy_actor="local/stub-strategize",
|
||||
execution_actor="local/stub-execute",
|
||||
)
|
||||
plan: Plan = service.use_action(action_name=str(action.namespaced_name))
|
||||
return service, plan.identity.plan_id
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI-level test setup (used by Scenarios 1, 2, 4)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PLAN_ID: str = "01ARZ3NDEKTSV4RRFFQ69G5967"
|
||||
|
||||
|
||||
@given("a CLI runner and mocked services for bug 967")
|
||||
def step_cli_runner_and_mocked_services(context: Context) -> None:
|
||||
"""Set up a CliRunner and mock service/executor for CLI-level testing."""
|
||||
context.runner_967 = CliRunner()
|
||||
context.mock_service_967 = MagicMock()
|
||||
context.mock_executor_967 = MagicMock()
|
||||
|
||||
|
||||
@given("a plan in Strategize/QUEUED state for bug 967")
|
||||
def step_plan_in_strategize_queued(context: Context) -> None:
|
||||
"""Configure the mocked service to return a plan in Strategize/QUEUED.
|
||||
|
||||
The mock ``get_plan`` returns the QUEUED plan for the initial checks,
|
||||
then returns an Execute/QUEUED plan after strategize runs (simulating
|
||||
auto_progress), and finally an Execute/COMPLETE plan after run_execute.
|
||||
"""
|
||||
queued_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.STRATEGIZE,
|
||||
state=ProcessingState.QUEUED,
|
||||
)
|
||||
execute_queued_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.EXECUTE,
|
||||
state=ProcessingState.QUEUED,
|
||||
)
|
||||
execute_complete_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.EXECUTE,
|
||||
state=ProcessingState.COMPLETE,
|
||||
)
|
||||
|
||||
# get_plan call sequence in the CLI execute_plan handler:
|
||||
# 1. pre_plan (read-only check)
|
||||
# 2. current_plan (phase detection)
|
||||
# 3. re-fetch after run_strategize (auto_progress moved to Execute)
|
||||
# 4. re-fetch for inline execute check
|
||||
# 5. re-fetch after run_execute
|
||||
context.mock_service_967.get_plan.side_effect = [
|
||||
queued_plan,
|
||||
queued_plan,
|
||||
execute_queued_plan,
|
||||
execute_queued_plan,
|
||||
execute_complete_plan,
|
||||
]
|
||||
|
||||
context.queued_plan_967 = queued_plan
|
||||
context.execute_complete_plan_967 = execute_complete_plan
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scenario 1: CLI execute command handles plan in Strategize/QUEUED state
|
||||
# Scenario 2: CLI execute command orchestrates full lifecycle for QUEUED plan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I invoke the plan execute CLI command for the QUEUED plan for bug 967")
|
||||
def step_invoke_cli_execute(context: Context) -> None:
|
||||
"""Invoke the ``plan execute`` CLI command via CliRunner.
|
||||
|
||||
This exercises the CLI orchestration layer — the actual code path that
|
||||
was buggy in #967. The handler should detect Strategize/QUEUED and
|
||||
call ``run_strategize`` before transitioning to Execute.
|
||||
"""
|
||||
with (
|
||||
patch(
|
||||
"cleveragents.cli.commands.plan._get_lifecycle_service",
|
||||
return_value=context.mock_service_967,
|
||||
),
|
||||
patch(
|
||||
"cleveragents.cli.commands.plan._get_plan_executor",
|
||||
return_value=context.mock_executor_967,
|
||||
),
|
||||
):
|
||||
context.cli_result_967 = context.runner_967.invoke(
|
||||
plan_app, ["execute", _PLAN_ID]
|
||||
)
|
||||
|
||||
|
||||
@then("the CLI should succeed and the plan should reach Execute phase for bug 967")
|
||||
def step_cli_succeed_execute_phase(context: Context) -> None:
|
||||
"""Assert the CLI command completed without error.
|
||||
|
||||
Bug #967 regression guard: if the CLI orchestration is broken, this
|
||||
assertion fails because the command would abort with an error.
|
||||
"""
|
||||
result = context.cli_result_967
|
||||
assert result.exit_code == 0, (
|
||||
f"Bug #967: CLI 'plan execute' failed with exit code {result.exit_code}. "
|
||||
f"The CLI should handle Strategize/QUEUED plans by running "
|
||||
f"run_strategize() before transitioning. Output:\n{result.output}"
|
||||
)
|
||||
assert "Execute" in result.output or "execute" in result.output, (
|
||||
f"Bug #967: CLI output does not mention Execute phase. Output:\n{result.output}"
|
||||
)
|
||||
|
||||
|
||||
@then("the executor should have run strategize for the plan for bug 967")
|
||||
def step_executor_ran_strategize(context: Context) -> None:
|
||||
"""Assert the CLI handler invoked run_strategize on the executor.
|
||||
|
||||
Bug #967 regression guard: the CLI handler must call
|
||||
``executor.run_strategize(plan_id)`` for plans in Strategize/QUEUED.
|
||||
"""
|
||||
context.mock_executor_967.run_strategize.assert_called_once_with(_PLAN_ID)
|
||||
|
||||
|
||||
@then("the plan should have completed execute phase processing via CLI for bug 967")
|
||||
def step_plan_completed_execute_via_cli(context: Context) -> None:
|
||||
"""Assert the CLI handler also ran the execute phase.
|
||||
|
||||
Bug #967 regression guard: after strategize, the CLI should run
|
||||
``executor.run_execute(plan_id)`` to complete the Execute phase.
|
||||
"""
|
||||
context.mock_executor_967.run_execute.assert_called_once_with(_PLAN_ID)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scenario 3: Positive control — proper orchestration works
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a real plan executor with a plan in Strategize/QUEUED for bug 967")
|
||||
def step_real_executor_queued_plan(context: Context) -> None:
|
||||
"""Set up a real PlanExecutor and PlanLifecycleService with a QUEUED plan."""
|
||||
service, plan_id = _create_service_and_queued_plan()
|
||||
context.service_967 = service
|
||||
context.plan_id_967 = plan_id
|
||||
context.executor_967 = PlanExecutor(lifecycle_service=service)
|
||||
|
||||
|
||||
@when("I run the proper orchestration of strategize then execute for bug 967")
|
||||
def step_proper_orchestration(context: Context) -> None:
|
||||
"""Run the correct orchestration: run_strategize → execute_plan → verify.
|
||||
|
||||
This positive control demonstrates that when the caller properly
|
||||
orchestrates the lifecycle (as the fixed CLI does), a QUEUED plan
|
||||
can be taken through to Execute phase.
|
||||
"""
|
||||
service: PlanLifecycleService = context.service_967
|
||||
executor: PlanExecutor = context.executor_967
|
||||
plan_id: str = context.plan_id_967
|
||||
context.orchestration_error_967 = None
|
||||
context.orchestrated_plan_967 = None
|
||||
|
||||
try:
|
||||
# Step 1: Run strategize to completion (QUEUED → PROCESSING → COMPLETE)
|
||||
executor.run_strategize(plan_id)
|
||||
|
||||
# Step 2: Re-fetch the plan — auto_progress in complete_strategize
|
||||
# may have already advanced the plan to Execute phase.
|
||||
plan: Plan = service.get_plan(plan_id)
|
||||
|
||||
if plan.phase == PlanPhase.EXECUTE:
|
||||
# auto_progress already moved it
|
||||
context.orchestrated_plan_967 = plan
|
||||
elif (
|
||||
plan.phase == PlanPhase.STRATEGIZE
|
||||
and plan.state == ProcessingState.COMPLETE
|
||||
):
|
||||
# Transition to Execute phase
|
||||
context.orchestrated_plan_967 = service.execute_plan(plan_id)
|
||||
else:
|
||||
context.orchestration_error_967 = PlanError(
|
||||
f"Unexpected plan state after strategize: "
|
||||
f"{plan.phase.value}/{plan.state.value}"
|
||||
)
|
||||
except (PlanError, PlanNotReadyError) as exc:
|
||||
context.orchestration_error_967 = exc
|
||||
|
||||
|
||||
@then("the plan should be in Execute/QUEUED state via orchestration for bug 967")
|
||||
def step_plan_in_execute_via_orchestration(context: Context) -> None:
|
||||
"""Assert the properly orchestrated plan reached Execute phase with QUEUED state.
|
||||
|
||||
This is a positive control: when the orchestration is done correctly,
|
||||
the plan should be in Execute phase. This scenario is NOT tagged
|
||||
``@tdd_expected_fail`` and must always pass.
|
||||
"""
|
||||
if context.orchestration_error_967 is not None:
|
||||
raise AssertionError(
|
||||
f"Proper orchestration failed unexpectedly: "
|
||||
f"{context.orchestration_error_967}"
|
||||
)
|
||||
plan: Plan | None = context.orchestrated_plan_967
|
||||
assert plan is not None, "Orchestration returned no plan"
|
||||
assert plan.phase == PlanPhase.EXECUTE, (
|
||||
f"Expected Execute phase, got {plan.phase.value}"
|
||||
)
|
||||
assert plan.state == ProcessingState.QUEUED, (
|
||||
f"Expected QUEUED state, got {plan.state.value}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scenario 4: CLI auto-discovery finds plans in Strategize/QUEUED state
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given(
|
||||
"a single plan in Strategize/QUEUED state eligible for auto-discovery for bug 967"
|
||||
)
|
||||
def step_single_queued_plan_auto_discovery(context: Context) -> None:
|
||||
"""Configure the mocked service for auto-discovery with a QUEUED plan.
|
||||
|
||||
The CLI auto-discovery (no plan_id) calls ``service.list_plans()`` and
|
||||
filters for eligible plans. Bug #967 had a filter that only accepted
|
||||
Strategize/COMPLETE, missing QUEUED plans. The fix includes QUEUED.
|
||||
"""
|
||||
queued_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.STRATEGIZE,
|
||||
state=ProcessingState.QUEUED,
|
||||
)
|
||||
execute_queued_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.EXECUTE,
|
||||
state=ProcessingState.QUEUED,
|
||||
)
|
||||
execute_complete_plan: Plan = _make_cli_plan(
|
||||
plan_id=_PLAN_ID,
|
||||
phase=PlanPhase.EXECUTE,
|
||||
state=ProcessingState.COMPLETE,
|
||||
)
|
||||
|
||||
# list_plans is called twice: once for STRATEGIZE, once for EXECUTE
|
||||
context.mock_service_967.list_plans.side_effect = [
|
||||
[queued_plan], # Strategize phase plans
|
||||
[], # Execute phase plans (none)
|
||||
]
|
||||
|
||||
# get_plan call sequence after auto-discovery selects the plan:
|
||||
# 1. pre_plan (read-only check)
|
||||
# 2. current_plan (phase detection)
|
||||
# 3. re-fetch after run_strategize
|
||||
# 4. re-fetch for inline execute check
|
||||
# 5. re-fetch after run_execute
|
||||
context.mock_service_967.get_plan.side_effect = [
|
||||
queued_plan,
|
||||
queued_plan,
|
||||
execute_queued_plan,
|
||||
execute_queued_plan,
|
||||
execute_complete_plan,
|
||||
]
|
||||
|
||||
|
||||
@when("I invoke the plan execute CLI command without a plan id for bug 967")
|
||||
def step_invoke_cli_execute_no_plan_id(context: Context) -> None:
|
||||
"""Invoke ``plan execute`` without a plan ID to trigger auto-discovery.
|
||||
|
||||
Bug #967 regression guard: the CLI auto-discovery filter must include
|
||||
Strategize/QUEUED plans, not just Strategize/COMPLETE.
|
||||
"""
|
||||
with (
|
||||
patch(
|
||||
"cleveragents.cli.commands.plan._get_lifecycle_service",
|
||||
return_value=context.mock_service_967,
|
||||
),
|
||||
patch(
|
||||
"cleveragents.cli.commands.plan._get_plan_executor",
|
||||
return_value=context.mock_executor_967,
|
||||
),
|
||||
):
|
||||
context.cli_result_967 = context.runner_967.invoke(plan_app, ["execute"])
|
||||
|
||||
|
||||
@then("the CLI should succeed and auto-discover the QUEUED plan for bug 967")
|
||||
def step_cli_auto_discovery_succeeds(context: Context) -> None:
|
||||
"""Assert the CLI auto-discovered and processed the Strategize/QUEUED plan.
|
||||
|
||||
Bug #967 regression guard: if the auto-discovery filter excludes QUEUED
|
||||
plans, the CLI would abort with "No plans ready for execution."
|
||||
"""
|
||||
result = context.cli_result_967
|
||||
assert result.exit_code == 0, (
|
||||
"Bug #967: CLI 'plan execute' (no plan_id) failed with exit code "
|
||||
f"{result.exit_code}. The auto-discovery filter should include "
|
||||
f"Strategize/QUEUED plans. Output:\n{result.output}"
|
||||
)
|
||||
# Verify the executor was actually invoked (plan was found and processed)
|
||||
context.mock_executor_967.run_strategize.assert_called_once_with(_PLAN_ID)
|
||||
@@ -0,0 +1,28 @@
|
||||
@tdd_expected_fail @tdd_bug @tdd_bug_783
|
||||
Feature: TDD Bug #783 — agents init --yes should not require user input
|
||||
As a developer
|
||||
I want to verify that `agents init --yes` completes without any user
|
||||
input prompts
|
||||
So that the bug is captured and will be caught by a regression test
|
||||
|
||||
The root cause is that `init_command()` in project.py receives the
|
||||
`--yes` flag but does not pass it through to the migration runner.
|
||||
The migration runner's `_default_prompt_for_migration` uses
|
||||
`typer.confirm()` when a TTY is detected, causing the command to block
|
||||
for user input even when `--yes` was specified.
|
||||
|
||||
Scenario: Init with --yes completes without stdin
|
||||
Given a fresh environment for tdd-init-yes-no-input
|
||||
When I invoke agents init --yes with no stdin
|
||||
Then the tdd-init-yes-no-input command should exit successfully
|
||||
And the tdd-init-yes-no-input output should not contain "Apply migrations now"
|
||||
|
||||
# Note: While the bug is present, the first assertion (exit code 0) will
|
||||
# fail and Behave will skip subsequent steps. This means the "contains
|
||||
# Initialized" assertion is only evaluated after the bug is fixed, at
|
||||
# which point this scenario provides distinct post-fix regression value.
|
||||
Scenario: Init with --yes does not prompt for migration approval
|
||||
Given a fresh environment for tdd-init-yes-no-input
|
||||
When I invoke agents init --yes with no stdin
|
||||
Then the tdd-init-yes-no-input command should exit successfully
|
||||
And the tdd-init-yes-no-input output should contain "Initialized"
|
||||
@@ -0,0 +1,42 @@
|
||||
@tdd_bug @tdd_bug_967 @mock_only
|
||||
Feature: TDD Bug #967 — plan execute only transitions state, does not run strategize or execute phase processing
|
||||
As a developer
|
||||
I want to verify that the plan execute CLI command handles plans in Strategize/QUEUED state
|
||||
by running strategize phase processing before transitioning to Execute
|
||||
So that the bug is captured and will be caught by a regression test
|
||||
|
||||
Bug #967: The plan execute CLI command originally only called
|
||||
service.execute_plan(plan_id), which is a state transition only
|
||||
(Strategize/COMPLETE → Execute/QUEUED). It did not construct a
|
||||
PlanExecutor or call run_strategize() / run_execute(). When a plan
|
||||
was in Strategize/QUEUED state (immediately after plan use), the command
|
||||
failed because execute_plan() requires Strategize/COMPLETE.
|
||||
|
||||
These tests exercise the CLI orchestration layer (the execute_plan command
|
||||
handler in plan.py) via CliRunner to verify the bug is fixed. The fix
|
||||
added phase-aware orchestration: when a plan is in Strategize/QUEUED,
|
||||
the CLI runs PlanExecutor.run_strategize() before transitioning.
|
||||
|
||||
Scenario: CLI execute command handles plan in Strategize/QUEUED state
|
||||
Given a CLI runner and mocked services for bug 967
|
||||
And a plan in Strategize/QUEUED state for bug 967
|
||||
When I invoke the plan execute CLI command for the QUEUED plan for bug 967
|
||||
Then the CLI should succeed and the plan should reach Execute phase for bug 967
|
||||
|
||||
Scenario: CLI execute command orchestrates full lifecycle for QUEUED plan
|
||||
Given a CLI runner and mocked services for bug 967
|
||||
And a plan in Strategize/QUEUED state for bug 967
|
||||
When I invoke the plan execute CLI command for the QUEUED plan for bug 967
|
||||
Then the executor should have run strategize for the plan for bug 967
|
||||
And the plan should have completed execute phase processing via CLI for bug 967
|
||||
|
||||
Scenario: Positive control — proper orchestration transitions QUEUED plan to Execute
|
||||
Given a real plan executor with a plan in Strategize/QUEUED for bug 967
|
||||
When I run the proper orchestration of strategize then execute for bug 967
|
||||
Then the plan should be in Execute/QUEUED state via orchestration for bug 967
|
||||
|
||||
Scenario: CLI auto-discovery finds plans in Strategize/QUEUED state
|
||||
Given a CLI runner and mocked services for bug 967
|
||||
And a single plan in Strategize/QUEUED state eligible for auto-discovery for bug 967
|
||||
When I invoke the plan execute CLI command without a plan id for bug 967
|
||||
Then the CLI should succeed and auto-discover the QUEUED plan for bug 967
|
||||
@@ -0,0 +1,242 @@
|
||||
"""Helper script for tdd_init_yes_no_input.robot smoke tests.
|
||||
|
||||
Each subcommand exercises the real ``agents init --yes`` code path
|
||||
simulating a TTY environment with no real stdin input to reproduce
|
||||
bug #783. The helper reports the **real** outcome: it exits 0 and
|
||||
prints the sentinel when the operation succeeds (bug is fixed), and
|
||||
exits 1 when the bug is still present. The
|
||||
``tdd_expected_fail_listener`` on the Robot side handles pass/fail
|
||||
inversion while the bug remains open.
|
||||
|
||||
Mock strategy
|
||||
~~~~~~~~~~~~~
|
||||
CliRunner replaces ``sys.stdin`` during ``invoke()``, so patching
|
||||
``sys.stdin.isatty()`` alone is ineffective. Instead we:
|
||||
|
||||
1. Patch ``sys.stdin`` on the real ``sys`` module (documents intent
|
||||
and is the correct mock target per the project convention).
|
||||
2. Patch ``MigrationRunner._default_prompt_for_migration`` with a
|
||||
replacement that returns ``False`` — simulating a TTY prompt where
|
||||
the user declines migration.
|
||||
3. Set ``CLEVERAGENTS_DATABASE_URL`` to a path whose filename does
|
||||
**not** match the ``before_scenario`` template-DB prefixes so the
|
||||
real Alembic migration path runs.
|
||||
|
||||
With bug #783, ``require_confirmation=True`` is hardcoded, the prompt
|
||||
fires and returns ``False``, causing ``MigrationNotApprovedError``.
|
||||
After the fix, ``--yes`` should bypass the prompt entirely.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
# Ensure local source tree is importable
|
||||
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||
if _SRC not in sys.path:
|
||||
sys.path.insert(0, _SRC)
|
||||
|
||||
from typer.testing import CliRunner # noqa: E402
|
||||
|
||||
from cleveragents.cli.commands.project import app as project_app # noqa: E402
|
||||
from cleveragents.infrastructure.database.migration_runner import ( # noqa: E402
|
||||
MigrationRunner,
|
||||
)
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def _tty_prompt_declines(_message: str) -> bool:
|
||||
"""Simulate a TTY migration prompt that declines (no user input).
|
||||
|
||||
Replaces ``_default_prompt_for_migration`` to reproduce the TTY
|
||||
code path without depending on ``sys.stdin.isatty()``, which
|
||||
CliRunner overrides during ``invoke()``.
|
||||
"""
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Subcommands
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def init_no_stdin() -> None:
|
||||
"""Invoke ``agents init --yes`` simulating a TTY with no real input.
|
||||
|
||||
Patches ``sys.stdin`` on the real ``sys`` module for intent
|
||||
documentation and replaces ``_default_prompt_for_migration`` with
|
||||
a function that returns ``False`` (simulating a TTY prompt that
|
||||
declines). See module docstring for the full mock strategy.
|
||||
|
||||
Exits 0 with sentinel when the command completes successfully
|
||||
(bug is fixed). Exits 1 when the command fails (bug still
|
||||
present).
|
||||
"""
|
||||
tmpdir = tempfile.mkdtemp(prefix="tdd_init_yes_robot_")
|
||||
try:
|
||||
# Remove env vars that would bypass the prompt
|
||||
env_save: dict[str, str | None] = {}
|
||||
for key in (
|
||||
"CLEVERAGENTS_AUTO_APPLY_MIGRATIONS",
|
||||
"CI",
|
||||
"BEHAVE_TESTING",
|
||||
"CLEVERAGENTS_HOME",
|
||||
"CLEVERAGENTS_DATABASE_URL",
|
||||
"CLEVERAGENTS_TEST_DATABASE_URL",
|
||||
):
|
||||
env_save[key] = os.environ.pop(key, None)
|
||||
|
||||
os.environ["CLEVERAGENTS_HOME"] = tmpdir
|
||||
|
||||
# Use a DB path whose filename does NOT start with the
|
||||
# template-fast-path prefixes (``cleveragents_*``, ``test_*``,
|
||||
# ``db.*``) so the real Alembic migration path runs.
|
||||
tdd_db_path = os.path.join(tmpdir, ".cleveragents", "init_yes_test.sqlite")
|
||||
os.environ["CLEVERAGENTS_DATABASE_URL"] = f"sqlite:///{tdd_db_path}"
|
||||
|
||||
# Create a mock stdin that reports as a TTY
|
||||
mock_stdin = MagicMock(spec=sys.stdin)
|
||||
mock_stdin.isatty.return_value = True
|
||||
|
||||
# Patch sys.stdin on the real sys module and replace the
|
||||
# prompt function to simulate a TTY decline.
|
||||
with (
|
||||
patch.object(sys, "stdin", mock_stdin),
|
||||
patch.object(
|
||||
MigrationRunner,
|
||||
"_default_prompt_for_migration",
|
||||
staticmethod(_tty_prompt_declines),
|
||||
),
|
||||
):
|
||||
result = runner.invoke(
|
||||
project_app,
|
||||
["init", "--yes", "--path", tmpdir],
|
||||
input="",
|
||||
)
|
||||
|
||||
# Restore env vars
|
||||
for key, val in env_save.items():
|
||||
if val is not None:
|
||||
os.environ[key] = val
|
||||
else:
|
||||
os.environ.pop(key, None)
|
||||
|
||||
if result.exit_code != 0:
|
||||
print(
|
||||
f"init --yes failed with exit code {result.exit_code}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(f"Output: {result.output}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Bug fixed — command succeeded without input.
|
||||
print("tdd-init-yes-no-input-ok")
|
||||
finally:
|
||||
# Clean up tmpdir
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
def init_no_prompt() -> None:
|
||||
"""Invoke ``agents init --yes`` and verify no migration prompt appears.
|
||||
|
||||
Uses the same mock strategy as ``init_no_stdin``. See module
|
||||
docstring for details.
|
||||
|
||||
Exits 0 with sentinel when the output does not contain the
|
||||
migration prompt text (bug is fixed). Exits 1 when the prompt
|
||||
text appears or the command fails (bug still present).
|
||||
"""
|
||||
tmpdir = tempfile.mkdtemp(prefix="tdd_init_yes_robot_")
|
||||
try:
|
||||
# Remove env vars that would bypass the prompt
|
||||
env_save: dict[str, str | None] = {}
|
||||
for key in (
|
||||
"CLEVERAGENTS_AUTO_APPLY_MIGRATIONS",
|
||||
"CI",
|
||||
"BEHAVE_TESTING",
|
||||
"CLEVERAGENTS_HOME",
|
||||
"CLEVERAGENTS_DATABASE_URL",
|
||||
"CLEVERAGENTS_TEST_DATABASE_URL",
|
||||
):
|
||||
env_save[key] = os.environ.pop(key, None)
|
||||
|
||||
os.environ["CLEVERAGENTS_HOME"] = tmpdir
|
||||
|
||||
# Use a DB path whose filename does NOT start with the
|
||||
# template-fast-path prefixes.
|
||||
tdd_db_path = os.path.join(tmpdir, ".cleveragents", "init_yes_test.sqlite")
|
||||
os.environ["CLEVERAGENTS_DATABASE_URL"] = f"sqlite:///{tdd_db_path}"
|
||||
|
||||
# Create a mock stdin that reports as a TTY
|
||||
mock_stdin = MagicMock(spec=sys.stdin)
|
||||
mock_stdin.isatty.return_value = True
|
||||
|
||||
# Patch sys.stdin on the real sys module and replace the
|
||||
# prompt function to simulate a TTY decline.
|
||||
with (
|
||||
patch.object(sys, "stdin", mock_stdin),
|
||||
patch.object(
|
||||
MigrationRunner,
|
||||
"_default_prompt_for_migration",
|
||||
staticmethod(_tty_prompt_declines),
|
||||
),
|
||||
):
|
||||
result = runner.invoke(
|
||||
project_app,
|
||||
["init", "--yes", "--path", tmpdir],
|
||||
input="",
|
||||
)
|
||||
|
||||
# Restore env vars
|
||||
for key, val in env_save.items():
|
||||
if val is not None:
|
||||
os.environ[key] = val
|
||||
else:
|
||||
os.environ.pop(key, None)
|
||||
|
||||
if result.exit_code != 0:
|
||||
print(
|
||||
f"init --yes failed with exit code {result.exit_code}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(f"Output: {result.output}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Check that no migration prompt text appears
|
||||
output = result.output.lower()
|
||||
if "apply migrations now" in output:
|
||||
print(
|
||||
"Bug present: migration prompt appeared in output",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Bug fixed — no prompt appeared.
|
||||
print("tdd-init-yes-no-prompt-ok")
|
||||
finally:
|
||||
# Clean up tmpdir
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dispatcher
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_COMMANDS: dict[str, Callable[[], None]] = {
|
||||
"init-no-stdin": init_no_stdin,
|
||||
"init-no-prompt": init_no_prompt,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
|
||||
print(f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
cmd = _COMMANDS[sys.argv[1]]
|
||||
cmd()
|
||||
@@ -0,0 +1,217 @@
|
||||
"""Helper script for tdd_plan_execute_phase_processing.robot smoke tests.
|
||||
|
||||
Each subcommand exercises the **CLI orchestration layer** by replicating the
|
||||
logic of the ``execute_plan`` CLI handler in ``plan.py``. The helper uses
|
||||
a real ``PlanLifecycleService`` (in-memory mode) and ``PlanExecutor`` (with
|
||||
stub actors) to verify that the orchestration correctly handles plans in
|
||||
Strategize/QUEUED state.
|
||||
|
||||
Bug #967: The ``plan execute`` CLI command originally only called
|
||||
``service.execute_plan(plan_id)``, which is a state transition only
|
||||
(Strategize/COMPLETE → Execute/QUEUED). It did not construct a
|
||||
``PlanExecutor`` or call ``run_strategize()`` / ``run_execute()``. The fix
|
||||
added phase-aware orchestration; these tests verify it works.
|
||||
|
||||
This test was written to capture bug #967 per ticket #977. The bug fix is
|
||||
already in the codebase, so these tests serve as permanent regression guards.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import NoReturn
|
||||
|
||||
_ROOT: Path = Path(__file__).resolve().parents[1]
|
||||
_SRC: str = str(_ROOT / "src")
|
||||
if _SRC not in sys.path:
|
||||
sys.path.insert(0, _SRC)
|
||||
|
||||
from cleveragents.application.services.plan_executor import PlanExecutor # noqa: E402
|
||||
from cleveragents.application.services.plan_lifecycle_service import ( # noqa: E402
|
||||
PlanLifecycleService,
|
||||
)
|
||||
from cleveragents.config.settings import Settings # noqa: E402
|
||||
from cleveragents.domain.models.core.plan import ( # noqa: E402
|
||||
Plan,
|
||||
PlanPhase,
|
||||
ProcessingState,
|
||||
)
|
||||
|
||||
|
||||
def _fail(msg: str) -> NoReturn:
|
||||
"""Print an error message to stderr and exit with code 1."""
|
||||
print(msg, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _create_service_and_queued_plan() -> tuple[PlanLifecycleService, str]:
|
||||
"""Create a real in-memory PlanLifecycleService and a plan in Strategize/QUEUED."""
|
||||
service: PlanLifecycleService = PlanLifecycleService(settings=Settings())
|
||||
action = service.create_action(
|
||||
name="local/tdd-967-robot-action",
|
||||
description="TDD action for bug #967 (Robot)",
|
||||
definition_of_done="Complete the test plan",
|
||||
strategy_actor="local/stub-strategize",
|
||||
execution_actor="local/stub-execute",
|
||||
)
|
||||
plan: Plan = service.use_action(action_name=str(action.namespaced_name))
|
||||
return service, plan.identity.plan_id
|
||||
|
||||
|
||||
def _cli_execute_from_queued() -> None:
|
||||
"""Test the CLI orchestration path for a plan in Strategize/QUEUED.
|
||||
|
||||
Replicates the ``execute_plan`` CLI handler logic: detect the plan is
|
||||
in Strategize/QUEUED, run strategize via ``PlanExecutor``, then
|
||||
transition to Execute phase.
|
||||
"""
|
||||
service, plan_id = _create_service_and_queued_plan()
|
||||
executor: PlanExecutor = PlanExecutor(lifecycle_service=service)
|
||||
|
||||
# Replicate CLI orchestration: detect phase, run strategize if QUEUED
|
||||
plan: Plan = service.get_plan(plan_id)
|
||||
if plan.phase == PlanPhase.STRATEGIZE and plan.state == ProcessingState.QUEUED:
|
||||
executor.run_strategize(plan_id)
|
||||
plan = service.get_plan(plan_id)
|
||||
|
||||
# Transition to Execute if still in Strategize/COMPLETE
|
||||
if plan.phase == PlanPhase.STRATEGIZE and plan.state == ProcessingState.COMPLETE:
|
||||
plan = service.execute_plan(plan_id)
|
||||
elif plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(
|
||||
f"Bug #967: CLI orchestration failed. Plan in unexpected state "
|
||||
f"after strategize: {plan.phase.value}/{plan.state.value}"
|
||||
)
|
||||
|
||||
if plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(f"Bug #967: Expected Execute phase, got {plan.phase.value}")
|
||||
print("tdd-cli-execute-from-queued-ok")
|
||||
|
||||
|
||||
def _cli_full_orchestration() -> None:
|
||||
"""Test the CLI orchestration runs both strategize and execute phases.
|
||||
|
||||
Replicates the full ``execute_plan`` CLI handler: strategize → transition
|
||||
→ run_execute.
|
||||
"""
|
||||
service, plan_id = _create_service_and_queued_plan()
|
||||
executor: PlanExecutor = PlanExecutor(lifecycle_service=service)
|
||||
|
||||
# Step 1: CLI detects Strategize/QUEUED and runs strategize
|
||||
plan: Plan = service.get_plan(plan_id)
|
||||
if plan.phase == PlanPhase.STRATEGIZE and plan.state in (
|
||||
ProcessingState.QUEUED,
|
||||
ProcessingState.PROCESSING,
|
||||
):
|
||||
executor.run_strategize(plan_id)
|
||||
plan = service.get_plan(plan_id)
|
||||
|
||||
# Step 2: Transition to Execute if needed
|
||||
if plan.phase == PlanPhase.STRATEGIZE and plan.state == ProcessingState.COMPLETE:
|
||||
plan = service.execute_plan(plan_id)
|
||||
elif plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(
|
||||
f"Bug #967: Unexpected state after strategize: "
|
||||
f"{plan.phase.value}/{plan.state.value}"
|
||||
)
|
||||
|
||||
# Step 3: Run execute phase
|
||||
plan = service.get_plan(plan_id)
|
||||
if plan.phase == PlanPhase.EXECUTE and plan.state == ProcessingState.QUEUED:
|
||||
executor.run_execute(plan_id)
|
||||
plan = service.get_plan(plan_id)
|
||||
|
||||
if plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(f"Bug #967: Expected Execute phase, got {plan.phase.value}")
|
||||
if plan.state not in (ProcessingState.COMPLETE, ProcessingState.QUEUED):
|
||||
_fail(f"Bug #967: Expected COMPLETE or QUEUED state, got {plan.state.value}")
|
||||
print("tdd-cli-full-orchestration-ok")
|
||||
|
||||
|
||||
def _proper_orchestration() -> None:
|
||||
"""Positive control: proper service-level orchestration works on a QUEUED plan."""
|
||||
service, plan_id = _create_service_and_queued_plan()
|
||||
executor: PlanExecutor = PlanExecutor(lifecycle_service=service)
|
||||
|
||||
# Run strategize first (QUEUED → PROCESSING → COMPLETE)
|
||||
executor.run_strategize(plan_id)
|
||||
|
||||
# Re-fetch — auto_progress may have moved it to Execute
|
||||
plan: Plan = service.get_plan(plan_id)
|
||||
if plan.phase == PlanPhase.STRATEGIZE and plan.state == ProcessingState.COMPLETE:
|
||||
plan = service.execute_plan(plan_id)
|
||||
elif plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(
|
||||
f"Bug #967: Unexpected state after orchestration: "
|
||||
f"{plan.phase.value}/{plan.state.value}"
|
||||
)
|
||||
|
||||
plan = service.get_plan(plan_id)
|
||||
if plan.phase != PlanPhase.EXECUTE:
|
||||
_fail(f"Bug #967: Expected Execute phase, got {plan.phase.value}")
|
||||
if plan.state != ProcessingState.QUEUED:
|
||||
_fail(f"Bug #967: Expected QUEUED state, got {plan.state.value}")
|
||||
print("tdd-proper-orchestration-ok")
|
||||
|
||||
|
||||
def _cli_auto_discovery() -> None:
|
||||
"""Test the CLI auto-discovery filter includes Strategize/QUEUED plans.
|
||||
|
||||
Replicates the CLI auto-discovery logic from the ``execute_plan`` handler:
|
||||
list plans and filter for eligible states including QUEUED.
|
||||
"""
|
||||
service: PlanLifecycleService = PlanLifecycleService(settings=Settings())
|
||||
|
||||
# Create an action
|
||||
action = service.create_action(
|
||||
name="local/tdd-967-discovery",
|
||||
description="TDD action for auto-discovery (Robot)",
|
||||
definition_of_done="Test auto-discovery",
|
||||
strategy_actor="local/stub-strategize",
|
||||
execution_actor="local/stub-execute",
|
||||
)
|
||||
action_name: str = str(action.namespaced_name)
|
||||
|
||||
# Plan 1: Strategize/QUEUED (fresh from use_action)
|
||||
plan_queued: Plan = service.use_action(action_name=action_name)
|
||||
queued_id: str = plan_queued.identity.plan_id
|
||||
|
||||
# Apply the PRODUCTION auto-discovery filter (the fixed version):
|
||||
# includes both QUEUED and COMPLETE Strategize plans
|
||||
plans_strat: list[Plan] = service.list_plans(phase=PlanPhase.STRATEGIZE)
|
||||
plans_exec: list[Plan] = service.list_plans(phase=PlanPhase.EXECUTE)
|
||||
eligible: list[Plan] = [
|
||||
p
|
||||
for p in plans_strat
|
||||
if p.state in (ProcessingState.QUEUED, ProcessingState.COMPLETE)
|
||||
] + [p for p in plans_exec if p.state == ProcessingState.QUEUED]
|
||||
|
||||
queued_in_eligible: list[Plan] = [
|
||||
p for p in eligible if p.identity.plan_id == queued_id
|
||||
]
|
||||
if len(queued_in_eligible) == 0:
|
||||
_fail(
|
||||
"Bug #967: Auto-discovery filter does not find Strategize/QUEUED "
|
||||
f"plans. Eligible count: {len(eligible)}, QUEUED plan: {queued_id}"
|
||||
)
|
||||
print("tdd-cli-auto-discovery-ok")
|
||||
|
||||
|
||||
_COMMANDS: dict[str, Callable[[], None]] = {
|
||||
"cli-execute-from-queued": _cli_execute_from_queued,
|
||||
"cli-full-orchestration": _cli_full_orchestration,
|
||||
"proper-orchestration": _proper_orchestration,
|
||||
"cli-auto-discovery": _cli_auto_discovery,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in _COMMANDS:
|
||||
print(
|
||||
f"Usage: {sys.argv[0]} <{'|'.join(_COMMANDS)}>",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
cmd: Callable[[], None] = _COMMANDS[sys.argv[1]]
|
||||
cmd()
|
||||
@@ -0,0 +1,33 @@
|
||||
*** Settings ***
|
||||
Documentation TDD Bug #783 — agents init --yes should not require user input
|
||||
... Integration smoke tests verifying that the ``init --yes`` command
|
||||
... completes without blocking for user input. Bug #783 reports that
|
||||
... the migration runner still prompts for approval even with ``--yes``.
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment auto_apply_migrations=${FALSE}
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Variables ***
|
||||
${HELPER} ${CURDIR}/helper_tdd_init_yes_no_input.py
|
||||
|
||||
*** Test Cases ***
|
||||
TDD Init Yes No Input Completes Without Stdin
|
||||
[Documentation] Verify that ``init --yes`` exits successfully without any
|
||||
... stdin input. The helper runs the real CLI init command in
|
||||
... a fresh directory with stdin closed.
|
||||
[Tags] tdd_bug tdd_bug_783 tdd_expected_fail
|
||||
${result}= Run Process ${PYTHON} ${HELPER} init-no-stdin cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-init-yes-no-input-ok
|
||||
|
||||
TDD Init Yes No Migration Prompt
|
||||
[Documentation] Verify that ``init --yes`` output does not contain the
|
||||
... migration approval prompt text.
|
||||
[Tags] tdd_bug tdd_bug_783 tdd_expected_fail
|
||||
${result}= Run Process ${PYTHON} ${HELPER} init-no-prompt cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-init-yes-no-prompt-ok
|
||||
@@ -0,0 +1,60 @@
|
||||
*** Settings ***
|
||||
Documentation TDD Bug #967 — plan execute only transitions state, does not run
|
||||
... strategize or execute phase processing. Integration smoke tests
|
||||
... exercising the CLI orchestration layer via real
|
||||
... PlanLifecycleService (in-memory) and PlanExecutor (stub actors)
|
||||
... to verify that a plan in Strategize/QUEUED state can be taken
|
||||
... through to Execute phase. The bug fix is already in the
|
||||
... codebase; these tests serve as permanent regression guards.
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment With Database Isolation
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Variables ***
|
||||
${HELPER} ${CURDIR}/helper_tdd_plan_execute_phase_processing.py
|
||||
|
||||
*** Test Cases ***
|
||||
TDD CLI Execute Plan From Strategize QUEUED State
|
||||
[Documentation] Verify that the CLI execute_plan handler correctly
|
||||
... orchestrates run_strategize before transitioning when
|
||||
... a plan is in Strategize/QUEUED state.
|
||||
... Bug #967: originally the CLI only called service.execute_plan().
|
||||
[Tags] tdd_bug tdd_bug_967
|
||||
${result}= Run Process ${PYTHON} ${HELPER} cli-execute-from-queued cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-cli-execute-from-queued-ok
|
||||
|
||||
TDD CLI Execute Orchestrates Full Lifecycle
|
||||
[Documentation] Verify that the CLI execute_plan handler calls both
|
||||
... run_strategize and run_execute for a QUEUED plan.
|
||||
... Bug #967: the CLI skipped PlanExecutor orchestration.
|
||||
[Tags] tdd_bug tdd_bug_967
|
||||
${result}= Run Process ${PYTHON} ${HELPER} cli-full-orchestration cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-cli-full-orchestration-ok
|
||||
|
||||
TDD Positive Control Proper Orchestration
|
||||
[Documentation] Positive control: verify that the proper service-level
|
||||
... orchestration path (run_strategize then execute_plan)
|
||||
... works correctly.
|
||||
[Tags] tdd_bug tdd_bug_967
|
||||
${result}= Run Process ${PYTHON} ${HELPER} proper-orchestration cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-proper-orchestration-ok
|
||||
|
||||
TDD CLI Auto Discovery Includes QUEUED Plans
|
||||
[Documentation] Verify that the CLI auto-discovery filter includes plans
|
||||
... in Strategize/QUEUED state, not just Strategize/COMPLETE.
|
||||
... Bug #967: the original filter excluded QUEUED plans.
|
||||
[Tags] tdd_bug tdd_bug_967
|
||||
${result}= Run Process ${PYTHON} ${HELPER} cli-auto-discovery cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} tdd-cli-auto-discovery-ok
|
||||
Reference in New Issue
Block a user