Files
temp/features/steps/actor_run_signature_cli_steps.py
CoreRasurae e4c01492d5 refactor(cli): align actor run signature with spec positional args
Aligned the `agents actor run` command signature with the
specification by introducing positional NAME and PROMPT arguments.
The --config/-c option is preserved as an optional fallback for
direct YAML invocation. When NAME is provided without --config,
the actor is resolved from the Actor Registry.

Updated both actor_run.py and actor.py run commands. Added backward
compatibility: if --config is provided, it takes precedence over
name-based resolution.

Review fixes applied (code review round 1):
- P1-1: Narrowed bare `except Exception` to `except NotFoundError`
  in _resolve_config_files to avoid masking infrastructure errors.
- P1-2: Moved _resolve_config_files call inside the try block in
  run() so container/registry init errors get user-friendly messages.
  Added `except click.exceptions.Exit: raise` to let typer.Exit
  propagate through the broadened try scope.
- P1-3: Added atexit.register cleanup for temp files created by
  _resolve_config_files (resource leak fix).
- P1-4: Added CHANGELOG.md entry for the breaking CLI change.
- P2-1: Extracted duplicated _resolve_config_files to shared module
  `_resolve_actor.py`; both actor.py and actor_run.py now import it.
- P2-2: Added guard for actors with no configuration data
  (config_blob=None) to produce a clear error instead of invalid YAML.
- P2-3/P2-4: Added 5 BDD scenarios exercising the real
  resolve_config_files function (registry path, yaml_text path,
  config_blob fallback, no-config-data error, not-found error).
- P2-5: Added @coverage tags to all new BDD scenarios.
- P3-1: Added timeout=120s and on_timeout=kill to Robot tests.
- Fixed rxpy_route_validation.robot tests that used the removed
  --prompt/-p option (replaced with positional NAME + PROMPT args).

Review fixes applied (code review round 2):
- P2-1: Aligned actor_run.py exception handler from
  `CleverAgentsException` to `CleverAgentsError`, matching actor.py
  so infrastructure errors from registry resolution get user-friendly
  messages instead of falling through to the generic handler.
- P2-2: Changed `yaml.dump` to `yaml.safe_dump` in _resolve_actor.py
  for fail-fast behavior on unexpected types, consistent with the
  codebase's dominant pattern.
- P3-6: Replaced defensive `getattr(actor, ...)` calls with direct
  Pydantic model attribute access (`actor.yaml_text`, `actor.config_blob`)
  for type-checker coverage.
- P3-1: Switched BDD temp file cleanup from post-assertion
  `unlink()` to `context.add_cleanup()` for leak-proof teardown.
- P2-3/P3-2/P3-3/P3-4: Added 3 BDD edge-case scenarios
  (empty config_blob dict, infrastructure error propagation, empty
  string name) and 1 Robot test case (actor_app registry resolution).

Review fixes applied (code review round 3):
- P1-1: Migrated 48 remaining `-p` invocations across 9 Robot test
  files to the new positional `NAME PROMPT` pattern (context_delete_all_yes,
  load_context_test, scientific_paper_e2e_test, routing_prefix_stripping,
  scientific_paper_basic, scientific_paper_writer_test,
  context_management_test, initial_next_command_test,
  system_prompt_template_rendering).
- P2-1: Documented `--config/-c` as a spec deviation in
  `_resolve_actor.py` module docstring (spec lines 4562-4566 define
  `actor run` with no --config option; issue #901 AC accepts keeping
  it as optional).
- P2-2: Corrected `--config` help text from "fallback" to "overrides
  registry-based name resolution" — the option takes precedence, not
  the other way around.
- P3-1: Added `.strip()` to `yaml_text` emptiness check in
  `_resolve_actor.py` to handle whitespace-only values that would
  otherwise bypass the `config_blob` fallback.
- P3-2: Added `from None` to the no-configuration-data
  `typer.Exit(code=2)` for consistency with the not-found path.
- P3-3: Added BDD scenario testing `--config` precedence for
  `actor_run_app` (was only tested for `actor_app`).
- P3-4: Strengthened config_blob BDD scenario to verify generated
  YAML is parseable via `yaml.safe_load` round-trip.
- P3-7: Replaced hardcoded `/tmp/dummy.yaml` with
  `tempfile.gettempdir()` for portability.
- P3-8: Moved 5 inline imports to module level per CONTRIBUTING.md
  §1292-1294 (3x `import click`, 1x InfrastructureError in steps;
  1x `import typer` in robot helper).
- P3-9: Added `encoding="utf-8"` to `_write_yaml` in Robot helper
  for consistency with production code.

Review fixes applied (code review round 4):
- P2-1: Wrapped `yaml.safe_dump` in `_resolve_actor.py` with
  `try/except yaml.YAMLError` so non-serialisable config_blob values
  produce a user-friendly error message instead of a raw traceback.
- P2-2: Moved remaining inline `import yaml` to module level in
  `actor_run_signature_steps.py` per CONTRIBUTING.md §1292-1294.
- P2-3: Replaced 3 bare `assert` statements in Robot helper
  `helper_actor_run_signature.py` with diagnostic `if/print/sys.exit`
  pattern matching the rest of the file, improving failure diagnostics.

Review fixes applied (code review round 5):
- P3-4: Replaced per-call `atexit.register(lambda)` in
  `_resolve_actor.py` with a module-level `_temp_files` set and a
  single `atexit` handler (`_cleanup_temp_files`) to prevent
  unbounded handler accumulation in same-process usage (e.g. test
  suites running multiple CliRunner invocations).
- P3-1/P3-2/P3-3: Added 4 BDD scenarios: whitespace-only
  `yaml_text` fallback to config_blob, config-precedence
  registry-not-consulted assertion for both `actor_app` and
  `actor_run_app`, multiple `--config` files with positional NAME.
- P4-1: Strengthened error-path BDD assertions to verify error
  message content (not-found, no-config-data, serialisation-error)
  alongside exit codes via captured stderr.
- P4-2: Added Robot test case for `actor_app` unknown-name error
  path and corresponding helper function.

ISSUES CLOSED: #901
2026-03-26 13:41:31 +00:00

454 lines
14 KiB
Python

"""CLI invocation step definitions for actor run positional-argument tests.
Covers the ``actor.py`` and ``actor_run.py`` CLI entry-points with
positional NAME and PROMPT arguments, config fallback, registry
resolution, error paths, config-precedence registry spy assertions,
and multiple config-file forwarding.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock, patch
import typer
from behave import then, when
from cleveragents.cli.commands.actor import app as actor_app
from cleveragents.cli.commands.actor_run import app as actor_run_app
from features.steps.actor_run_signature_resolve_steps import (
_get_combined_output,
_make_app,
)
def _not_found_resolve(name: str, config: list[Any]) -> list[Any]:
"""Side-effect for _resolve_config_files that simulates registry miss."""
typer.echo(
f"Error: Actor '{name}' not found in registry and no --config provided.",
err=True,
)
raise typer.Exit(code=2)
# ---------------------------------------------------------------------------
# actor.py run command — positional args with --config fallback
# ---------------------------------------------------------------------------
@when("I run actor run with positional args and config fallback")
def step_run_actor_positional_with_config(context: Any) -> None:
context.prompt = "hello positional"
context.run_result = "positional response"
app_exec = _make_app(result=context.run_result)
with patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"--config",
str(context.actor_config_path),
"my-actor",
context.prompt,
],
)
context.app_exec = app_exec
@when("I run actor run with name resolved from the actor registry")
def step_run_actor_name_from_registry(context: Any) -> None:
context.prompt = "registry prompt"
context.run_result = "registry response"
app_exec = _make_app(result=context.run_result)
# Mock _resolve_config_files to return the existing config path
# (simulating successful registry resolution)
def _mock_resolve(name: str, config: list[Any]) -> list[Any]:
return [context.actor_config_path]
with (
patch(
"cleveragents.cli.commands.actor._resolve_config_files",
side_effect=_mock_resolve,
),
patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
),
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"local/my-actor",
context.prompt,
],
)
context.app_exec = app_exec
@when("I run actor run with a positional prompt containing flag-like text")
def step_run_actor_prompt_with_flag_like_text(context: Any) -> None:
context.prompt = "Fix the --output flag without changing --context"
context.run_result = "flag-like prompt response"
app_exec = _make_app(result=context.run_result)
with patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"--config",
str(context.actor_config_path),
"my-actor",
context.prompt,
],
)
context.app_exec = app_exec
@when("I run actor run through registry integration path")
def step_run_actor_registry_integration_path(context: Any) -> None:
context.prompt = "registry integration prompt"
context.run_result = "registry integration response"
app_exec = _make_app(result=context.run_result)
mock_actor = MagicMock()
mock_actor.yaml_text = "name: integration-actor\ntype: custom\n"
mock_actor.config_blob = None
mock_registry = MagicMock()
mock_registry.get.return_value = mock_actor
mock_container = MagicMock()
mock_container.actor_registry.return_value = mock_registry
with (
patch(
"cleveragents.cli.commands._resolve_actor.get_container",
return_value=mock_container,
),
patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
) as mock_app_cls,
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"local/integration-actor",
context.prompt,
],
)
call_kwargs = mock_app_cls.call_args.kwargs
config_files = call_kwargs.get("config_files", [])
if config_files:
temp_config = Path(config_files[0])
context.add_cleanup(lambda p=temp_config: p.unlink(missing_ok=True))
context.app_exec = app_exec
context.mock_registry = mock_registry
@when("I run actor run with config flag taking precedence")
def step_run_actor_config_takes_precedence(context: Any) -> None:
context.prompt = "precedence prompt"
context.run_result = "precedence response"
app_exec = _make_app(result=context.run_result)
with patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"--config",
str(context.actor_config_path),
"ignored-name",
context.prompt,
],
)
context.app_exec = app_exec
@when("I run actor run with an unknown actor name")
def step_run_actor_unknown_name(context: Any) -> None:
with patch(
"cleveragents.cli.commands.actor._resolve_config_files",
side_effect=_not_found_resolve,
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"nonexistent/actor",
"test prompt",
],
)
# ---------------------------------------------------------------------------
# actor_run.py module — positional args
# ---------------------------------------------------------------------------
@when("I invoke the actor-run module with config flag taking precedence")
def step_invoke_actor_run_config_takes_precedence(context: Any) -> None:
context.prompt = "actor-run precedence prompt"
context.run_result = "actor-run precedence response"
app_exec = _make_app(result=context.run_result)
with patch(
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
return_value=app_exec,
):
context.result = context.runner.invoke(
actor_run_app,
[
"--config",
str(context.actor_config_path),
"ignored-name",
context.prompt,
],
)
context.app_exec = app_exec
@when("I invoke the actor-run module with positional args and config fallback")
def step_invoke_actor_run_positional_with_config(context: Any) -> None:
context.prompt = "actor-run positional"
context.run_result = "actor-run positional response"
app_exec = _make_app(result=context.run_result)
with patch(
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
return_value=app_exec,
):
context.result = context.runner.invoke(
actor_run_app,
[
"--config",
str(context.actor_config_path),
"my-actor",
context.prompt,
],
)
context.app_exec = app_exec
@when("I invoke the actor-run module with name from the actor registry")
def step_invoke_actor_run_name_from_registry(context: Any) -> None:
context.prompt = "actor-run registry prompt"
context.run_result = "actor-run registry response"
app_exec = _make_app(result=context.run_result)
# Mock _resolve_config_files to return the existing config path
# (simulating successful registry resolution)
def _mock_resolve(name: str, config: list[Any]) -> list[Any]:
return [context.actor_config_path]
with (
patch(
"cleveragents.cli.commands.actor_run._resolve_config_files",
side_effect=_mock_resolve,
),
patch(
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
return_value=app_exec,
),
):
context.result = context.runner.invoke(
actor_run_app,
[
"local/my-actor",
context.prompt,
],
)
context.app_exec = app_exec
@when("I invoke the actor-run module with an unknown actor name")
def step_invoke_actor_run_unknown_name(context: Any) -> None:
with patch(
"cleveragents.cli.commands.actor_run._resolve_config_files",
side_effect=_not_found_resolve,
):
context.result = context.runner.invoke(
actor_run_app,
[
"nonexistent/actor",
"test prompt",
],
)
# ---------------------------------------------------------------------------
# Shared assertions
# ---------------------------------------------------------------------------
@then("the actor run should exit with registry not found error")
def step_actor_run_registry_not_found(context: Any) -> None:
assert context.result.exit_code == 2
combined = _get_combined_output(context.result)
assert "not found in registry" in combined
@then("the actor run should pass the exact positional prompt")
def step_actor_run_exact_prompt(context: Any) -> None:
assert context.result.exit_code == 0
run_args = context.app_exec.run_single_shot.call_args.args
assert run_args[0] == context.prompt
@then("the actor registry should have been consulted with requested actor name")
def step_registry_consulted_with_actor_name(context: Any) -> None:
context.mock_registry.get.assert_called_once_with("local/integration-actor")
# ---------------------------------------------------------------------------
# Review round 5 — P3-2: config precedence with registry spy
# ---------------------------------------------------------------------------
@when("I run actor run with config flag and a registry spy")
def step_run_actor_config_with_registry_spy(context: Any) -> None:
context.prompt = "spy prompt"
context.run_result = "spy response"
app_exec = _make_app(result=context.run_result)
mock_registry = MagicMock()
mock_container = MagicMock()
mock_container.actor_registry.return_value = mock_registry
with (
patch(
"cleveragents.cli.commands._resolve_actor.get_container",
return_value=mock_container,
),
patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
),
):
context.result = context.runner.invoke(
actor_app,
[
"run",
"--config",
str(context.actor_config_path),
"ignored-name",
context.prompt,
],
)
context.mock_registry = mock_registry
@when("I invoke the actor-run module with config flag and a registry spy")
def step_invoke_actor_run_config_with_registry_spy(context: Any) -> None:
context.prompt = "actor-run spy prompt"
context.run_result = "actor-run spy response"
app_exec = _make_app(result=context.run_result)
mock_registry = MagicMock()
mock_container = MagicMock()
mock_container.actor_registry.return_value = mock_registry
with (
patch(
"cleveragents.cli.commands._resolve_actor.get_container",
return_value=mock_container,
),
patch(
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
return_value=app_exec,
),
):
context.result = context.runner.invoke(
actor_run_app,
[
"--config",
str(context.actor_config_path),
"ignored-name",
context.prompt,
],
)
context.mock_registry = mock_registry
@then("the actor registry should not have been consulted")
def step_registry_not_consulted(context: Any) -> None:
assert context.result.exit_code == 0
context.mock_registry.get.assert_not_called()
# ---------------------------------------------------------------------------
# Review round 5 — P3-3: multiple config files with positional NAME
# ---------------------------------------------------------------------------
@when("I run actor run with multiple config files and a positional name")
def step_run_actor_multiple_configs(context: Any) -> None:
context.prompt = "multi-config prompt"
context.run_result = "multi-config response"
app_exec = _make_app(result=context.run_result)
second_config = Path(tempfile.gettempdir()) / "second_actor_config.yaml"
second_config.write_text(
"name: second\ntype: custom\nprovider: openai\nmodel: gpt-4\n",
encoding="utf-8",
)
context.add_cleanup(lambda: second_config.unlink(missing_ok=True))
context.second_actor_config_path = second_config
with patch(
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
return_value=app_exec,
) as mock_app_cls:
context.result = context.runner.invoke(
actor_app,
[
"run",
"--config",
str(context.actor_config_path),
"--config",
str(second_config),
"ignored-name",
context.prompt,
],
)
context.mock_app_cls = mock_app_cls
@then("all config files should be passed to the application")
def step_all_configs_passed(context: Any) -> None:
assert context.result.exit_code == 0
call_kwargs = context.mock_app_cls.call_args
config_files = call_kwargs[1].get(
"config_files", call_kwargs[0][0] if call_kwargs[0] else None
)
assert config_files is not None
assert len(config_files) == 2
assert Path(config_files[0]) == context.actor_config_path
assert Path(config_files[1]) == context.second_actor_config_path