forked from HAL9000/cleveragents-core
e4c01492d5
Aligned the `agents actor run` command signature with the specification by introducing positional NAME and PROMPT arguments. The --config/-c option is preserved as an optional fallback for direct YAML invocation. When NAME is provided without --config, the actor is resolved from the Actor Registry. Updated both actor_run.py and actor.py run commands. Added backward compatibility: if --config is provided, it takes precedence over name-based resolution. Review fixes applied (code review round 1): - P1-1: Narrowed bare `except Exception` to `except NotFoundError` in _resolve_config_files to avoid masking infrastructure errors. - P1-2: Moved _resolve_config_files call inside the try block in run() so container/registry init errors get user-friendly messages. Added `except click.exceptions.Exit: raise` to let typer.Exit propagate through the broadened try scope. - P1-3: Added atexit.register cleanup for temp files created by _resolve_config_files (resource leak fix). - P1-4: Added CHANGELOG.md entry for the breaking CLI change. - P2-1: Extracted duplicated _resolve_config_files to shared module `_resolve_actor.py`; both actor.py and actor_run.py now import it. - P2-2: Added guard for actors with no configuration data (config_blob=None) to produce a clear error instead of invalid YAML. - P2-3/P2-4: Added 5 BDD scenarios exercising the real resolve_config_files function (registry path, yaml_text path, config_blob fallback, no-config-data error, not-found error). - P2-5: Added @coverage tags to all new BDD scenarios. - P3-1: Added timeout=120s and on_timeout=kill to Robot tests. - Fixed rxpy_route_validation.robot tests that used the removed --prompt/-p option (replaced with positional NAME + PROMPT args). Review fixes applied (code review round 2): - P2-1: Aligned actor_run.py exception handler from `CleverAgentsException` to `CleverAgentsError`, matching actor.py so infrastructure errors from registry resolution get user-friendly messages instead of falling through to the generic handler. - P2-2: Changed `yaml.dump` to `yaml.safe_dump` in _resolve_actor.py for fail-fast behavior on unexpected types, consistent with the codebase's dominant pattern. - P3-6: Replaced defensive `getattr(actor, ...)` calls with direct Pydantic model attribute access (`actor.yaml_text`, `actor.config_blob`) for type-checker coverage. - P3-1: Switched BDD temp file cleanup from post-assertion `unlink()` to `context.add_cleanup()` for leak-proof teardown. - P2-3/P3-2/P3-3/P3-4: Added 3 BDD edge-case scenarios (empty config_blob dict, infrastructure error propagation, empty string name) and 1 Robot test case (actor_app registry resolution). Review fixes applied (code review round 3): - P1-1: Migrated 48 remaining `-p` invocations across 9 Robot test files to the new positional `NAME PROMPT` pattern (context_delete_all_yes, load_context_test, scientific_paper_e2e_test, routing_prefix_stripping, scientific_paper_basic, scientific_paper_writer_test, context_management_test, initial_next_command_test, system_prompt_template_rendering). - P2-1: Documented `--config/-c` as a spec deviation in `_resolve_actor.py` module docstring (spec lines 4562-4566 define `actor run` with no --config option; issue #901 AC accepts keeping it as optional). - P2-2: Corrected `--config` help text from "fallback" to "overrides registry-based name resolution" — the option takes precedence, not the other way around. - P3-1: Added `.strip()` to `yaml_text` emptiness check in `_resolve_actor.py` to handle whitespace-only values that would otherwise bypass the `config_blob` fallback. - P3-2: Added `from None` to the no-configuration-data `typer.Exit(code=2)` for consistency with the not-found path. - P3-3: Added BDD scenario testing `--config` precedence for `actor_run_app` (was only tested for `actor_app`). - P3-4: Strengthened config_blob BDD scenario to verify generated YAML is parseable via `yaml.safe_load` round-trip. - P3-7: Replaced hardcoded `/tmp/dummy.yaml` with `tempfile.gettempdir()` for portability. - P3-8: Moved 5 inline imports to module level per CONTRIBUTING.md §1292-1294 (3x `import click`, 1x InfrastructureError in steps; 1x `import typer` in robot helper). - P3-9: Added `encoding="utf-8"` to `_write_yaml` in Robot helper for consistency with production code. Review fixes applied (code review round 4): - P2-1: Wrapped `yaml.safe_dump` in `_resolve_actor.py` with `try/except yaml.YAMLError` so non-serialisable config_blob values produce a user-friendly error message instead of a raw traceback. - P2-2: Moved remaining inline `import yaml` to module level in `actor_run_signature_steps.py` per CONTRIBUTING.md §1292-1294. - P2-3: Replaced 3 bare `assert` statements in Robot helper `helper_actor_run_signature.py` with diagnostic `if/print/sys.exit` pattern matching the rest of the file, improving failure diagnostics. Review fixes applied (code review round 5): - P3-4: Replaced per-call `atexit.register(lambda)` in `_resolve_actor.py` with a module-level `_temp_files` set and a single `atexit` handler (`_cleanup_temp_files`) to prevent unbounded handler accumulation in same-process usage (e.g. test suites running multiple CliRunner invocations). - P3-1/P3-2/P3-3: Added 4 BDD scenarios: whitespace-only `yaml_text` fallback to config_blob, config-precedence registry-not-consulted assertion for both `actor_app` and `actor_run_app`, multiple `--config` files with positional NAME. - P4-1: Strengthened error-path BDD assertions to verify error message content (not-found, no-config-data, serialisation-error) alongside exit codes via captured stderr. - P4-2: Added Robot test case for `actor_app` unknown-name error path and corresponding helper function. ISSUES CLOSED: #901
454 lines
14 KiB
Python
454 lines
14 KiB
Python
"""CLI invocation step definitions for actor run positional-argument tests.
|
|
|
|
Covers the ``actor.py`` and ``actor_run.py`` CLI entry-points with
|
|
positional NAME and PROMPT arguments, config fallback, registry
|
|
resolution, error paths, config-precedence registry spy assertions,
|
|
and multiple config-file forwarding.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import tempfile
|
|
from pathlib import Path
|
|
from typing import Any
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import typer
|
|
from behave import then, when
|
|
|
|
from cleveragents.cli.commands.actor import app as actor_app
|
|
from cleveragents.cli.commands.actor_run import app as actor_run_app
|
|
from features.steps.actor_run_signature_resolve_steps import (
|
|
_get_combined_output,
|
|
_make_app,
|
|
)
|
|
|
|
|
|
def _not_found_resolve(name: str, config: list[Any]) -> list[Any]:
|
|
"""Side-effect for _resolve_config_files that simulates registry miss."""
|
|
typer.echo(
|
|
f"Error: Actor '{name}' not found in registry and no --config provided.",
|
|
err=True,
|
|
)
|
|
raise typer.Exit(code=2)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# actor.py run command — positional args with --config fallback
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I run actor run with positional args and config fallback")
|
|
def step_run_actor_positional_with_config(context: Any) -> None:
|
|
context.prompt = "hello positional"
|
|
context.run_result = "positional response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"my-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with name resolved from the actor registry")
|
|
def step_run_actor_name_from_registry(context: Any) -> None:
|
|
context.prompt = "registry prompt"
|
|
context.run_result = "registry response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
# Mock _resolve_config_files to return the existing config path
|
|
# (simulating successful registry resolution)
|
|
def _mock_resolve(name: str, config: list[Any]) -> list[Any]:
|
|
return [context.actor_config_path]
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor._resolve_config_files",
|
|
side_effect=_mock_resolve,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"local/my-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with a positional prompt containing flag-like text")
|
|
def step_run_actor_prompt_with_flag_like_text(context: Any) -> None:
|
|
context.prompt = "Fix the --output flag without changing --context"
|
|
context.run_result = "flag-like prompt response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"my-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run through registry integration path")
|
|
def step_run_actor_registry_integration_path(context: Any) -> None:
|
|
context.prompt = "registry integration prompt"
|
|
context.run_result = "registry integration response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
mock_actor = MagicMock()
|
|
mock_actor.yaml_text = "name: integration-actor\ntype: custom\n"
|
|
mock_actor.config_blob = None
|
|
|
|
mock_registry = MagicMock()
|
|
mock_registry.get.return_value = mock_actor
|
|
|
|
mock_container = MagicMock()
|
|
mock_container.actor_registry.return_value = mock_registry
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands._resolve_actor.get_container",
|
|
return_value=mock_container,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_app_cls,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"local/integration-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
call_kwargs = mock_app_cls.call_args.kwargs
|
|
config_files = call_kwargs.get("config_files", [])
|
|
if config_files:
|
|
temp_config = Path(config_files[0])
|
|
context.add_cleanup(lambda p=temp_config: p.unlink(missing_ok=True))
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_registry = mock_registry
|
|
|
|
|
|
@when("I run actor run with config flag taking precedence")
|
|
def step_run_actor_config_takes_precedence(context: Any) -> None:
|
|
context.prompt = "precedence prompt"
|
|
context.run_result = "precedence response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"ignored-name",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with an unknown actor name")
|
|
def step_run_actor_unknown_name(context: Any) -> None:
|
|
with patch(
|
|
"cleveragents.cli.commands.actor._resolve_config_files",
|
|
side_effect=_not_found_resolve,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"nonexistent/actor",
|
|
"test prompt",
|
|
],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# actor_run.py module — positional args
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I invoke the actor-run module with config flag taking precedence")
|
|
def step_invoke_actor_run_config_takes_precedence(context: Any) -> None:
|
|
context.prompt = "actor-run precedence prompt"
|
|
context.run_result = "actor-run precedence response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"ignored-name",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I invoke the actor-run module with positional args and config fallback")
|
|
def step_invoke_actor_run_positional_with_config(context: Any) -> None:
|
|
context.prompt = "actor-run positional"
|
|
context.run_result = "actor-run positional response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"my-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I invoke the actor-run module with name from the actor registry")
|
|
def step_invoke_actor_run_name_from_registry(context: Any) -> None:
|
|
context.prompt = "actor-run registry prompt"
|
|
context.run_result = "actor-run registry response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
# Mock _resolve_config_files to return the existing config path
|
|
# (simulating successful registry resolution)
|
|
def _mock_resolve(name: str, config: list[Any]) -> list[Any]:
|
|
return [context.actor_config_path]
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run._resolve_config_files",
|
|
side_effect=_mock_resolve,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"local/my-actor",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I invoke the actor-run module with an unknown actor name")
|
|
def step_invoke_actor_run_unknown_name(context: Any) -> None:
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run._resolve_config_files",
|
|
side_effect=_not_found_resolve,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"nonexistent/actor",
|
|
"test prompt",
|
|
],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Shared assertions
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@then("the actor run should exit with registry not found error")
|
|
def step_actor_run_registry_not_found(context: Any) -> None:
|
|
assert context.result.exit_code == 2
|
|
combined = _get_combined_output(context.result)
|
|
assert "not found in registry" in combined
|
|
|
|
|
|
@then("the actor run should pass the exact positional prompt")
|
|
def step_actor_run_exact_prompt(context: Any) -> None:
|
|
assert context.result.exit_code == 0
|
|
run_args = context.app_exec.run_single_shot.call_args.args
|
|
assert run_args[0] == context.prompt
|
|
|
|
|
|
@then("the actor registry should have been consulted with requested actor name")
|
|
def step_registry_consulted_with_actor_name(context: Any) -> None:
|
|
context.mock_registry.get.assert_called_once_with("local/integration-actor")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Review round 5 — P3-2: config precedence with registry spy
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I run actor run with config flag and a registry spy")
|
|
def step_run_actor_config_with_registry_spy(context: Any) -> None:
|
|
context.prompt = "spy prompt"
|
|
context.run_result = "spy response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
mock_registry = MagicMock()
|
|
mock_container = MagicMock()
|
|
mock_container.actor_registry.return_value = mock_registry
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands._resolve_actor.get_container",
|
|
return_value=mock_container,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"ignored-name",
|
|
context.prompt,
|
|
],
|
|
)
|
|
context.mock_registry = mock_registry
|
|
|
|
|
|
@when("I invoke the actor-run module with config flag and a registry spy")
|
|
def step_invoke_actor_run_config_with_registry_spy(context: Any) -> None:
|
|
context.prompt = "actor-run spy prompt"
|
|
context.run_result = "actor-run spy response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
mock_registry = MagicMock()
|
|
mock_container = MagicMock()
|
|
mock_container.actor_registry.return_value = mock_registry
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands._resolve_actor.get_container",
|
|
return_value=mock_container,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"ignored-name",
|
|
context.prompt,
|
|
],
|
|
)
|
|
context.mock_registry = mock_registry
|
|
|
|
|
|
@then("the actor registry should not have been consulted")
|
|
def step_registry_not_consulted(context: Any) -> None:
|
|
assert context.result.exit_code == 0
|
|
context.mock_registry.get.assert_not_called()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Review round 5 — P3-3: multiple config files with positional NAME
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I run actor run with multiple config files and a positional name")
|
|
def step_run_actor_multiple_configs(context: Any) -> None:
|
|
context.prompt = "multi-config prompt"
|
|
context.run_result = "multi-config response"
|
|
app_exec = _make_app(result=context.run_result)
|
|
|
|
second_config = Path(tempfile.gettempdir()) / "second_actor_config.yaml"
|
|
second_config.write_text(
|
|
"name: second\ntype: custom\nprovider: openai\nmodel: gpt-4\n",
|
|
encoding="utf-8",
|
|
)
|
|
context.add_cleanup(lambda: second_config.unlink(missing_ok=True))
|
|
context.second_actor_config_path = second_config
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_app_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--config",
|
|
str(second_config),
|
|
"ignored-name",
|
|
context.prompt,
|
|
],
|
|
)
|
|
context.mock_app_cls = mock_app_cls
|
|
|
|
|
|
@then("all config files should be passed to the application")
|
|
def step_all_configs_passed(context: Any) -> None:
|
|
assert context.result.exit_code == 0
|
|
call_kwargs = context.mock_app_cls.call_args
|
|
config_files = call_kwargs[1].get(
|
|
"config_files", call_kwargs[0][0] if call_kwargs[0] else None
|
|
)
|
|
assert config_files is not None
|
|
assert len(config_files) == 2
|
|
assert Path(config_files[0]) == context.actor_config_path
|
|
assert Path(config_files[1]) == context.second_actor_config_path
|