forked from HAL9000/cleveragents-core
hurui200320
554d6889cc
## Summary Add the missing `--skill` repeatable flag to `actor run` and `actor-run` CLI commands, aligning the implementation with the specification (CLI Synopsis line 277). The flag enables ad-hoc skill injection at runtime without modifying YAML configuration. Closes #887 ## Changes ### DI Container - **`container.py`**: Added `_build_skill_service()` factory and `skill_service` Singleton provider, following the established `_build_*` pattern. Falls back to in-memory `SkillService()` when the database is unavailable. Exception handling narrowed to `(ImportError, OperationalError, DatabaseError, OSError)` with `exc_info=True` for traceability. ### CLI Layer - **`actor.py`**: Added `--skill` Typer option (`list[str] | None`, repeatable, `metavar="NAME"`). Help text notes that skills only augment tool-bearing agents. Wrapped constructor in the existing `try/except` block so `CleverAgentsException` from skill resolution is properly caught. - **`actor_run.py`**: Same `--skill` option with `metavar="NAME"`. Exception handler catches `CleverAgentsException` (matching master — not broadened to `CleverAgentsError`). - **`skill.py`**: Removed module-level `_service` cache. `_get_skill_service()` now always delegates to `get_container().skill_service()` so that `reset_container()` correctly invalidates the cached instance. `_reset_skill_service()` now overrides the container's provider via `providers.Object()`. Removed dead `validate_skill_names()` function. ### Runtime Layer - **`application.py`** (438 lines, down from 625): `ReactiveCleverAgentsApp` gains `skill_names` parameter with automatic deduplication via `dict.fromkeys`. `_resolve_skills()` obtains `SkillService` from the DI container (no CLI layer import). Separate `except KeyError` and `except ValueError` produce distinct error messages (`"not found in registry"` vs `"resolution failed: {exc}"`). Skill tools are only injected into agents that already have tools (`if self._resolved_skill_tools and tools:`), preventing LLM agents from being converted to pass-through `SimpleToolAgent` instances. When skill tools are skipped for tool-less agents, `logger.debug` emits a diagnostic message. `_sanitize_skill_name()` validates skill name format with tightened regex: `^[\w.-]{1,127}/[\w.-]{1,127}$` with `re.ASCII` flag. Zero-tool skill warning now uses `logger.warning` (not `print(stderr)`), ensuring structured log output and proper log-level filtering. - **`graph_executor.py`** (334 lines): Extracted graph execution logic. Type annotations improved. ### Tests - 24+ Behave scenarios across feature files covering: single/multiple/unknown skill flags, skill+context combined, duplicate deduplication, skill resolution, ValueError path, zero-tool resolution, error handling, tool merging, default behavior, overrides, LLM agent guard, `_sanitize_skill_name` edge cases (empty string, too-long name, ANSI escape codes, disallowed characters), `_build_skill_service` happy+fallback paths, `_get_skill_service` container delegation. - CLI "unknown skill" tests for **both** `actor.py` and `actor_run.py` exercise the real error chain (mock only `get_container()`, not the entire `ReactiveCleverAgentsApp`), testing `_resolve_skills()` → `CleverAgentsException` → `except CleverAgentsException` → exit code 2 end-to-end. - Combined skill+context tests assert `ContextManager` was instantiated and `exists()` was called in dedicated **Then** steps. - `@coverage` tags added to all new scenarios. - **Robot Framework smoke tests** added (`robot/skill_actor_run.robot` + `robot/helper_skill_actor_run.py`): unknown-skill error path and valid-skill acceptance path. ### Changelog - Added entry under `## Unreleased` in `CHANGELOG.md`. ## Review Fixes Applied (Brent Edwards, Rounds 1 & 2) | # | Finding | Resolution | |---|---------|------------| | **P1-1** | `print(stderr)` for zero-tool skill warning | **Fixed** — replaced with `logger.warning("Skill '%s' resolved to zero tools", name)`, removed unused `import sys` | | **P2-2** | Skill tools silently skipped for tool-less agents | **Fixed** — added `logger.debug` when skipped; updated `--skill` help text to note "only augments tool-bearing agents" | | **P2-3** | `container.py` at 739 lines | **Acknowledged** — pre-existing growth (+59 lines for `_build_skill_service`); extracting factories is a separate refactoring task | | **P2-4↑** | `CleverAgentsException` → `CleverAgentsError` broadens catch scope | **Fixed** — reverted `actor_run.py` to `except CleverAgentsException` matching master | | **P3-5** | No Robot Framework smoke test for `--skill` | **Fixed** — added `skill_actor_run.robot` with 2 test cases (unknown-skill error, valid-skill acceptance) | | **P3-6** | `GraphExecutor._follow_chained_edges` static-calling-static | **Acknowledged** — cosmetic pattern that doesn't affect correctness; can address in a follow-up | ## Known Limitations / Deferred Items | Item | Reason | |------|--------| | `actor.py` at 679 lines (500-line guideline) | Pre-existing (670 on master), +9 lines for `--skill`. Refactoring the shared `_execute()` closure is a separate task. | | `container.py` at 739 lines (500-line guideline) | Was 680 lines on master, +59 lines for `_build_skill_service()` and `skill_service` provider. Refactoring into sub-modules is a separate task. | | Code duplication between `actor.py` and `actor_run.py` `run()` | ~47 lines identical code. Coupled with the line-count issue above — both require extracting shared execution logic into a helper module. | | `SimpleToolAgent` only executes `tools[0]` | Deferred to #974. Pre-existing architectural limitation, not introduced by this PR. | | `GraphExecutor._follow_chained_edges` static-calling-static pattern | Cosmetic, doesn't affect behavior. | ## Quality Gates - `nox -s lint`: ✅ PASS - `nox -s typecheck`: ✅ PASS (0 errors) - `nox -s unit_tests`: ✅ PASS (11,130 scenarios, 0 failures) - `nox -s integration_tests`: ✅ PASS (1,559 tests, 0 failures) - `nox -s coverage_report`: ✅ 97% (meets threshold) - Branch rebased onto latest `master` (`ab1fd19b`) Reviewed-on: cleveragents/cleveragents-core#971 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
1041 lines
31 KiB
Python
1041 lines
31 KiB
Python
"""Step definitions for actor CLI run coverage."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import tempfile
|
|
from pathlib import Path
|
|
from types import SimpleNamespace
|
|
from typing import Any
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
from behave import given, then, when
|
|
|
|
from cleveragents.cli.commands.actor import app as actor_app
|
|
from cleveragents.cli.commands.actor_run import app as actor_run_app
|
|
from cleveragents.core.exceptions import (
|
|
CleverAgentsError,
|
|
CleverAgentsException,
|
|
UnsafeConfigurationError,
|
|
)
|
|
|
|
|
|
def _register_cleanup(context, path: Path) -> None:
|
|
context._cleanup_handlers.append(lambda: path.unlink(missing_ok=True))
|
|
|
|
|
|
def _make_app(
|
|
*,
|
|
result: str,
|
|
config_global_context: dict[str, Any] | None = None,
|
|
run_side_effect: Exception | None = None,
|
|
) -> MagicMock:
|
|
app_exec = MagicMock()
|
|
app_exec.config = SimpleNamespace(global_context=dict(config_global_context or {}))
|
|
app_exec.run_single_shot = AsyncMock(return_value=result)
|
|
if run_side_effect is not None:
|
|
app_exec.run_single_shot.side_effect = run_side_effect
|
|
return app_exec
|
|
|
|
|
|
def _make_context_manager(
|
|
*,
|
|
global_context: dict[str, Any] | None = None,
|
|
exists: bool = False,
|
|
) -> MagicMock:
|
|
ctx_mgr = MagicMock()
|
|
ctx_mgr.global_context = dict(global_context or {})
|
|
ctx_mgr.exists.return_value = exists
|
|
return ctx_mgr
|
|
|
|
|
|
@given("I have a saved context JSON file")
|
|
def step_saved_context_json(context):
|
|
context.load_context_data = {"global_context": {"session": "stored"}}
|
|
with tempfile.NamedTemporaryFile(
|
|
delete=False, suffix=".json", mode="w", encoding="utf-8"
|
|
) as handle:
|
|
json.dump(context.load_context_data, handle)
|
|
handle.flush()
|
|
context.load_context_path = Path(handle.name)
|
|
_register_cleanup(context, context.load_context_path)
|
|
|
|
|
|
@given("I have an actor output file path")
|
|
def step_actor_output_path(context):
|
|
with tempfile.NamedTemporaryFile(
|
|
delete=False, suffix=".txt", mode="w", encoding="utf-8"
|
|
) as handle:
|
|
pass
|
|
context.output_path = Path(handle.name)
|
|
_register_cleanup(context, context.output_path)
|
|
|
|
|
|
@when("I run actor run with load context and context name")
|
|
def step_run_actor_with_load_and_context(context):
|
|
context.prompt = "hello from context"
|
|
context.run_result = "actor response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(global_context={"from": "context"})
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"session-1",
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
"--output",
|
|
str(context.output_path),
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I run actor run with load context only")
|
|
def step_run_actor_with_load_only(context):
|
|
context.prompt = "load-only"
|
|
context.run_result = "loaded context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with context only")
|
|
def step_run_actor_with_context_only(context):
|
|
context.prompt = "context-only"
|
|
context.run_result = "context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(
|
|
global_context={"cached": "value"},
|
|
exists=True,
|
|
)
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"session-2",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I run actor run without context")
|
|
def step_run_actor_without_context(context):
|
|
context.prompt = "no-context"
|
|
context.run_result = "direct response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with load context and context name allowing rxpy")
|
|
def step_run_actor_with_load_context_and_rxpy(context):
|
|
context.prompt = "rxpy-context"
|
|
context.run_result = "rxpy response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(global_context={"from": "context"})
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"session-rxpy",
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
"--output",
|
|
str(context.output_path),
|
|
"--allow-rxpy-in-run-mode",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I run actor run with load context only allowing rxpy")
|
|
def step_run_actor_with_load_only_rxpy(context):
|
|
context.prompt = "rxpy-load-only"
|
|
context.run_result = "rxpy load response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
"--allow-rxpy-in-run-mode",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with context only allowing rxpy")
|
|
def step_run_actor_with_context_only_rxpy(context):
|
|
context.prompt = "rxpy-context-only"
|
|
context.run_result = "rxpy context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(
|
|
global_context={"cached": "value"},
|
|
exists=True,
|
|
)
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"session-rxpy-only",
|
|
"--allow-rxpy-in-run-mode",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I run actor run without context allowing rxpy")
|
|
def step_run_actor_without_context_rxpy(context):
|
|
context.prompt = "rxpy-no-context"
|
|
context.run_result = "rxpy direct response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--allow-rxpy-in-run-mode",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I run actor run with unsafe configuration error")
|
|
def step_run_actor_unsafe_error(context):
|
|
context.prompt = "unsafe"
|
|
error = UnsafeConfigurationError("unsafe config")
|
|
app_exec = _make_app(
|
|
result="unused",
|
|
config_global_context={"existing": "value"},
|
|
run_side_effect=error,
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.expected_error = "Error: unsafe config"
|
|
context.expected_exit_code = 1
|
|
|
|
|
|
@when("I run actor run with clever agents error")
|
|
def step_run_actor_clever_agents_error(context):
|
|
context.prompt = "failed"
|
|
error = CleverAgentsError("request failed")
|
|
app_exec = _make_app(
|
|
result="unused",
|
|
config_global_context={"existing": "value"},
|
|
run_side_effect=error,
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.expected_error = "Error: request failed"
|
|
context.expected_exit_code = 2
|
|
|
|
|
|
@when("I invoke the actor-run command with load context and a context name")
|
|
def step_invoke_actor_run_with_load_and_context(context):
|
|
context.prompt = "hello from actor-run"
|
|
context.run_result = "actor-run response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(global_context={"from": "context"})
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"actor-run-session",
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
"--output",
|
|
str(context.output_path),
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I invoke the actor-run command with load context only")
|
|
def step_invoke_actor_run_with_load_only(context):
|
|
context.prompt = "actor-run load-only"
|
|
context.run_result = "actor-run loaded response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--load-context",
|
|
str(context.load_context_path),
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I invoke the actor-run command with context only")
|
|
def step_invoke_actor_run_with_context_only(context):
|
|
context.prompt = "actor-run context-only"
|
|
context.run_result = "actor-run context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(
|
|
global_context={"cached": "value"},
|
|
exists=True,
|
|
)
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ContextManager",
|
|
return_value=ctx_mgr,
|
|
),
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--context",
|
|
"actor-run-session-2",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.ctx_mgr = ctx_mgr
|
|
|
|
|
|
@when("I invoke the actor-run command without any context")
|
|
def step_invoke_actor_run_without_context(context):
|
|
context.prompt = "actor-run no context"
|
|
context.run_result = "actor-run direct response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
|
|
|
|
@when("I invoke the actor-run command with an unsafe configuration error")
|
|
def step_invoke_actor_run_unsafe_error(context):
|
|
context.prompt = "actor-run unsafe"
|
|
error = UnsafeConfigurationError("unsafe config")
|
|
app_exec = _make_app(
|
|
result="unused",
|
|
config_global_context={"existing": "value"},
|
|
run_side_effect=error,
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.expected_error = "Error: unsafe config"
|
|
context.expected_exit_code = 1
|
|
|
|
|
|
@when("I invoke the actor-run command with a clever agents exception")
|
|
def step_invoke_actor_run_clever_agents_exception(context):
|
|
context.prompt = "actor-run failed"
|
|
error = CleverAgentsException("request failed")
|
|
app_exec = _make_app(
|
|
result="unused",
|
|
config_global_context={"existing": "value"},
|
|
run_side_effect=error,
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
],
|
|
)
|
|
|
|
context.expected_error = "Error: request failed"
|
|
context.expected_exit_code = 2
|
|
|
|
|
|
@then("the actor run should write output and persist context")
|
|
def step_actor_run_writes_output(context):
|
|
assert context.result.exit_code == 0
|
|
assert context.output_path.read_text() == context.run_result
|
|
assert f"Output written to {context.output_path}" in context.result.output
|
|
context.ctx_mgr.import_context.assert_called_once_with(context.load_context_path)
|
|
context.ctx_mgr.add_message.assert_any_call("user", context.prompt)
|
|
context.ctx_mgr.add_message.assert_any_call("assistant", context.run_result)
|
|
context.ctx_mgr.save_global_context.assert_called_once_with(
|
|
context.app_exec.config.global_context
|
|
)
|
|
assert context.app_exec.config.global_context.get("from") == "context"
|
|
|
|
|
|
@then("the actor run should update global context and echo result")
|
|
def step_actor_run_updates_global_context(context):
|
|
assert context.result.exit_code == 0
|
|
assert context.run_result in context.result.output
|
|
for key, value in context.load_context_data["global_context"].items():
|
|
assert context.app_exec.config.global_context.get(key) == value
|
|
call_kwargs = context.app_exec.run_single_shot.call_args.kwargs
|
|
assert call_kwargs.get("context_manager") is None
|
|
|
|
|
|
@then("the actor run should reuse saved context")
|
|
def step_actor_run_reuses_context(context):
|
|
assert context.result.exit_code == 0
|
|
context.ctx_mgr.exists.assert_called_once()
|
|
context.ctx_mgr.add_message.assert_any_call("user", context.prompt)
|
|
context.ctx_mgr.add_message.assert_any_call("assistant", context.run_result)
|
|
context.ctx_mgr.save_global_context.assert_called_once_with(
|
|
context.app_exec.config.global_context
|
|
)
|
|
call_kwargs = context.app_exec.run_single_shot.call_args.kwargs
|
|
assert call_kwargs.get("context_manager") == context.ctx_mgr
|
|
assert context.app_exec.config.global_context.get("cached") == "value"
|
|
|
|
|
|
@then("the actor run should call single shot without context manager")
|
|
def step_actor_run_single_shot_no_context(context):
|
|
assert context.result.exit_code == 0
|
|
assert context.run_result in context.result.output
|
|
call_kwargs = context.app_exec.run_single_shot.call_args.kwargs
|
|
assert call_kwargs.get("context_manager") is None
|
|
|
|
|
|
@then("the actor run should exit with error code 1")
|
|
def step_actor_run_exit_code_one(context):
|
|
assert context.result.exit_code == 1
|
|
assert context.expected_error in context.result.output
|
|
|
|
|
|
@then("the actor run should exit with error code 2")
|
|
def step_actor_run_exit_code_two(context):
|
|
assert context.result.exit_code == 2
|
|
assert context.expected_error in context.result.output
|
|
|
|
|
|
@then("the actor run should pass allow rxpy flag")
|
|
def step_actor_run_allow_rxpy_flag(context):
|
|
call_kwargs = context.app_exec.run_single_shot.call_args.kwargs
|
|
assert call_kwargs.get("allow_rxpy_in_run_mode") is True
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# --skill flag: actor.py run command
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I run actor run with a single skill flag")
|
|
def step_run_actor_with_single_skill(context):
|
|
context.prompt = "skill-single"
|
|
context.run_result = "skill response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.expected_skill_names = ["local/web-tools"]
|
|
|
|
|
|
@when("I run actor run with multiple skill flags")
|
|
def step_run_actor_with_multiple_skills(context):
|
|
context.prompt = "skill-multi"
|
|
context.run_result = "multi skill response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
"--skill",
|
|
"local/db-tools",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.expected_skill_names = ["local/web-tools", "local/db-tools"]
|
|
|
|
|
|
@when("I run actor run with an unknown skill flag")
|
|
def step_run_actor_with_unknown_skill(context):
|
|
"""Exercise the real error chain: ReactiveCleverAgentsApp.__init__
|
|
calls _resolve_skills(), which obtains SkillService from the DI
|
|
container. We mock only the container so the real constructor and
|
|
_resolve_skills() execute end-to-end.
|
|
"""
|
|
context.prompt = "skill-unknown"
|
|
|
|
mock_service = MagicMock()
|
|
mock_service.resolve_tools.side_effect = KeyError(
|
|
"Skill 'local/nonexistent' is not registered"
|
|
)
|
|
mock_container = MagicMock()
|
|
mock_container.skill_service.return_value = mock_service
|
|
|
|
with patch(
|
|
"cleveragents.reactive.application.get_container",
|
|
return_value=mock_container,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/nonexistent",
|
|
],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# --skill flag: actor_run.py run command
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I invoke the actor-run command with a single skill flag")
|
|
def step_invoke_actor_run_with_single_skill(context):
|
|
context.prompt = "actor-run skill-single"
|
|
context.run_result = "actor-run skill response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.expected_skill_names = ["local/web-tools"]
|
|
|
|
|
|
@when("I invoke the actor-run command with multiple skill flags")
|
|
def step_invoke_actor_run_with_multiple_skills(context):
|
|
context.prompt = "actor-run skill-multi"
|
|
context.run_result = "actor-run multi skill response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
"--skill",
|
|
"local/db-tools",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.expected_skill_names = ["local/web-tools", "local/db-tools"]
|
|
|
|
|
|
@when("I invoke the actor-run command with an unknown skill flag")
|
|
def step_invoke_actor_run_with_unknown_skill(context):
|
|
"""Exercise the real error chain: ReactiveCleverAgentsApp.__init__
|
|
calls _resolve_skills(), which obtains SkillService from the DI
|
|
container. We mock only the container so the real constructor and
|
|
_resolve_skills() execute end-to-end.
|
|
"""
|
|
context.prompt = "actor-run skill-unknown"
|
|
|
|
mock_service = MagicMock()
|
|
mock_service.resolve_tools.side_effect = KeyError(
|
|
"Skill 'local/nonexistent' is not registered"
|
|
)
|
|
mock_container = MagicMock()
|
|
mock_container.skill_service.return_value = mock_service
|
|
|
|
with patch(
|
|
"cleveragents.reactive.application.get_container",
|
|
return_value=mock_container,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/nonexistent",
|
|
],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# --skill flag: shared assertions
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@then("the actor run should pass skill names to the runtime")
|
|
@then("the actor run should pass all skill names to the runtime")
|
|
def step_actor_run_pass_skill_names(context):
|
|
assert context.result.exit_code == 0
|
|
call_kwargs = context.mock_cls.call_args.kwargs
|
|
assert call_kwargs.get("skill_names") == context.expected_skill_names
|
|
|
|
|
|
@then("the actor run should exit with skill not found error")
|
|
def step_actor_run_skill_not_found(context):
|
|
assert context.result.exit_code == 2
|
|
assert "Error: Skill 'local/nonexistent' not found in registry" in (
|
|
context.result.output
|
|
)
|
|
|
|
|
|
@then("the context manager should have been instantiated")
|
|
def step_context_manager_instantiated(context):
|
|
context.mock_ctx_cls.assert_called_once()
|
|
|
|
|
|
@then("the context manager exists should have been called")
|
|
def step_context_manager_exists_called(context):
|
|
context.ctx_mgr.exists.assert_called_once()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# --skill + --context combined flag test
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I run actor run with skill and context flags")
|
|
def step_run_actor_with_skill_and_context(context):
|
|
context.prompt = "skill-context-combo"
|
|
context.run_result = "skill context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(
|
|
global_context={"cached": "value"},
|
|
exists=True,
|
|
)
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls,
|
|
patch(
|
|
"cleveragents.cli.commands.actor.ContextManager",
|
|
return_value=ctx_mgr,
|
|
) as mock_ctx_cls,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_app,
|
|
[
|
|
"run",
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
"--context",
|
|
"test-session",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.mock_ctx_cls = mock_ctx_cls
|
|
context.ctx_mgr = ctx_mgr
|
|
context.expected_skill_names = ["local/web-tools"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# --skill + --context combined flag test for actor_run.py (M6)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I invoke the actor-run command with skill and context flags")
|
|
def step_invoke_actor_run_with_skill_and_context(context):
|
|
context.prompt = "actor-run skill-context-combo"
|
|
context.run_result = "actor-run skill context response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
ctx_mgr = _make_context_manager(
|
|
global_context={"cached": "value"},
|
|
exists=True,
|
|
)
|
|
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls,
|
|
patch(
|
|
"cleveragents.cli.commands.actor_run.ContextManager",
|
|
return_value=ctx_mgr,
|
|
) as mock_ctx_cls,
|
|
):
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
"--context",
|
|
"test-session",
|
|
],
|
|
)
|
|
|
|
context.app_exec = app_exec
|
|
context.mock_cls = mock_cls
|
|
context.mock_ctx_cls = mock_ctx_cls
|
|
context.ctx_mgr = ctx_mgr
|
|
context.expected_skill_names = ["local/web-tools"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Duplicate skill deduplication test (M7)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I invoke the actor-run command with duplicate skill flags")
|
|
def step_invoke_actor_run_with_duplicate_skills(context):
|
|
context.prompt = "actor-run skill-dedup"
|
|
context.run_result = "actor-run dedup response"
|
|
app_exec = _make_app(
|
|
result=context.run_result,
|
|
config_global_context={"existing": "value"},
|
|
)
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.actor_run.ReactiveCleverAgentsApp",
|
|
return_value=app_exec,
|
|
) as mock_cls:
|
|
context.result = context.runner.invoke(
|
|
actor_run_app,
|
|
[
|
|
"--config",
|
|
str(context.actor_config_path),
|
|
"--prompt",
|
|
context.prompt,
|
|
"--skill",
|
|
"local/web-tools",
|
|
"--skill",
|
|
"local/web-tools",
|
|
],
|
|
)
|
|
|
|
context.mock_cls = mock_cls
|
|
|
|
|
|
@then("the actor run should pass duplicate skill names to the runtime")
|
|
def step_actor_run_pass_duplicate_skill_names(context):
|
|
assert context.result.exit_code == 0
|
|
call_kwargs = context.mock_cls.call_args.kwargs
|
|
# CLI passes duplicate names through unchanged; deduplication
|
|
# happens inside ReactiveCleverAgentsApp.__init__ via dict.fromkeys.
|
|
assert call_kwargs.get("skill_names") == [
|
|
"local/web-tools",
|
|
"local/web-tools",
|
|
]
|