forked from HAL9000/cleveragents-core
554d6889cc
## Summary Add the missing `--skill` repeatable flag to `actor run` and `actor-run` CLI commands, aligning the implementation with the specification (CLI Synopsis line 277). The flag enables ad-hoc skill injection at runtime without modifying YAML configuration. Closes #887 ## Changes ### DI Container - **`container.py`**: Added `_build_skill_service()` factory and `skill_service` Singleton provider, following the established `_build_*` pattern. Falls back to in-memory `SkillService()` when the database is unavailable. Exception handling narrowed to `(ImportError, OperationalError, DatabaseError, OSError)` with `exc_info=True` for traceability. ### CLI Layer - **`actor.py`**: Added `--skill` Typer option (`list[str] | None`, repeatable, `metavar="NAME"`). Help text notes that skills only augment tool-bearing agents. Wrapped constructor in the existing `try/except` block so `CleverAgentsException` from skill resolution is properly caught. - **`actor_run.py`**: Same `--skill` option with `metavar="NAME"`. Exception handler catches `CleverAgentsException` (matching master — not broadened to `CleverAgentsError`). - **`skill.py`**: Removed module-level `_service` cache. `_get_skill_service()` now always delegates to `get_container().skill_service()` so that `reset_container()` correctly invalidates the cached instance. `_reset_skill_service()` now overrides the container's provider via `providers.Object()`. Removed dead `validate_skill_names()` function. ### Runtime Layer - **`application.py`** (438 lines, down from 625): `ReactiveCleverAgentsApp` gains `skill_names` parameter with automatic deduplication via `dict.fromkeys`. `_resolve_skills()` obtains `SkillService` from the DI container (no CLI layer import). Separate `except KeyError` and `except ValueError` produce distinct error messages (`"not found in registry"` vs `"resolution failed: {exc}"`). Skill tools are only injected into agents that already have tools (`if self._resolved_skill_tools and tools:`), preventing LLM agents from being converted to pass-through `SimpleToolAgent` instances. When skill tools are skipped for tool-less agents, `logger.debug` emits a diagnostic message. `_sanitize_skill_name()` validates skill name format with tightened regex: `^[\w.-]{1,127}/[\w.-]{1,127}$` with `re.ASCII` flag. Zero-tool skill warning now uses `logger.warning` (not `print(stderr)`), ensuring structured log output and proper log-level filtering. - **`graph_executor.py`** (334 lines): Extracted graph execution logic. Type annotations improved. ### Tests - 24+ Behave scenarios across feature files covering: single/multiple/unknown skill flags, skill+context combined, duplicate deduplication, skill resolution, ValueError path, zero-tool resolution, error handling, tool merging, default behavior, overrides, LLM agent guard, `_sanitize_skill_name` edge cases (empty string, too-long name, ANSI escape codes, disallowed characters), `_build_skill_service` happy+fallback paths, `_get_skill_service` container delegation. - CLI "unknown skill" tests for **both** `actor.py` and `actor_run.py` exercise the real error chain (mock only `get_container()`, not the entire `ReactiveCleverAgentsApp`), testing `_resolve_skills()` → `CleverAgentsException` → `except CleverAgentsException` → exit code 2 end-to-end. - Combined skill+context tests assert `ContextManager` was instantiated and `exists()` was called in dedicated **Then** steps. - `@coverage` tags added to all new scenarios. - **Robot Framework smoke tests** added (`robot/skill_actor_run.robot` + `robot/helper_skill_actor_run.py`): unknown-skill error path and valid-skill acceptance path. ### Changelog - Added entry under `## Unreleased` in `CHANGELOG.md`. ## Review Fixes Applied (Brent Edwards, Rounds 1 & 2) | # | Finding | Resolution | |---|---------|------------| | **P1-1** | `print(stderr)` for zero-tool skill warning | **Fixed** — replaced with `logger.warning("Skill '%s' resolved to zero tools", name)`, removed unused `import sys` | | **P2-2** | Skill tools silently skipped for tool-less agents | **Fixed** — added `logger.debug` when skipped; updated `--skill` help text to note "only augments tool-bearing agents" | | **P2-3** | `container.py` at 739 lines | **Acknowledged** — pre-existing growth (+59 lines for `_build_skill_service`); extracting factories is a separate refactoring task | | **P2-4↑** | `CleverAgentsException` → `CleverAgentsError` broadens catch scope | **Fixed** — reverted `actor_run.py` to `except CleverAgentsException` matching master | | **P3-5** | No Robot Framework smoke test for `--skill` | **Fixed** — added `skill_actor_run.robot` with 2 test cases (unknown-skill error, valid-skill acceptance) | | **P3-6** | `GraphExecutor._follow_chained_edges` static-calling-static | **Acknowledged** — cosmetic pattern that doesn't affect correctness; can address in a follow-up | ## Known Limitations / Deferred Items | Item | Reason | |------|--------| | `actor.py` at 679 lines (500-line guideline) | Pre-existing (670 on master), +9 lines for `--skill`. Refactoring the shared `_execute()` closure is a separate task. | | `container.py` at 739 lines (500-line guideline) | Was 680 lines on master, +59 lines for `_build_skill_service()` and `skill_service` provider. Refactoring into sub-modules is a separate task. | | Code duplication between `actor.py` and `actor_run.py` `run()` | ~47 lines identical code. Coupled with the line-count issue above — both require extracting shared execution logic into a helper module. | | `SimpleToolAgent` only executes `tools[0]` | Deferred to #974. Pre-existing architectural limitation, not introduced by this PR. | | `GraphExecutor._follow_chained_edges` static-calling-static pattern | Cosmetic, doesn't affect behavior. | ## Quality Gates - `nox -s lint`: ✅ PASS - `nox -s typecheck`: ✅ PASS (0 errors) - `nox -s unit_tests`: ✅ PASS (11,130 scenarios, 0 failures) - `nox -s integration_tests`: ✅ PASS (1,559 tests, 0 failures) - `nox -s coverage_report`: ✅ 97% (meets threshold) - Branch rebased onto latest `master` (`ab1fd19b`) Reviewed-on: cleveragents/cleveragents-core#971 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
322 lines
12 KiB
Python
322 lines
12 KiB
Python
"""Step definitions for application_container_coverage_boost.feature.
|
|
|
|
Targets the remaining uncovered lines in
|
|
``cleveragents.application.container``:
|
|
|
|
- Lines 187-195: ``_build_checkpoint_service`` function body
|
|
- Lines 204-211: ``_build_trace_service`` function body
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
from unittest.mock import MagicMock
|
|
|
|
from behave import given, then, when
|
|
|
|
from cleveragents.application.container import (
|
|
_build_checkpoint_service,
|
|
_build_session_factory,
|
|
_build_skill_service,
|
|
_build_trace_service,
|
|
reset_container,
|
|
)
|
|
from cleveragents.application.services.checkpoint_service import CheckpointService
|
|
from cleveragents.application.services.skill_service import SkillService
|
|
from cleveragents.application.services.trace_service import TraceService
|
|
from cleveragents.infrastructure.database.llm_trace_repository import (
|
|
LLMTraceRepository,
|
|
)
|
|
from cleveragents.infrastructure.database.repositories import CheckpointRepository
|
|
|
|
_IN_MEMORY_URL = "sqlite:///:memory:"
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Background
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
@given("a clean container state for coverage boost tests")
|
|
def step_clean_container_state(context):
|
|
"""Reset the global container to avoid cross-test interference."""
|
|
reset_container()
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# _build_checkpoint_service scenarios
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
@given("a mock plan lifecycle service")
|
|
def step_mock_plan_lifecycle_service(context):
|
|
"""Create a mock PlanLifecycleService for injection."""
|
|
context.boost_mock_lifecycle = MagicMock()
|
|
context.boost_mock_lifecycle.__class__.__name__ = "PlanLifecycleService"
|
|
|
|
|
|
@when("I build a checkpoint service with an in-memory database URL")
|
|
def step_build_checkpoint_service_default(context):
|
|
"""Call _build_checkpoint_service with only a database URL (no lifecycle)."""
|
|
context.boost_checkpoint_svc = _build_checkpoint_service(_IN_MEMORY_URL)
|
|
|
|
|
|
@when("I build a checkpoint service with the mock plan lifecycle service")
|
|
def step_build_checkpoint_service_with_lifecycle(context):
|
|
"""Call _build_checkpoint_service with a database URL and lifecycle service."""
|
|
context.boost_checkpoint_svc = _build_checkpoint_service(
|
|
_IN_MEMORY_URL,
|
|
plan_lifecycle_service=context.boost_mock_lifecycle,
|
|
)
|
|
|
|
|
|
@then("the result should be a CheckpointService instance")
|
|
def step_verify_checkpoint_service_type(context):
|
|
"""Assert the returned object is a CheckpointService."""
|
|
assert isinstance(context.boost_checkpoint_svc, CheckpointService), (
|
|
f"Expected CheckpointService, got {type(context.boost_checkpoint_svc).__name__}"
|
|
)
|
|
|
|
|
|
@then("the checkpoint service should have a repository")
|
|
def step_verify_checkpoint_has_repository(context):
|
|
"""Assert the service was initialised with a CheckpointRepository."""
|
|
repo = context.boost_checkpoint_svc._repository
|
|
assert repo is not None, "Expected a repository, got None"
|
|
assert isinstance(repo, CheckpointRepository), (
|
|
f"Expected CheckpointRepository, got {type(repo).__name__}"
|
|
)
|
|
|
|
|
|
@then("the checkpoint service should have no plan lifecycle service")
|
|
def step_verify_checkpoint_no_lifecycle(context):
|
|
"""Assert the service has no plan_lifecycle_service (default None)."""
|
|
assert context.boost_checkpoint_svc._plan_lifecycle_service is None
|
|
|
|
|
|
@then("the checkpoint service should reference the mock plan lifecycle service")
|
|
def step_verify_checkpoint_has_lifecycle(context):
|
|
"""Assert the service references the mock lifecycle we injected."""
|
|
assert (
|
|
context.boost_checkpoint_svc._plan_lifecycle_service
|
|
is context.boost_mock_lifecycle
|
|
)
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# _build_trace_service scenarios
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
@given("explicit application settings for trace service")
|
|
def step_explicit_settings_for_trace(context):
|
|
"""Create a real Settings instance to pass explicitly."""
|
|
from cleveragents.config.settings import get_settings
|
|
|
|
context.boost_explicit_settings = get_settings()
|
|
|
|
|
|
@when("I build a trace service with an in-memory database URL and no explicit settings")
|
|
def step_build_trace_service_default(context):
|
|
"""Call _build_trace_service with only a database URL."""
|
|
context.boost_trace_svc = _build_trace_service(_IN_MEMORY_URL)
|
|
|
|
|
|
@when("I build a trace service with the explicit settings")
|
|
def step_build_trace_service_with_settings(context):
|
|
"""Call _build_trace_service with a database URL and explicit settings."""
|
|
context.boost_trace_svc = _build_trace_service(
|
|
_IN_MEMORY_URL,
|
|
settings=context.boost_explicit_settings,
|
|
)
|
|
|
|
|
|
@then("the result should be a TraceService instance")
|
|
def step_verify_trace_service_type(context):
|
|
"""Assert the returned object is a TraceService."""
|
|
assert isinstance(context.boost_trace_svc, TraceService), (
|
|
f"Expected TraceService, got {type(context.boost_trace_svc).__name__}"
|
|
)
|
|
|
|
|
|
@then("the trace service should have a repository")
|
|
def step_verify_trace_has_repository(context):
|
|
"""Assert the service was initialised with an LLMTraceRepository."""
|
|
repo = context.boost_trace_svc._repository
|
|
assert repo is not None, "Expected a repository, got None"
|
|
assert isinstance(repo, LLMTraceRepository), (
|
|
f"Expected LLMTraceRepository, got {type(repo).__name__}"
|
|
)
|
|
|
|
|
|
@then("the trace service should have resolved settings from defaults")
|
|
def step_verify_trace_default_settings(context):
|
|
"""Assert the service has settings resolved via get_settings() fallback."""
|
|
from cleveragents.config.settings import Settings
|
|
|
|
svc_settings = context.boost_trace_svc._settings
|
|
assert svc_settings is not None, "Expected settings, got None"
|
|
assert isinstance(svc_settings, Settings), (
|
|
f"Expected Settings, got {type(svc_settings).__name__}"
|
|
)
|
|
|
|
|
|
@then("the trace service should use the explicitly provided settings")
|
|
def step_verify_trace_explicit_settings(context):
|
|
"""Assert the service uses the exact Settings object we injected."""
|
|
assert context.boost_trace_svc._settings is context.boost_explicit_settings, (
|
|
"Expected the explicitly provided settings instance"
|
|
)
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# _build_session_factory scenarios
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
@when("I build a session factory with an in-memory database URL")
|
|
def step_build_session_factory(context: Any) -> None:
|
|
"""Call _build_session_factory with an in-memory database URL."""
|
|
context.boost_session_factory = _build_session_factory(_IN_MEMORY_URL)
|
|
|
|
|
|
@then("the result should be a callable sessionmaker")
|
|
def step_verify_sessionmaker_callable(context: Any) -> None:
|
|
"""Assert the returned object is a callable sessionmaker."""
|
|
from sqlalchemy.orm import sessionmaker
|
|
|
|
assert isinstance(context.boost_session_factory, sessionmaker), (
|
|
f"Expected sessionmaker, got {type(context.boost_session_factory).__name__}"
|
|
)
|
|
assert callable(context.boost_session_factory)
|
|
|
|
|
|
@then("calling the session factory should produce a Session")
|
|
def step_verify_session_factory_produces_session(context: Any) -> None:
|
|
"""Assert that calling the factory returns a Session instance."""
|
|
from sqlalchemy.orm import Session
|
|
|
|
session = context.boost_session_factory()
|
|
try:
|
|
assert isinstance(session, Session), (
|
|
f"Expected Session, got {type(session).__name__}"
|
|
)
|
|
finally:
|
|
session.close()
|
|
|
|
|
|
@when("I build a session factory with an invalid database URL")
|
|
def step_build_session_factory_invalid(context: Any) -> None:
|
|
"""Call _build_session_factory with an invalid URL — error deferred to usage."""
|
|
# SQLAlchemy lazily connects; the factory itself succeeds, but
|
|
# calling it to create a session and executing SQL raises.
|
|
context.boost_session_factory = _build_session_factory(
|
|
"sqlite:///nonexistent/path/to/db.sqlite"
|
|
)
|
|
|
|
|
|
@then("a database error should be raised when the factory is called")
|
|
def step_verify_session_factory_error(context: Any) -> None:
|
|
"""Assert that using the factory with an invalid URL raises an error."""
|
|
from sqlalchemy import text as sa_text
|
|
from sqlalchemy.exc import OperationalError
|
|
|
|
session = context.boost_session_factory()
|
|
try:
|
|
# Force a connection attempt by executing a trivial query.
|
|
session.execute(sa_text("SELECT 1"))
|
|
raise AssertionError("Expected OperationalError, but no error was raised")
|
|
except OperationalError:
|
|
pass # Expected
|
|
finally:
|
|
session.close()
|
|
|
|
|
|
@when("I resolve session_factory from the container with an in-memory database URL")
|
|
def step_resolve_container_session_factory(context: Any) -> None:
|
|
"""Resolve session_factory from the DI container using an in-memory URL."""
|
|
from unittest.mock import patch
|
|
|
|
with patch.dict("os.environ", {"CLEVERAGENTS_DATABASE_URL": _IN_MEMORY_URL}):
|
|
from cleveragents.application.container import get_container
|
|
|
|
reset_container()
|
|
try:
|
|
container = get_container()
|
|
context.boost_resolved_factory = container.session_factory()
|
|
finally:
|
|
reset_container()
|
|
|
|
|
|
@then("the resolved session factory should be callable")
|
|
def step_verify_resolved_factory_callable(context: Any) -> None:
|
|
"""Assert the container-resolved factory is callable."""
|
|
assert callable(context.boost_resolved_factory), (
|
|
"Expected a callable session factory from the container"
|
|
)
|
|
|
|
|
|
@then("calling the resolved session factory should produce a Session")
|
|
def step_verify_resolved_factory_produces_session(context: Any) -> None:
|
|
"""Assert that calling the resolved factory returns a Session instance."""
|
|
from sqlalchemy.orm import Session
|
|
|
|
session = context.boost_resolved_factory()
|
|
try:
|
|
assert isinstance(session, Session), (
|
|
f"Expected Session, got {type(session).__name__}"
|
|
)
|
|
finally:
|
|
session.close()
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# _build_skill_service scenarios
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
@when("I build a skill service with an in-memory database URL")
|
|
def step_build_skill_service_default(context):
|
|
"""Call _build_skill_service with a valid in-memory database URL."""
|
|
context.boost_skill_svc = _build_skill_service(_IN_MEMORY_URL)
|
|
|
|
|
|
@when("I build a skill service with an invalid database URL")
|
|
def step_build_skill_service_invalid(context):
|
|
"""Mock create_engine to raise OperationalError, triggering the fallback."""
|
|
from unittest.mock import patch
|
|
|
|
from sqlalchemy.exc import OperationalError
|
|
|
|
with patch(
|
|
"sqlalchemy.create_engine",
|
|
side_effect=OperationalError("mock", {}, Exception("DB unavailable")),
|
|
):
|
|
context.boost_skill_svc = _build_skill_service(
|
|
"sqlite:////nonexistent/path/db.sqlite"
|
|
)
|
|
|
|
|
|
@then("the result should be a SkillService instance")
|
|
def step_verify_skill_service_type(context):
|
|
"""Assert the returned object is a SkillService."""
|
|
assert isinstance(context.boost_skill_svc, SkillService), (
|
|
f"Expected SkillService, got {type(context.boost_skill_svc).__name__}"
|
|
)
|
|
|
|
|
|
@then("the skill service should have a repository")
|
|
def step_verify_skill_has_repository(context):
|
|
"""Assert the service was initialised with a SkillRepository."""
|
|
assert context.boost_skill_svc._skill_repo is not None, (
|
|
"Expected a SkillRepository, got None"
|
|
)
|
|
|
|
|
|
@then("the skill service should have no repository")
|
|
def step_verify_skill_no_repository(context):
|
|
"""Assert the fallback service has no DB-backed repository."""
|
|
assert context.boost_skill_svc._skill_repo is None, (
|
|
"Expected no SkillRepository for in-memory fallback"
|
|
)
|