fix(actor): validate v3 YAML via ActorConfigSchema in agents actor add CLI
CI / lint (pull_request) Successful in 17s
CI / push-validation (pull_request) Successful in 20s
CI / helm (pull_request) Successful in 22s
CI / quality (pull_request) Successful in 53s
CI / typecheck (pull_request) Successful in 57s
CI / security (pull_request) Successful in 58s
CI / e2e_tests (pull_request) Successful in 3m13s
CI / build (pull_request) Successful in 3m20s
CI / unit_tests (pull_request) Successful in 5m16s
CI / docker (pull_request) Successful in 1m32s
CI / integration_tests (pull_request) Successful in 7m7s
CI / coverage (pull_request) Successful in 12m43s
CI / status-check (pull_request) Successful in 1s

Adds full v3 YAML schema validation to the 'agents actor add --config'
and 'agents actor update --config' CLI commands. When a config blob contains
a 'type' field (or version '3.0'/'3'), the CLI now validates it through
ActorConfigSchema before the registry touches it.

Key changes:

- schema.py: Add is_v3_yaml() as the single canonical detection function.
  Detection now treats ANY 'type' field as v3 (any value — schema validates
  the enum), removing the gap where invalid type values like 'robot' bypassed
  validation (C1). Add tool string namespace validation: tool references must
  follow 'namespace/tool_name' format (M7). Expose is_v3_yaml in __all__.

- actor.py CLI: Import is_v3_yaml from schema (removing local duplicate —
  M3 DRY fix). Remove _is_v3_yaml local function. Update _validate_v3_yaml
  to use ActorConfigSchema.model_validate(config_blob) instead of
  from_yaml_file() — eliminates TOCTOU (M1). Narrow except clauses from
  broad Exception to pydantic.ValidationError, yaml.YAMLError, OSError (m2,
  n2). Change return type to None (m3). Add v3 validation gate to update()
  command (M2).

- registry.py: Replace self._is_v3_yaml() with imported is_v3_yaml() from
  schema — removes the second copy of the detection logic (M3). Replace
  except ValueError with except pydantic.ValidationError (n2).

- BDD steps: Replace direct calls to _is_v3_yaml()/_validate_v3_yaml() with
  real CLI invocation via typer.testing.CliRunner (C2). Add
  context.add_cleanup() for temp directory (m1). Assert registered_actor_type
  from registry call kwargs — non-tautological (M6).

- Feature file: Replace misleading '<config_file>' step name with a clear
  descriptive step (M8). Add three new DoD scenarios: invalid node ID format,
  unnamespaced tool name, unreachable node detection (M7).

- Robot helper: Replace internal function calls with subprocess.run(['agents',
  'actor', 'add', ...]) to test the actual CLI (C3).

- CHANGELOG.md: Preserve all existing master entries and append the #5869
  entry (C4/M4 data-loss regression fix).

- CONTRIBUTORS.md: Preserve all existing master entries and add Rui Hu
  contribution entry (M4).

ISSUES CLOSED: #5869
This commit is contained in:
2026-04-16 06:15:17 +00:00
parent bb97f1450e
commit 740996b83f
9 changed files with 1011 additions and 2 deletions
+7
View File
@@ -7,6 +7,13 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Fixed
- **Actor v3 YAML Schema Validation in CLI** (#5869): The `agents actor add --config`
command now validates v3 YAML files using `ActorConfigSchema`, ensuring proper
schema compliance including cycle detection for GRAPH actors, required field
validation, and enum validation. v3 YAML is detected by `type: llm|tool|graph`
or `version: "3.0"`. Invalid v3 actors are rejected with clear error messages
before registration.
- **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
`features/environment.py` now emits its non-assertion exception guard warning to
both the structured logger and `stderr` via a new `_warning_with_stderr` helper.
+1
View File
@@ -19,5 +19,6 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
* HAL 9000 has contributed the plan concurrency race-condition fix (#7989): wired `LockService` into the plan lifecycle, guarding `execute_plan()` and `apply_plan()` with plan-level advisory locks and unique per-invocation owner identities to prevent silent concurrent state corruption.
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
* Rui Hu has contributed the v3 actor YAML schema validation fix (#5869): added `ActorConfigSchema` validation to the `agents actor add --config` CLI command, covering cycle detection, required field validation, and enum validation for v3 YAML actor definitions.
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
@@ -0,0 +1,135 @@
Feature: Actor add CLI validates v3 YAML via ActorConfigSchema
As a CleverAgents v3 user
I want `agents actor add --config` to validate v3 YAML files using ActorConfigSchema
So that invalid v3 actor definitions (cycles, missing fields, etc.) are rejected at registration time
Background:
Given an actor CLI runner
And a temporary directory for test actor configs
# ────────────────────────────────────────────────────────────
# Valid v3 Actor Registration
# ────────────────────────────────────────────────────────────
Scenario: Register a valid v3 LLM actor via CLI
Given a v3 LLM actor YAML file with name "local/test-llm"
When I run the actor add command for "local/test-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "llm"
Scenario: Register a valid v3 TOOL actor via CLI
Given a v3 TOOL actor YAML file with name "local/test-tool"
When I run the actor add command for "local/test-tool" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "tool"
Scenario: Register a valid v3 GRAPH actor via CLI
Given a v3 GRAPH actor YAML file with name "local/test-graph"
When I run the actor add command for "local/test-graph" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "graph"
# ────────────────────────────────────────────────────────────
# v3 Schema Validation Failures
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor with cycle in route
Given a v3 GRAPH actor YAML file with a cycle in the route
When I run the actor add command for "local/broken-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "cycle"
And v3actor the error message should contain "nodes:"
Scenario: Reject v3 LLM actor without model field
Given a v3 LLM actor YAML file without the model field
When I run the actor add command for "local/no-model-llm" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "LLM actors require 'model' field"
Scenario: Reject v3 TOOL actor without tools field
Given a v3 TOOL actor YAML file without the tools field
When I run the actor add command for "local/no-tools-tool" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "TOOL actors require at least one tool"
Scenario: Reject v3 GRAPH actor without route field
Given a v3 GRAPH actor YAML file without the route field
When I run the actor add command for "local/no-route-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "GRAPH actors require 'route' field"
Scenario: Reject v3 GRAPH actor without model field
Given a v3 GRAPH actor YAML file without the model field
When I run the actor add command for "local/no-model-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "GRAPH actors require 'model' field"
Scenario: Reject v3 actor with invalid type enum
Given a v3 actor YAML file with invalid type "invalid_type"
When I run the actor add command for "local/invalid-type" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "type"
Scenario: Reject v3 actor with invalid context_view enum
Given a v3 LLM actor YAML file with invalid context_view "invalid_view"
When I run the actor add command for "local/invalid-view" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "context_view"
# ────────────────────────────────────────────────────────────
# Definition of Done: Node ID format validation
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor with invalid node ID format
Given a v3 GRAPH actor YAML file with an invalid node ID
When I run the actor add command for "local/bad-node-id" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "node"
# ────────────────────────────────────────────────────────────
# Definition of Done: Tool name namespace validation
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 TOOL actor with improperly namespaced tool name
Given a v3 TOOL actor YAML file with an unnamespaced tool name
When I run the actor add command for "local/bad-tool-name" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "tool"
# ────────────────────────────────────────────────────────────
# Definition of Done: Unreachable node detection
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor with unreachable node
Given a v3 GRAPH actor YAML file with an unreachable node
When I run the actor add command for "local/unreachable-node" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "unreachable"
# ────────────────────────────────────────────────────────────
# v3 Detection and Backward Compatibility
# ────────────────────────────────────────────────────────────
Scenario: Detect v3 YAML by type field (llm)
Given a v3 LLM actor YAML file with name "local/detect-llm"
When I run the actor add command for "local/detect-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Detect v3 YAML by type field (tool)
Given a v3 TOOL actor YAML file with name "local/detect-tool"
When I run the actor add command for "local/detect-tool" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Detect v3 YAML by type field (graph)
Given a v3 GRAPH actor YAML file with name "local/detect-graph"
When I run the actor add command for "local/detect-graph" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Detect v3 YAML by version field
Given a v3 actor YAML file with version "3.0"
When I run the actor add command for "local/detect-version" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
@@ -0,0 +1,499 @@
"""Step definitions for actor add CLI v3 schema validation.
Tests for features/actor_add_v3_schema_validation.feature validates that
the `agents actor add --config` command validates v3 YAML files using
ActorConfigSchema, detecting cycles, missing fields, and invalid enums.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch
from behave import given, then, when
from behave.runner import Context
from typer.testing import CliRunner
from cleveragents.cli.commands.actor import app as actor_app
from cleveragents.domain.models.core.actor import Actor
# ────────────────────────────────────────────────────────────
# Background Steps
# ────────────────────────────────────────────────────────────
@given("a temporary directory for test actor configs")
def step_temp_dir(context: Context) -> None:
"""Ensure temporary directory exists and initialize v3 test context."""
if not hasattr(context, "temp_dir"):
context.temp_dir = tempfile.TemporaryDirectory()
context.add_cleanup(context.temp_dir.cleanup)
if not hasattr(context, "config_file"):
context.config_file = None
if not hasattr(context, "command_result"):
context.command_result = None
if not hasattr(context, "command_error"):
context.command_error = None
if not hasattr(context, "actor_registered"):
context.actor_registered = False
# ────────────────────────────────────────────────────────────
# v3 LLM Actor YAML Fixtures
# ────────────────────────────────────────────────────────────
@given('a v3 LLM actor YAML file with name "{name}"')
def step_v3_llm_actor(context: Context, name: str) -> None:
"""Create a valid v3 LLM actor YAML file."""
yaml_content = f"""\
name: {name}
type: llm
description: Test LLM actor
provider: openai
model: gpt-4
system_prompt: "You are a helpful assistant"
"""
context.config_file = _write_yaml_file(context, name, yaml_content)
@given("a v3 LLM actor YAML file without the model field")
def step_v3_llm_actor_no_model(context: Context) -> None:
"""Create a v3 LLM actor YAML without model field."""
yaml_content = """\
name: local/no-model-llm
type: llm
description: LLM without model
provider: openai
system_prompt: "You are helpful"
"""
context.config_file = _write_yaml_file(context, "no-model-llm", yaml_content)
@given('a v3 LLM actor YAML file with invalid context_view "{invalid_view}"')
def step_v3_llm_actor_invalid_context_view(context: Context, invalid_view: str) -> None:
"""Create a v3 LLM actor with invalid context_view."""
yaml_content = f"""\
name: local/invalid-view
type: llm
description: LLM with invalid context_view
provider: openai
model: gpt-4
context_view: {invalid_view}
"""
context.config_file = _write_yaml_file(context, "invalid-view", yaml_content)
# ────────────────────────────────────────────────────────────
# v3 TOOL Actor YAML Fixtures
# ────────────────────────────────────────────────────────────
@given('a v3 TOOL actor YAML file with name "{name}"')
def step_v3_tool_actor(context: Context, name: str) -> None:
"""Create a valid v3 TOOL actor YAML file."""
yaml_content = f"""\
name: {name}
type: tool
description: Test TOOL actor
provider: openai
model: gpt-4
tools:
- files/read_file
- files/write_file
"""
context.config_file = _write_yaml_file(context, name, yaml_content)
@given("a v3 TOOL actor YAML file without the tools field")
def step_v3_tool_actor_no_tools(context: Context) -> None:
"""Create a v3 TOOL actor YAML without tools field."""
yaml_content = """\
name: local/no-tools-tool
type: tool
description: TOOL without tools
provider: openai
model: gpt-4
"""
context.config_file = _write_yaml_file(context, "no-tools-tool", yaml_content)
# ────────────────────────────────────────────────────────────
# v3 GRAPH Actor YAML Fixtures
# ────────────────────────────────────────────────────────────
@given('a v3 GRAPH actor YAML file with name "{name}"')
def step_v3_graph_actor(context: Context, name: str) -> None:
"""Create a valid v3 GRAPH actor YAML file."""
yaml_content = f"""\
name: {name}
type: graph
description: Test GRAPH actor
provider: openai
model: gpt-4
route:
nodes:
- id: node_a
type: agent
name: Node A
description: First node
config: {{}}
- id: node_b
type: agent
name: Node B
description: Second node
config: {{}}
edges:
- from_node: node_a
to_node: node_b
entry_node: node_a
exit_nodes:
- node_b
"""
context.config_file = _write_yaml_file(context, name, yaml_content)
@given("a v3 GRAPH actor YAML file with a cycle in the route")
def step_v3_graph_actor_with_cycle(context: Context) -> None:
"""Create a v3 GRAPH actor YAML with a cycle."""
yaml_content = """\
name: local/broken-graph
type: graph
description: Graph with cycle
provider: openai
model: gpt-4
route:
nodes:
- id: node_a
type: agent
name: Node A
description: First node
config: {}
- id: node_b
type: agent
name: Node B
description: Second node
config: {}
edges:
- from_node: node_a
to_node: node_b
- from_node: node_b
to_node: node_a
entry_node: node_a
exit_nodes:
- node_b
"""
context.config_file = _write_yaml_file(context, "broken-graph", yaml_content)
@given("a v3 GRAPH actor YAML file without the route field")
def step_v3_graph_actor_no_route(context: Context) -> None:
"""Create a v3 GRAPH actor YAML without route field."""
yaml_content = """\
name: local/no-route-graph
type: graph
description: GRAPH without route
provider: openai
model: gpt-4
"""
context.config_file = _write_yaml_file(context, "no-route-graph", yaml_content)
@given("a v3 GRAPH actor YAML file without the model field")
def step_v3_graph_actor_no_model(context: Context) -> None:
"""Create a v3 GRAPH actor YAML without model field."""
yaml_content = """\
name: local/no-model-graph
type: graph
description: GRAPH without model
provider: openai
route:
nodes:
- id: node_a
type: agent
name: Node A
description: First node
config: {}
edges: []
entry_node: node_a
exit_nodes:
- node_a
"""
context.config_file = _write_yaml_file(context, "no-model-graph", yaml_content)
# ────────────────────────────────────────────────────────────
# v3 Detection and Invalid Type Fixtures
# ────────────────────────────────────────────────────────────
@given('a v3 actor YAML file with invalid type "{invalid_type}"')
def step_v3_actor_invalid_type(context: Context, invalid_type: str) -> None:
"""Create a v3 actor YAML with invalid type."""
yaml_content = f"""\
name: local/invalid-type
type: {invalid_type}
description: Actor with invalid type
provider: openai
model: gpt-4
"""
context.config_file = _write_yaml_file(context, "invalid-type", yaml_content)
@given('a v3 actor YAML file with version "{version}"')
def step_v3_actor_with_version(context: Context, version: str) -> None:
"""Create a v3 actor YAML with explicit version field."""
yaml_content = f"""\
name: local/detect-version
type: llm
description: Actor with version field
provider: openai
version: "{version}"
model: gpt-4
"""
context.config_file = _write_yaml_file(context, "detect-version", yaml_content)
@given("a v3 GRAPH actor YAML file with an invalid node ID")
def step_v3_graph_actor_invalid_node_id(context: Context) -> None:
"""Create a v3 GRAPH actor YAML with an invalid node ID format.
Node IDs must be non-empty strings. An empty string node ID triggers
the schema validator's node ID format check.
"""
yaml_content = """\
name: local/bad-node-id
type: graph
description: Graph with invalid node ID
provider: openai
model: gpt-4
route:
nodes:
- id: ""
type: agent
name: Empty ID Node
description: Node with empty ID
config: {}
edges: []
entry_node: ""
exit_nodes:
- ""
"""
context.config_file = _write_yaml_file(context, "bad-node-id", yaml_content)
@given("a v3 TOOL actor YAML file with an unnamespaced tool name")
def step_v3_tool_actor_unnamespaced_tool(context: Context) -> None:
"""Create a v3 TOOL actor YAML with an improperly namespaced tool name.
Tool names must follow the ``namespace/tool_name`` format. A bare tool
name without a namespace prefix violates the convention and must be
rejected by schema validation.
"""
yaml_content = """\
name: local/bad-tool-name
type: tool
description: TOOL with unnamespaced tool name
provider: openai
model: gpt-4
tools:
- read_file
"""
context.config_file = _write_yaml_file(context, "bad-tool-name", yaml_content)
@given("a v3 GRAPH actor YAML file with an unreachable node")
def step_v3_graph_actor_unreachable_node(context: Context) -> None:
"""Create a v3 GRAPH actor YAML with an unreachable node.
A node that has no incoming edges and is not the entry_node is unreachable
from the graph's entry point and must be rejected.
"""
yaml_content = """\
name: local/unreachable-node
type: graph
description: Graph with unreachable node
provider: openai
model: gpt-4
route:
nodes:
- id: node_a
type: agent
name: Entry Node
description: Entry node
config: {}
- id: node_b
type: agent
name: Exit Node
description: Exit node
config: {}
- id: node_orphan
type: agent
name: Orphan Node
description: Unreachable node (no incoming edges)
config: {}
edges:
- from_node: node_a
to_node: node_b
entry_node: node_a
exit_nodes:
- node_b
"""
context.config_file = _write_yaml_file(context, "unreachable-node", yaml_content)
# ────────────────────────────────────────────────────────────
# When Steps (Command Execution via CLI runner)
# ────────────────────────────────────────────────────────────
@when('I run the actor add command for "{name}" with the prepared config file')
def step_run_actor_add(context: Context, name: str) -> None:
"""Run the agents actor add command via the real CLI (Typer CliRunner).
Uses typer.testing.CliRunner to invoke the actual CLI entrypoint the
same code path the user runs. The registry is mocked so that the test
does not require a real database, but all CLI-level validation (including
v3 schema validation) runs for real.
"""
if context.config_file is None:
raise ValueError("No config file set")
runner = getattr(context, "runner", CliRunner())
mock_actor = Actor(
id=1,
name=name,
provider="openai",
model="gpt-4",
config_blob={"name": name, "provider": "openai", "model": "gpt-4"},
config_hash=Actor.compute_hash(
{"name": name, "provider": "openai", "model": "gpt-4"}
),
graph_descriptor=None,
unsafe=False,
is_built_in=False,
is_default=False,
)
mock_registry = MagicMock()
mock_registry.upsert_actor.return_value = mock_actor
mock_registry.get_actor.side_effect = __import__(
"cleveragents.core.exceptions", fromlist=["NotFoundError"]
).NotFoundError("not found")
mock_service = MagicMock()
with patch("cleveragents.cli.commands.actor._get_services") as mock_get_services:
mock_get_services.return_value = (mock_service, mock_registry)
result = runner.invoke(
actor_app,
["add", name, "--config", str(context.config_file)],
catch_exceptions=True,
)
context.cli_result = result
success = result.exit_code == 0
context.command_result = {
"success": success,
"exit_code": result.exit_code,
"output": result.output,
}
if success:
context.command_error = None
context.actor_registered = True
# Verify registry was called and extract the actor_type from the config
if mock_registry.upsert_actor.called:
call_kwargs = mock_registry.upsert_actor.call_args
blob = call_kwargs.kwargs.get("config_blob") if call_kwargs.kwargs else None
context.registered_actor_type = (
blob.get("type") if isinstance(blob, dict) else None
)
else:
context.registered_actor_type = None
else:
context.command_error = result.output
context.actor_registered = False
context.registered_actor_type = None
# ────────────────────────────────────────────────────────────
# Then Steps (Assertions) — prefixed with "v3actor" to avoid conflicts
# ────────────────────────────────────────────────────────────
@then("v3actor the command should succeed")
def step_command_success(context: Context) -> None:
"""Assert that the command succeeded."""
assert context.command_result is not None, "No command result"
assert context.command_result.get("success", False), (
f"Command failed (exit {context.command_result.get('exit_code')}): "
f"{context.command_result.get('output', '')}"
)
@then("v3actor the command should fail")
def step_command_fail(context: Context) -> None:
"""Assert that the command failed."""
assert context.command_result is not None, "No command result"
assert not context.command_result.get("success", True), (
f"Command should have failed but succeeded with output: "
f"{context.command_result.get('output', '')}"
)
@then('v3actor the error message should contain "{text}"')
def step_error_contains(context: Context, text: str) -> None:
"""Assert that the error output contains specific text."""
output = context.command_result.get("output", "") if context.command_result else ""
assert text.lower() in output.lower(), (
f"Output does not contain '{text}':\n{output}"
)
@then('the actor should be registered with type "{actor_type}"')
def step_actor_registered_with_type(context: Context, actor_type: str) -> None:
"""Assert that the actor was registered with the correct type.
Checks the config_blob passed to registry.upsert_actor() to verify the
type field matches not the mock return value.
"""
assert context.actor_registered, (
f"Actor was not registered. Output: "
f"{context.command_result.get('output', '') if context.command_result else ''}"
)
registered_type = getattr(context, "registered_actor_type", None)
assert registered_type == actor_type, (
f"Expected actor type '{actor_type}' but got '{registered_type}'"
)
@then("the actor should be validated via ActorConfigSchema")
def step_actor_validated_via_schema(context: Context) -> None:
"""Assert that the actor was validated via ActorConfigSchema.
The CLI's v3 validation gate runs ActorConfigSchema.model_validate() before
reaching the registry. If the command succeeded, the schema was applied.
"""
assert context.actor_registered, (
f"Actor was not registered. Output: "
f"{context.command_result.get('output', '') if context.command_result else ''}"
)
# ────────────────────────────────────────────────────────────
# Helper Functions
# ────────────────────────────────────────────────────────────
def _write_yaml_file(context: Context, name: str, content: str) -> Path:
"""Write a YAML file to the temporary directory."""
if not hasattr(context, "temp_dir") or context.temp_dir is None:
context.temp_dir = tempfile.TemporaryDirectory()
context.add_cleanup(context.temp_dir.cleanup)
# Use just the basename to avoid creating subdirectories
safe_name = name.replace("/", "_")
file_path = Path(context.temp_dir.name) / f"{safe_name}.yaml"
file_path.write_text(content)
return file_path
+177
View File
@@ -0,0 +1,177 @@
*** Settings ***
Documentation Integration tests for actor add CLI v3 schema validation
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER} ${CURDIR}/helper_actor_add_v3_schema_validation.py
*** Test Cases ***
Register Valid v3 LLM Actor Via CLI
[Documentation] Register a valid v3 LLM actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_llm.yaml
Create File ${yaml_file} name: local/test-llm\ntype: llm\ndescription: Test LLM\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/test-llm ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Register Valid v3 TOOL Actor Via CLI
[Documentation] Register a valid v3 TOOL actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_tool.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/test-tool
... type: tool
... description: Test TOOL
... tools:
... ${SPACE}${SPACE}- files/read_file
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/test-tool ${yaml_file} cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Register Valid v3 GRAPH Actor Via CLI
[Documentation] Register a valid v3 GRAPH actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_graph.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/test-graph
... type: graph
... description: Test GRAPH
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/test-graph ${yaml_file} cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Reject v3 GRAPH Actor With Cycle In Route
[Documentation] Reject v3 GRAPH actor with cycle in route via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}broken_graph_cycle.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/broken-graph
... type: graph
... description: Graph with cycle
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node B
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Second node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges:
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_a
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/broken-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} cycle
Reject v3 LLM Actor Without Model Field
[Documentation] Reject v3 LLM actor without model field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_llm_no_model.yaml
Create File ${yaml_file} name: local/no-model-llm\ntype: llm\ndescription: LLM without model\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-llm ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} model
Reject v3 TOOL Actor Without Tools Field
[Documentation] Reject v3 TOOL actor without tools field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_tool_no_tools.yaml
Create File ${yaml_file} name: local/no-tools-tool\ntype: tool\ndescription: TOOL without tools\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-tools-tool ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} tool
Reject v3 GRAPH Actor Without Route Field
[Documentation] Reject v3 GRAPH actor without route field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_route.yaml
Create File ${yaml_file} name: local/no-route-graph\ntype: graph\ndescription: GRAPH without route\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-route-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} route
Reject v3 GRAPH Actor Without Model Field
[Documentation] Reject v3 GRAPH actor without model field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_model.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/no-model-graph
... type: graph
... description: GRAPH without model
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} model
Reject v3 Actor With Invalid Type Enum
[Documentation] Reject v3 actor with invalid type enum via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_type.yaml
Create File ${yaml_file} name: local/invalid-type\ntype: invalid_type\ndescription: Invalid type\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-type ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} type
Reject v3 Actor With Invalid Context View Enum
[Documentation] Reject v3 actor with invalid context_view enum via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_context_view.yaml
Create File ${yaml_file} name: local/invalid-view\ntype: llm\ndescription: Invalid context_view\nmodel: gpt-4\ncontext_view: invalid_view\n
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-view ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} context_view
+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env python3
"""Helper script for actor add v3 schema validation robot tests.
Tests the agents actor add CLI command with v3 YAML files to ensure
ActorConfigSchema validation is applied (cycle detection, required fields, etc.).
This helper delegates to the real CLI binary so that it exercises the actual
command path not internal helpers.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
def add_actor(actor_name: str, config_file: str) -> int:
"""Test adding an actor via the real CLI binary.
Invokes ``agents actor add <actor_name> --config <config_file>`` as a
subprocess so that all CLI-level validation (including v3 schema
validation) runs for real.
Args:
actor_name: The actor name (e.g., local/test-llm)
config_file: Path to the YAML config file
Returns:
0 on success, 1 on validation failure
"""
config_path = Path(config_file)
if not config_path.exists():
print(
f"actor-add-error: Config file not found: {config_file}",
file=sys.stderr,
)
return 1
result = subprocess.run(
["agents", "actor", "add", actor_name, "--config", config_file],
capture_output=True,
text=True,
)
if result.returncode == 0:
print("actor-add-success")
return 0
else:
output = result.stdout + result.stderr
print(f"actor-add-error: {output.strip()}", file=sys.stderr)
return 1
def main() -> int:
"""Main entry point."""
if len(sys.argv) < 3:
print(
"Usage: helper_actor_add_v3_schema_validation.py"
" <command> <actor_name> <config_file>"
)
return 1
command = sys.argv[1]
actor_name = sys.argv[2]
config_file = sys.argv[3]
if command == "add":
return add_actor(actor_name, config_file)
else:
print(f"Unknown command: {command}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+32 -1
View File
@@ -5,7 +5,10 @@ from __future__ import annotations
from dataclasses import asdict
from typing import Any
import pydantic
from cleveragents.actor.config import ActorConfiguration
from cleveragents.actor.schema import ActorConfigSchema, is_v3_yaml
from cleveragents.application.services.actor_service import ActorService
from cleveragents.config.settings import Settings
from cleveragents.core.exceptions import ValidationError
@@ -189,6 +192,10 @@ class ActorRegistry:
The YAML is parsed into an ``ActorConfiguration``, validated, and
persisted alongside the original *yaml_text* and *schema_version*.
For v3 YAML files (detected by type: llm|tool|graph or version: "3.0"),
the configuration is validated using ActorConfigSchema to ensure
proper schema compliance, including cycle detection for GRAPH actors.
Args:
yaml_text: The original actor YAML source.
update: When ``True`` allow overwriting an existing actor.
@@ -210,6 +217,15 @@ class ActorRegistry:
if not name_raw:
raise ValidationError("Actor YAML must include a 'name' field.")
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
# This ensures v3 actors are validated against the full schema, including
# cycle detection, required fields, and enum validation.
if is_v3_yaml(blob):
try:
ActorConfigSchema.model_validate(blob)
except pydantic.ValidationError as exc:
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
name = self._ensure_namespaced(name_raw)
provider_raw = blob.get("provider") or blob.get("provider_type", "")
model_raw = blob.get("model") or blob.get("model_id", "")
@@ -273,9 +289,24 @@ class ActorRegistry:
schema_version: str | None = None,
compiled_metadata: dict[str, Any] | None = None,
) -> Actor:
"""Parse, validate, and persist an actor configuration."""
"""Parse, validate, and persist an actor configuration.
For v3 YAML files (detected by type: llm|tool|graph or version: "3.0"),
the configuration is validated using ActorConfigSchema to ensure
proper schema compliance, including cycle detection for GRAPH actors.
"""
self.ensure_built_in_actors()
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
# This ensures v3 actors are validated against the full schema, including
# cycle detection, required fields, and enum validation.
if config_blob and is_v3_yaml(config_blob):
try:
ActorConfigSchema.model_validate(config_blob)
except pydantic.ValidationError as exc:
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
config = ActorConfiguration.from_blob(
blob=config_blob,
name=name,
+40
View File
@@ -831,6 +831,17 @@ class ActorConfigSchema(BaseModel):
msg = "TOOL actors require at least one tool"
raise ValueError(msg)
# Validate string tool references: must follow "namespace/name" format.
# ToolDefinition instances are validated separately by their own field
# validators.
for tool_ref in self.tools:
if isinstance(tool_ref, str) and "/" not in tool_ref:
msg = (
f"tool '{tool_ref}' must be namespaced "
f"(namespace/name format, e.g. 'files/read_file')"
)
raise ValueError(msg)
# GRAPH actors require model and route
if self.type == ActorType.GRAPH:
if not self.model:
@@ -906,6 +917,34 @@ class ActorConfigSchema(BaseModel):
)
def is_v3_yaml(config_blob: dict[str, Any] | None) -> bool:
"""Detect if a config blob is v3 YAML.
A config blob is considered v3 YAML if it contains ANY ``type`` field
(any value let the schema validate whether the value is legal) or a
``version`` field set to ``"3.0"`` or ``"3"``.
Treating any ``type`` field as a v3 signal prevents configs with invalid
type values (e.g. ``type: robot``) from bypassing schema validation.
Args:
config_blob: The parsed YAML/JSON configuration dictionary.
Returns:
True if the config appears to be v3 YAML, False otherwise.
"""
if not config_blob:
return False
# Any 'type' field = v3 YAML; let the schema validate the value
if "type" in config_blob:
return True
# Check for version field indicating v3
version = config_blob.get("version")
return bool(version and str(version) in {"3.0", "3"})
__all__ = [
"ActorConfigSchema",
"ActorType",
@@ -923,4 +962,5 @@ __all__ = [
"ToolParameter",
"ToolSourceRef",
"actor_role_warnings",
"is_v3_yaml",
]
+44 -1
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from typing import Annotated, Any, cast
import click
import pydantic
import typer
import yaml
from rich.console import Console
@@ -13,7 +14,7 @@ from rich.panel import Panel
from rich.table import Table
from cleveragents.actor.config import ActorConfiguration
from cleveragents.actor.schema import actor_role_warnings
from cleveragents.actor.schema import ActorConfigSchema, actor_role_warnings, is_v3_yaml
from cleveragents.application.container import get_container
from cleveragents.cli.commands._resolve_actor import (
resolve_config_files as _resolve_config_files,
@@ -345,6 +346,36 @@ def _parse_option_overrides(option_values: list[str] | None) -> dict[str, Any]:
return overrides
def _validate_v3_yaml(config_path: Path, config_blob: dict[str, Any]) -> None:
"""Validate a v3 YAML config using ActorConfigSchema.
Uses the already-parsed ``config_blob`` to avoid re-reading the file
(eliminates TOCTOU: the config the CLI parsed is the config that gets
validated, not a potentially-different file re-read at validation time).
Args:
config_path: Path to the config file (used only in error messages).
config_blob: The parsed YAML/JSON configuration dictionary.
Raises:
typer.BadParameter: If schema validation fails.
"""
try:
ActorConfigSchema.model_validate(config_blob)
except pydantic.ValidationError as exc:
raise typer.BadParameter(
f"Invalid actor configuration in {config_path}: {exc}"
) from exc
except yaml.YAMLError as exc:
raise typer.BadParameter(
f"Failed to parse actor configuration in {config_path}: {exc}"
) from exc
except OSError as exc:
raise typer.BadParameter(
f"Cannot read config file {config_path}: {exc}"
) from exc
def _canonicalize_actor_config(
*,
name: str,
@@ -570,6 +601,12 @@ def add(
assert loaded is not None, "unreachable: config is not None"
yaml_text, config_blob = loaded
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────────
# This ensures v3 actors are validated against the full schema, including
# cycle detection, required fields, and enum validation.
if is_v3_yaml(config_blob):
_validate_v3_yaml(config, config_blob)
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
name=name,
config_blob=config_blob,
@@ -713,6 +750,12 @@ def update(
if not option_overrides:
option_overrides = None
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────────
# Apply the same schema gate as the add command: if the updated config is
# v3, validate it fully before writing to the registry.
if config is not None and is_v3_yaml(new_config):
_validate_v3_yaml(config, new_config)
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
name=name,
config_blob=new_config,