fix(actor): validate v3 YAML via ActorConfigSchema in agents actor add CLI #8636

Merged
hurui200320 merged 1 commits from fix/actor-add-v3-schema-validation into master 2026-04-17 08:58:34 +00:00
22 changed files with 2178 additions and 36 deletions
+28
View File
@@ -7,6 +7,34 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Fixed
- **Actor v3 YAML Schema Validation in CLI** (#5869): The `agents actor add --config`
command now validates v3 YAML files using `ActorConfigSchema`, ensuring proper
schema compliance including cycle detection for GRAPH actors, required field
validation, and enum validation. v3 YAML is detected by the presence of ANY
`type` field (any value — invalid type values are then rejected by schema
validation) or a `version` field whose string value starts with `"3"` (e.g.
`"3"`, `"3.0"`, `"3.0.0"`). Configs with `type: null` are not treated as v3.
Invalid v3 actors are rejected with clear error messages before registration.
- **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
`features/environment.py` now emits its non-assertion exception guard warning to
both the structured logger and `stderr` via a new `_warning_with_stderr` helper.
This makes the guard firing visible in standard Behave console output and CI log
snippets where the structured logging sink may not be displayed. BDD infrastructure
coverage added: a new scenario in `tdd_expected_fail_infrastructure.feature`
asserts that the warning is emitted to stderr when a non-AssertionError exception
is encountered in an `@tdd_expected_fail` scenario, and a second scenario asserts
the warning is NOT emitted when the exception is an `AssertionError`. The
`CONTRIBUTING.md` now documents that `@tdd_expected_fail` step definitions must
signal expected failures via `AssertionError`.
- **Parallel Behave Runner Log Noise Reduction** (#8351): The parallel behave
runner now suppresses captured stdout/stderr for passing worker chunks and
only replays diagnostics for failed, errored, or crashed chunks. This makes
failure output significantly easier to spot in CI and local runs. A worker
crash (unhandled exception) is detected via an all-zero summary and the
captured traceback is always surfaced.
- **Automation Profile Silent Fallback** (#8232): `_resolve_profile_for_plan` in
`PlanLifecycleService` now raises a clear `ValidationError` when a plan's
automation profile name is not a known built-in profile, instead of silently
+1
View File
@@ -19,5 +19,6 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
* HAL 9000 has contributed the plan concurrency race-condition fix (#7989): wired `LockService` into the plan lifecycle, guarding `execute_plan()` and `apply_plan()` with plan-level advisory locks and unique per-invocation owner identities to prevent silent concurrent state corruption.
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
* Rui Hu has contributed the v3 actor YAML schema validation fix (#5869): added `ActorConfigSchema` validation to the `agents actor add --config` CLI command, covering cycle detection, required field validation, and enum validation for v3 YAML actor definitions.
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
@@ -0,0 +1,273 @@
Feature: Actor add CLI validates v3 YAML via ActorConfigSchema
As a CleverAgents v3 user
I want `agents actor add --config` to validate v3 YAML files using ActorConfigSchema
So that invalid v3 actor definitions (cycles, missing fields, etc.) are rejected at registration time
Background:
Given an actor CLI runner
And a temporary directory for test actor configs
# ────────────────────────────────────────────────────────────
# Valid v3 Actor Registration
# ────────────────────────────────────────────────────────────
Scenario: Register a valid v3 LLM actor via CLI
Given a v3 LLM actor YAML file with name "local/test-llm"
When I run the actor add command for "local/test-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "llm"
Scenario: Register a valid v3 TOOL actor via CLI
Given a v3 TOOL actor YAML file with name "local/test-tool"
When I run the actor add command for "local/test-tool" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "tool"
Scenario: Register a valid v3 GRAPH actor via CLI
Given a v3 GRAPH actor YAML file with name "local/test-graph"
When I run the actor add command for "local/test-graph" with the prepared config file
Then v3actor the command should succeed
And the actor should be registered with type "graph"
# ────────────────────────────────────────────────────────────
# v3 Schema Validation Failures
# ────────────────────────────────────────────────────────────
# NOTE: Only "cycle" is checked here (not specific node names like "node_a"
# or "node_b") because detect_cycles() only appends the single back-edge
# node, not the full cycle path. Node names appear in the error incidentally
# via Pydantic's input dump; checking them would couple the test to Pydantic's
# internal error formatting which may change across versions.
Scenario: Reject v3 GRAPH actor with cycle in route
Given a v3 GRAPH actor YAML file with a cycle in the route
When I run the actor add command for "local/broken-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "cycle"
Scenario: Reject v3 LLM actor without model field
Given a v3 LLM actor YAML file without the model field
When I run the actor add command for "local/no-model-llm" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "LLM actors require 'model' field"
Scenario: Reject v3 TOOL actor without tools field
Given a v3 TOOL actor YAML file without the tools field
When I run the actor add command for "local/no-tools-tool" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "TOOL actors require at least one tool"
Scenario: Reject v3 GRAPH actor without route field
Given a v3 GRAPH actor YAML file without the route field
When I run the actor add command for "local/no-route-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "GRAPH actors require 'route' field"
Scenario: Reject v3 GRAPH actor without model field
Given a v3 GRAPH actor YAML file without the model field
When I run the actor add command for "local/no-model-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "GRAPH actors require 'model' field"
Scenario: Reject v3 actor with invalid type enum
Given a v3 actor YAML file with invalid type "invalid_type"
When I run the actor add command for "local/invalid-type" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "Input should be 'llm', 'tool' or 'graph'"
Scenario: Reject v3 actor with invalid context_view enum
Given a v3 LLM actor YAML file with invalid context_view "invalid_view"
When I run the actor add command for "local/invalid-view" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "context_view"
And v3actor the error message should contain "Input should be"
# ────────────────────────────────────────────────────────────
# Definition of Done: Node ID format validation
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor with invalid node ID format
Given a v3 GRAPH actor YAML file with an invalid node ID
When I run the actor add command for "local/bad-node-id" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "Node ID must be alphanumeric"
# ────────────────────────────────────────────────────────────
# Definition of Done: Tool name namespace validation
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 TOOL actor with improperly namespaced tool name
Given a v3 TOOL actor YAML file with an unnamespaced tool name
When I run the actor add command for "local/bad-tool-name" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "must be namespaced"
# ────────────────────────────────────────────────────────────
# Definition of Done: Unreachable node detection
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor with unreachable node
Given a v3 GRAPH actor YAML file with an unreachable node
When I run the actor add command for "local/unreachable-node" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "unreachable"
# ────────────────────────────────────────────────────────────
# v3 Detection and Backward Compatibility
# ────────────────────────────────────────────────────────────
Scenario: Detect v3 YAML by type field (llm)
Given a v3 LLM actor YAML file with name "local/detect-llm"
When I run the actor add command for "local/detect-llm" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Detect v3 YAML by type field (tool)
Given a v3 TOOL actor YAML file with name "local/detect-tool"
When I run the actor add command for "local/detect-tool" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Detect v3 YAML by type field (graph)
Given a v3 GRAPH actor YAML file with name "local/detect-graph"
When I run the actor add command for "local/detect-graph" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
# NOTE: The fixture includes both version and type fields so that
# ActorConfigSchema.model_validate() succeeds. The version-only branch of
# is_v3_yaml() is exercised by the "Detect v3 YAML by version" direct unit
# scenarios below (using a config blob with only a version field).
Scenario: Register v3 actor with both version and type fields
Given a v3 actor YAML file with version "3.0"
When I run the actor add command for "local/detect-version" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
# ── Backward Compatibility: v2 Actors ─────────────────────────────────────
Scenario: Register a v2 actor without type field
Given a v2 actor YAML file with only name, provider, and model fields
When I run the actor add command for "local/v2-actor" with the prepared config file
Then v3actor the command should succeed
And the actor should NOT be validated via ActorConfigSchema
# ── Update Command v3 Validation ──────────────────────────────────────────
Scenario: Update a v3 actor with valid YAML succeeds
Given a registered actor "local/existing-actor" already exists in the registry
And a v3 LLM actor YAML file with name "local/existing-actor"
When I run the actor update command for "local/existing-actor" with the prepared config file
Then v3actor the command should succeed
And the actor should be validated via ActorConfigSchema
Scenario: Update a v3 actor with invalid YAML (cycle) fails with error
Given a registered actor "local/existing-graph" already exists in the registry
And a v3 GRAPH actor YAML file with a cycle in the route
When I run the actor update command for "local/existing-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "cycle"
# ────────────────────────────────────────────────────────────
# is_v3_yaml Version Branch Edge Cases (Fix 2)
# ────────────────────────────────────────────────────────────
Scenario: Detect v3 YAML by version "3" (short form)
Given a config blob with only version field "3"
When I call is_v3_yaml on the config blob
Then is_v3_yaml should return True
Scenario: Do not detect v3 for version "30" (false positive guard)
Given a config blob with only version field "30"
When I call is_v3_yaml on the config blob
Then is_v3_yaml should return False
Scenario: Do not detect v3 for null type field
Given a config blob with a null type field
When I call is_v3_yaml on the config blob
Then is_v3_yaml should return False
# ────────────────────────────────────────────────────────────
# Provider-Required Validation (Fix 5)
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 LLM actor without provider field
Given a v3 LLM actor YAML file without the provider field named "local/no-provider-llm"
When I run the actor add command for "local/no-provider-llm" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "provider"
Scenario: Reject v3 GRAPH actor without provider field
Given a v3 GRAPH actor YAML file without the provider field named "local/no-provider-graph"
When I run the actor add command for "local/no-provider-graph" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "provider"
# NOTE: CLI-level coverage for TOOL actors without provider is deferred.
# v3 TOOL actors without 'provider' pass ActorConfigSchema validation (tested
# below via direct schema call) but are rejected by the legacy v2
# ActorConfiguration.from_blob() canonicalization layer that the CLI invokes.
# This is a known limitation tracked in follow-up issue:
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
Scenario: Accept valid v3 TOOL actor without provider field (schema level)
Given a valid v3 TOOL actor YAML without provider for schema validation
When I validate the TOOL actor YAML blob directly via ActorConfigSchema
Then the schema validation should succeed
# ────────────────────────────────────────────────────────────
# Registry-Level Direct Tests (Fix 6)
# ────────────────────────────────────────────────────────────
Scenario: ActorRegistry.add directly rejects v3 GRAPH actor with cycle
Given a v3 GRAPH actor YAML with a cycle for direct registry testing named "local/registry-cycle-graph"
When I call ActorRegistry.add directly with the YAML
Then the registry should raise a ValidationError containing "cycle"
# ────────────────────────────────────────────────────────────
# M4: CLI-level test for type: null YAML (falls back to v2 handling)
# ────────────────────────────────────────────────────────────
Scenario: v3 actor with null type field falls back to v2 handling
Given a v3 actor YAML file with null type field
When I run the actor add command
Then v3actor the command should succeed
And the actor should NOT be validated via ActorConfigSchema
# ────────────────────────────────────────────────────────────
# M5: Update without --config flag skips schema validation
# ────────────────────────────────────────────────────────────
Scenario: Update a registered v3 actor without --config flag skips schema validation
Given a registered v3 LLM actor
When I run the actor update command without a config file
Then v3actor the command should succeed
And the actor should NOT be validated via ActorConfigSchema
# ────────────────────────────────────────────────────────────
# M6: CLI-level test for version "3" without type field
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 actor YAML with version "3" but no type field
Given a v3 actor YAML file with version "3" but no type field
When I run the actor add command
Then v3actor the command should fail
And v3actor the error message should contain "type"
# ────────────────────────────────────────────────────────────
# m5: Duplicate node IDs in GRAPH route
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor YAML with duplicate node IDs
Given a v3 GRAPH actor YAML file with duplicate node IDs
When I run the actor add command for "local/dup-node-ids" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "Duplicate node IDs"
# ────────────────────────────────────────────────────────────
# m6: Edge referencing non-existent node
# ────────────────────────────────────────────────────────────
Scenario: Reject v3 GRAPH actor YAML with edge referencing non-existent node
Given a v3 GRAPH actor YAML file with an edge referencing a non-existent node
When I run the actor add command for "local/bad-edge-ref" with the prepared config file
Then v3actor the command should fail
And v3actor the error message should contain "not found"
+10
View File
@@ -71,6 +71,7 @@ Feature: Actor YAML examples validation
type: llm
description: Strategic planning and task decomposition
version: "1.0"
provider: openai
model: gpt-4
system_prompt: |
You are a strategic planner. Break down goals into actionable steps.
@@ -100,6 +101,7 @@ Feature: Actor YAML examples validation
type: llm
description: Code implementation and execution
version: "1.0"
provider: openai
model: gpt-4
system_prompt: "You are an expert software engineer."
context_view: executor
@@ -128,6 +130,7 @@ Feature: Actor YAML examples validation
type: llm
description: Code quality and standards review
version: "1.0"
provider: openai
model: gpt-4
system_prompt: "You are a senior code reviewer."
context_view: reviewer
@@ -197,6 +200,7 @@ Feature: Actor YAML examples validation
type: llm
description: Python code validation and linting
version: "1.0"
provider: openai
model: gpt-4
system_prompt: "You are a validation expert."
context_view: reviewer
@@ -225,6 +229,7 @@ Feature: Actor YAML examples validation
type: graph
description: Linear three-step review workflow
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -282,6 +287,7 @@ Feature: Actor YAML examples validation
type: graph
description: Workflow with conditional routing
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -335,6 +341,7 @@ Feature: Actor YAML examples validation
type: graph
description: Pipeline with tool execution
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -389,6 +396,7 @@ Feature: Actor YAML examples validation
type: graph
description: Hierarchical workflow with subgraphs
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -438,6 +446,7 @@ Feature: Actor YAML examples validation
type: graph
description: Workflow with retry logic
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -506,6 +515,7 @@ Feature: Actor YAML examples validation
type: graph
description: Multi-level planning and execution
version: "1.0"
provider: openai
model: gpt-4
route:
nodes:
@@ -2,6 +2,7 @@ name: m2test/hierarchical-workflow
type: graph
description: M2 smoke test hierarchical actor with planner and executor nodes
version: "1.0"
provider: openai
model: gpt-4
context_view: full
memory:
File diff suppressed because it is too large Load Diff
@@ -55,6 +55,7 @@ def _build_graph_config(
name=name,
type=ActorType.GRAPH,
description="Coverage test graph actor",
provider="openai",
model="gpt-4",
route=route,
)
@@ -360,6 +361,7 @@ def step_given_resolver_llm_type(context: Context, ref: str) -> None:
name=ref,
type=ActorType.LLM,
description="An LLM actor",
provider="openai",
model="gpt-4",
)
context.actor_resolver = lambda name, _r=ref, _a=llm_actor: (
+2
View File
@@ -49,6 +49,7 @@ def _build_graph_config(
name=name,
type=ActorType.GRAPH,
description="Test graph actor",
provider="openai",
model="gpt-4",
route=route,
)
@@ -299,6 +300,7 @@ def step_given_llm_config(context: Context) -> None:
name="assistants/simple",
type=ActorType.LLM,
description="Simple LLM",
provider="openai",
model="gpt-4",
)
+3
View File
@@ -30,6 +30,7 @@ def _base_graph_yaml(
"name": "local/hierarchy-test",
"type": "graph",
"description": "Test hierarchical actor",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": nodes,
@@ -519,6 +520,7 @@ def step_hierarchy_loader_setup(context: Context) -> None:
"name": "local/hierarchy-demo",
"type": "graph",
"description": "Hierarchical demo",
"provider": "openai",
"model": "gpt-4",
"skills": ["local/file-ops"],
"lsp": ["local/pyright"],
@@ -561,6 +563,7 @@ def step_hierarchy_loader_invalid(context: Context) -> None:
"name": "local/bad-lsp",
"type": "graph",
"description": "Bad LSP binding",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": [
+1
View File
@@ -29,6 +29,7 @@ def _minimal_actor_yaml(
'version: "1.0"',
]
if actor_type in ("llm", "graph"):
lines.append("provider: openai")
lines.append(f"model: {model}")
if tools:
lines.append("tools:")
@@ -9,7 +9,7 @@ from behave.runner import Context
from cleveragents.actor.registry import ActorRegistry
from cleveragents.config.settings import ProviderDefaults
from cleveragents.core.exceptions import ValidationError
from cleveragents.core.exceptions import NotFoundError, ValidationError
from cleveragents.domain.models.core.actor import Actor
from cleveragents.providers.registry import ProviderInfo
@@ -74,7 +74,7 @@ class _StubActorService:
def get_actor(self, name: str) -> Actor:
if name not in self.actors:
raise ValueError(f"Actor '{name}' not found")
raise NotFoundError(f"Actor '{name}' not found")
return self.actors[name]
def list_actors(self) -> list[Actor]:
@@ -82,7 +82,7 @@ class _StubActorService:
def remove_actor(self, name: str) -> None:
if name not in self.actors:
raise ValueError(f"Actor '{name}' not found")
raise NotFoundError(f"Actor '{name}' not found")
del self.actors[name]
if self.default_actor_name == name:
self.default_actor_name = None
+20
View File
@@ -24,6 +24,7 @@ _MINIMAL_LLM_YAML = """\
name: assistants/simple
type: llm
description: A simple LLM actor
provider: openai
model: gpt-4
"""
@@ -31,6 +32,7 @@ _LLM_WITH_PROMPT_YAML = """\
name: assistants/expert
type: llm
description: An expert assistant
provider: openai
model: gpt-4
system_prompt: "You are an expert Python developer"
"""
@@ -39,6 +41,7 @@ _LLM_WITH_TOOLS_YAML = """\
name: assistants/helper
type: llm
description: Helper with tools
provider: openai
model: gpt-4
tools:
- files/read_file
@@ -49,6 +52,7 @@ _LLM_WITH_MEMORY_YAML = """\
name: assistants/chatbot
type: llm
description: Chatbot with memory
provider: openai
model: gpt-4
memory:
enabled: true
@@ -60,6 +64,7 @@ _LLM_WITH_CONTEXT_YAML = """\
name: assistants/analyzer
type: llm
description: Code analyzer
provider: openai
model: gpt-4
context:
include_files:
@@ -101,6 +106,7 @@ _GRAPH_MINIMAL_YAML = """\
name: workflows/simple
type: graph
description: Simple workflow
provider: openai
model: gpt-4
route:
nodes:
@@ -120,6 +126,7 @@ _GRAPH_LINEAR_YAML = """\
name: workflows/linear
type: graph
description: Linear workflow
provider: openai
model: gpt-4
route:
nodes:
@@ -155,6 +162,7 @@ _GRAPH_CONDITIONAL_YAML = """\
name: workflows/conditional
type: graph
description: Workflow with conditionals
provider: openai
model: gpt-4
route:
nodes:
@@ -199,6 +207,7 @@ _GRAPH_SUBGRAPH_YAML = """\
name: workflows/composed
type: graph
description: Workflow with subgraph
provider: openai
model: gpt-4
route:
nodes:
@@ -305,6 +314,7 @@ def step_given_actor_model_for_role_warning(context: Context) -> None:
name="local/model-warning-actor",
type="llm",
description="Model-input warnings path",
provider="openai",
model="gpt-4",
role_hint="estimation",
context_view="executor",
@@ -319,6 +329,7 @@ def step_given_actor_model_without_response_format(context: Context) -> None:
name="local/model-warning-no-schema",
type="llm",
description="Model-input missing response_format",
provider="openai",
model="gpt-4",
role_hint="estimation",
context_view="strategist",
@@ -732,6 +743,7 @@ def step_given_specific_node_id(context: Context, node_id: str) -> None:
name: workflows/test
type: graph
description: Test workflow
provider: openai
model: gpt-4
route:
nodes:
@@ -755,6 +767,7 @@ def step_given_context_view(context: Context, view: str) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
context_view: {view}
"""
@@ -767,6 +780,7 @@ def step_given_env_vars(context: Context) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
env_vars:
LOG_LEVEL: info
@@ -787,6 +801,7 @@ def step_given_valid_actor_object(context: Context) -> None:
name="test/actor",
type="llm",
description="Test actor",
provider="openai",
model="gpt-4",
)
@@ -804,6 +819,7 @@ def step_given_edge_priorities(context: Context) -> None:
name: workflows/priorities
type: graph
description: Workflow with priorities
provider: openai
model: gpt-4
route:
nodes:
@@ -836,6 +852,7 @@ def step_given_memory_disabled(context: Context) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
memory:
enabled: false
@@ -849,6 +866,7 @@ def step_given_max_messages(context: Context, count: int) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
memory:
max_messages: {count}
@@ -862,6 +880,7 @@ def step_given_max_tokens(context: Context, count: int) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
memory:
max_tokens: {count}
@@ -875,6 +894,7 @@ def step_given_summarize_old(context: Context) -> None:
name: assistants/test
type: llm
description: Test actor
provider: openai
model: gpt-4
memory:
summarize_old: true
@@ -170,6 +170,7 @@ def step_pfg_estimation_actor_model_missing_response_format(context: Context) ->
name="local/estimator-model",
type="llm",
description="Model-based estimation actor",
provider="openai",
model="gpt-4",
role_hint="estimation",
context_view="strategist",
+335
View File
@@ -0,0 +1,335 @@
*** Settings ***
Documentation Integration tests for actor add CLI v3 schema validation
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment With Database Isolation
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER} ${CURDIR}/helper_actor_add_v3_schema_validation.py
*** Test Cases ***
Register Valid v3 LLM Actor Via CLI
[Documentation] Register a valid v3 LLM actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_llm.yaml
Create File ${yaml_file} name: local/test-llm\ntype: llm\ndescription: Test LLM\nprovider: openai\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/test-llm ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Register Valid v3 TOOL Actor Via CLI
[Documentation] Register a valid v3 TOOL actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_tool.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/test-tool
... type: tool
... description: Test TOOL
... provider: openai
... model: gpt-4
... tools:
... ${SPACE}${SPACE}- files/read_file
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/test-tool ${yaml_file} cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Register Valid v3 GRAPH Actor Via CLI
[Documentation] Register a valid v3 GRAPH actor via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_graph.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/test-graph
... type: graph
... description: Test GRAPH
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/test-graph ${yaml_file} cwd=${WORKSPACE}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} actor-add-success
Reject v3 GRAPH Actor With Cycle In Route
[Documentation] Reject v3 GRAPH actor with cycle in route via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}broken_graph_cycle.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/broken-graph
... type: graph
... description: Graph with cycle
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node B
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Second node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges:
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_a
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/broken-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} cycle
Reject v3 LLM Actor Without Model Field
[Documentation] Reject v3 LLM actor without model field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_llm_no_model.yaml
Create File ${yaml_file} name: local/no-model-llm\ntype: llm\ndescription: LLM without model\nprovider: openai\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-llm ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} LLM actors require 'model' field
Reject v3 TOOL Actor Without Tools Field
[Documentation] Reject v3 TOOL actor without tools field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_tool_no_tools.yaml
Create File ${yaml_file} name: local/no-tools-tool\ntype: tool\ndescription: TOOL without tools\nprovider: openai\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-tools-tool ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} TOOL actors require at least one tool
Reject v3 GRAPH Actor Without Route Field
[Documentation] Reject v3 GRAPH actor without route field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_route.yaml
Create File ${yaml_file} name: local/no-route-graph\ntype: graph\ndescription: GRAPH without route\nprovider: openai\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/no-route-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} GRAPH actors require 'route' field
Reject v3 GRAPH Actor Without Model Field
[Documentation] Reject v3 GRAPH actor without model field via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_model.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/no-model-graph
... type: graph
... description: GRAPH without model
... provider: openai
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} GRAPH actors require 'model' field
Reject v3 Actor With Invalid Type Enum
[Documentation] Reject v3 actor with invalid type enum via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_type.yaml
Create File ${yaml_file} name: local/invalid-type\ntype: invalid_type\ndescription: Invalid type\nprovider: openai\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-type ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} Input should be 'llm', 'tool' or 'graph'
Reject v3 Actor With Invalid Context View Enum
[Documentation] Reject v3 actor with invalid context_view enum via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_context_view.yaml
Create File ${yaml_file} name: local/invalid-view\ntype: llm\ndescription: Invalid context_view\nprovider: openai\nmodel: gpt-4\ncontext_view: invalid_view\n
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-view ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} context_view
Should Contain ${result.stderr} Input should be
Reject v3 GRAPH Actor With Cycle Via Update Command
[Documentation] Reject v3 GRAPH actor with cycle via agents actor update
[Tags] slow
# First register a valid v3 GRAPH actor so that the update command can
# resolve it. The update command calls registry.get_actor(name) before
# validating the new config — without this step the command aborts with
# "Actor not found" instead of a cycle validation error.
${valid_yaml_file}= Set Variable ${TEMPDIR}${/}update_valid_graph.yaml
${valid_yaml_content}= Catenate SEPARATOR=\n
... name: local/update-test-graph
... type: graph
... description: Valid graph actor for update test setup
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
Create File ${valid_yaml_file} ${valid_yaml_content}
${setup_result}= Run Process ${PYTHON} ${HELPER} add local/update-test-graph ${valid_yaml_file} cwd=${WORKSPACE}
Log ${setup_result.stdout}
Log ${setup_result.stderr}
Should Be Equal As Integers ${setup_result.rc} 0
# Now attempt the update with a cyclic YAML — should fail with cycle error
${yaml_file}= Set Variable ${TEMPDIR}${/}update_broken_cycle.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/update-test-graph
... type: graph
... description: Graph with cycle for update test
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node B
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Second node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges:
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_a
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} update local/update-test-graph ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} cycle
Reject v3 GRAPH Actor With Invalid Node ID
[Documentation] Reject v3 GRAPH actor with non-alphanumeric node ID via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_node_id.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/bad-node-id
... type: graph
... description: Graph with invalid node ID
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: "bad!node"
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Bad Node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Node with invalid ID
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges: []
... ${SPACE}${SPACE}entry_node: "bad!node"
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- "bad!node"
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/bad-node-id ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} Node ID must be alphanumeric
Reject v3 TOOL Actor With Unnamespaced Tool Name
[Documentation] Reject v3 TOOL actor with unnamespaced tool name via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}unnamespaced_tool.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/bad-tool-name
... type: tool
... description: TOOL with unnamespaced tool
... provider: openai
... model: gpt-4
... tools:
... ${SPACE}${SPACE}- read_file
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/bad-tool-name ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} must be namespaced
Reject v3 GRAPH Actor With Unreachable Node
[Documentation] Reject v3 GRAPH actor with unreachable node via agents actor add
[Tags] slow
${yaml_file}= Set Variable ${TEMPDIR}${/}unreachable_node.yaml
${yaml_content}= Catenate SEPARATOR=\n
... name: local/unreachable-node
... type: graph
... description: Graph with unreachable node
... provider: openai
... model: gpt-4
... route:
... ${SPACE}${SPACE}nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Entry Node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Entry node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Exit Node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Exit node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_orphan
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Orphan Node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Unreachable node
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
... ${SPACE}${SPACE}edges:
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
... ${SPACE}${SPACE}entry_node: node_a
... ${SPACE}${SPACE}exit_nodes:
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
Create File ${yaml_file} ${yaml_content}
${result}= Run Process ${PYTHON} ${HELPER} add local/unreachable-node ${yaml_file} cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 1
Should Contain ${result.stderr} unreachable
+1 -1
View File
@@ -21,7 +21,7 @@ Discover Actors From Custom Directory
${tmp}= Set Variable ${TEMPDIR}${/}actor_loading_robot
Run Keyword And Ignore Error Remove Directory ${tmp} recursive=True
Create Directory ${tmp}
Create File ${tmp}${/}test.yaml name: local/robot-test\ntype: llm\ndescription: Robot test actor\nversion: "1.0"\nmodel: gpt-4\n
Create File ${tmp}${/}test.yaml name: local/robot-test\ntype: llm\ndescription: Robot test actor\nversion: "1.0"\nprovider: openai\nmodel: gpt-4\n
${result}= Run Process ${PYTHON} ${HELPER} discover ${tmp} cwd=${WORKSPACE}
Log ${result.stdout}
Should Be Equal As Integers ${result.rc} 0
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""Helper script for actor add v3 schema validation robot tests.
Tests the agents actor add CLI command with v3 YAML files to ensure
ActorConfigSchema validation is applied (cycle detection, required fields, etc.).
This helper delegates to the real CLI binary so that it exercises the actual
command path not internal helpers.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
def add_actor(actor_name: str, config_file: str) -> int:
"""Test adding an actor via the real CLI binary.
Invokes ``agents actor add <actor_name> --config <config_file>`` as a
subprocess so that all CLI-level validation (including v3 schema
validation) runs for real.
Args:
actor_name: The actor name (e.g., local/test-llm)
config_file: Path to the YAML config file
Returns:
0 on success, 1 on validation failure or timeout
"""
config_path = Path(config_file)
if not config_path.exists():
print(
f"actor-add-error: Config file not found: {config_file}",
file=sys.stderr,
)
return 1
try:
result = subprocess.run(
["agents", "actor", "add", actor_name, "--config", config_file],
capture_output=True,
text=True,
timeout=60,
)
except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
print(f"actor-add-error: command failed: {exc}", file=sys.stderr)
return 1
if result.returncode == 0:
print("actor-add-success")
return 0
else:
output = result.stdout + result.stderr
print(f"actor-add-error: {output.strip()}", file=sys.stderr)
return 1
def update_actor(actor_name: str, config_file: str) -> int:
"""Test updating an actor via the real CLI binary.
Invokes ``agents actor update <actor_name> --config <config_file>`` as a
subprocess so that all CLI-level validation (including v3 schema
validation) runs for real.
Args:
actor_name: The actor name (e.g., local/test-llm)
config_file: Path to the YAML config file
Returns:
0 on success, 1 on validation failure or timeout
"""
config_path = Path(config_file)
if not config_path.exists():
print(
f"actor-update-error: Config file not found: {config_file}",
file=sys.stderr,
)
return 1
try:
result = subprocess.run(
["agents", "actor", "update", actor_name, "--config", config_file],
capture_output=True,
text=True,
timeout=60,
)
except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
print(f"actor-update-error: command failed: {exc}", file=sys.stderr)
return 1
if result.returncode == 0:
print("actor-update-success")
return 0
else:
output = result.stdout + result.stderr
print(f"actor-update-error: {output.strip()}", file=sys.stderr)
return 1
def main() -> int:
"""Main entry point."""
if len(sys.argv) < 4:
print(
"Usage: helper_actor_add_v3_schema_validation.py"
" <command> <actor_name> <config_file>"
)
return 1
command = sys.argv[1]
actor_name = sys.argv[2]
config_file = sys.argv[3]
if command == "add":
return add_actor(actor_name, config_file)
elif command == "update":
return update_actor(actor_name, config_file)
else:
print(f"Unknown command: {command}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+4
View File
@@ -61,6 +61,7 @@ def _parse_lsp_binding() -> int:
"name": "local/lsp-test",
"type": "graph",
"description": "LSP binding test",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": [
@@ -94,6 +95,7 @@ def _parse_tool_sources() -> int:
"name": "local/ts-test",
"type": "graph",
"description": "Tool sources test",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": [
@@ -125,6 +127,7 @@ def _parse_subgraph_ref() -> int:
"name": "local/sg-test",
"type": "graph",
"description": "Subgraph ref test",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": [
@@ -159,6 +162,7 @@ def _reject_bad_lsp() -> int:
"name": "local/bad-lsp-test",
"type": "graph",
"description": "Bad LSP test",
"provider": "openai",
"model": "gpt-4",
"route": {
"nodes": [
+1
View File
@@ -100,6 +100,7 @@ def _namespace_default() -> int:
"type: llm\n"
"description: Test namespace defaulting\n"
'version: "1.0"\n'
"provider: openai\n"
"model: gpt-4\n"
)
(tmp / "bare.yaml").write_text(yaml_content)
+4
View File
@@ -119,6 +119,7 @@ def actor_yaml_create_load() -> None:
"type: graph\n"
"description: M2 E2E test actor\n"
'version: "1.0"\n'
"provider: openai\n"
"model: gpt-4\n"
"route:\n"
" nodes:\n"
@@ -299,6 +300,7 @@ def actor_yaml_parse_validate() -> None:
"type: graph\n"
"description: Valid actor\n"
'version: "1.0"\n'
"provider: openai\n"
"model: gpt-4\n"
"route:\n"
" nodes:\n"
@@ -327,6 +329,7 @@ def actor_yaml_parse_validate() -> None:
"type: llm\n"
"description: LLM actor\n"
'version: "1.0"\n'
"provider: openai\n"
"model: gpt-4\n"
)
(tmp / "llm.yaml").write_text(llm_yaml)
@@ -373,6 +376,7 @@ def actor_compile_stategraph() -> None:
"type: llm\n"
"description: LLM only actor\n"
'version: "1.0"\n'
"provider: openai\n"
"model: gpt-4\n"
)
(tmp / "llm.yaml").write_text(llm_yaml)
+52 -9
View File
@@ -5,10 +5,13 @@ from __future__ import annotations
from dataclasses import asdict
from typing import Any
import pydantic
from cleveragents.actor.config import ActorConfiguration
from cleveragents.actor.schema import ActorConfigSchema, is_v3_yaml
from cleveragents.application.services.actor_service import ActorService
from cleveragents.config.settings import Settings
from cleveragents.core.exceptions import ValidationError
from cleveragents.core.exceptions import NotFoundError, ValidationError
from cleveragents.domain.models.core import Actor
from cleveragents.providers.registry import (
ProviderCapabilities,
@@ -189,6 +192,11 @@ class ActorRegistry:
The YAML is parsed into an ``ActorConfiguration``, validated, and
persisted alongside the original *yaml_text* and *schema_version*.
For v3 YAML files (detected by any non-null ``type`` field or a
``version`` starting with ``'3'``), the configuration is validated
using ``ActorConfigSchema`` to ensure proper schema compliance,
including cycle detection for GRAPH actors.
Args:
yaml_text: The original actor YAML source.
update: When ``True`` allow overwriting an existing actor.
@@ -210,29 +218,42 @@ class ActorRegistry:
if not name_raw:
raise ValidationError("Actor YAML must include a 'name' field.")
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
# This ensures v3 actors are validated against the full schema, including
# cycle detection, required fields, and enum validation.
blob_is_v3 = is_v3_yaml(blob)
if blob_is_v3:
try:
ActorConfigSchema.model_validate(blob)
except pydantic.ValidationError as exc:
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
name = self._ensure_namespaced(name_raw)
provider_raw = blob.get("provider") or blob.get("provider_type", "")
model_raw = blob.get("model") or blob.get("model_id", "")
if not provider_raw or not model_raw:
if not blob_is_v3 and (not provider_raw or not model_raw):
raise ValidationError(
"Actor YAML must include 'provider' and 'model' fields."
)
provider = str(provider_raw)
model = str(model_raw)
# For v3 actors without provider/model (e.g. TOOL actors), provider_raw
# and model_raw may be empty strings here. This is expected — the legacy
# canonicalization path in upsert_actor() / from_blob() will reject
# provider-less TOOL actors. See follow-up issue #9971.
provider = str(provider_raw) if provider_raw else ""
model = str(model_raw) if model_raw else ""
# Check for existing actor when not updating
if not update:
try:
self._actor_service.get_actor(name)
except NotFoundError:
pass # Actor does not exist yet — proceed with add
else:
raise ValidationError(
f"Actor '{name}' already exists. Pass update=True to overwrite."
)
except Exception as exc:
if "already exists" in str(exc):
raise
# NotFoundError is expected for new actors
version = schema_version or self.DEFAULT_SCHEMA_VERSION
config_blob: dict[str, Any] = dict(blob)
@@ -273,9 +294,31 @@ class ActorRegistry:
schema_version: str | None = None,
compiled_metadata: dict[str, Any] | None = None,
) -> Actor:
"""Parse, validate, and persist an actor configuration."""
"""Parse, validate, and persist an actor configuration.
For v3 YAML files (detected by any non-null ``type`` field or a
``version`` starting with ``'3'``), the configuration is validated
using ``ActorConfigSchema`` to ensure proper schema compliance,
including cycle detection for GRAPH actors.
"""
self.ensure_built_in_actors()
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
# This ensures v3 actors are validated against the full schema, including
# cycle detection, required fields, and enum validation.
if config_blob and is_v3_yaml(config_blob):
try:
ActorConfigSchema.model_validate(config_blob)
except pydantic.ValidationError as exc:
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
# NOTE: v3 TOOL actors without 'provider' will be rejected here by the legacy
# v2 canonicalization layer (ActorConfiguration.from_blob requires provider).
# This is a known limitation tracked in follow-up issue:
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
# v3 schema validation (above) already accepts provider-less TOOL actors
# correctly; the gap is only in the legacy from_blob() canonicalization.
config = ActorConfiguration.from_blob(
blob=config_blob,
name=name,
+114 -19
View File
@@ -544,6 +544,30 @@ class RouteDefinition(BaseModel):
raise ValueError(msg)
return v
def _build_adjacency(self) -> dict[str, list[str]]:
"""Build the full adjacency list for the route graph.
Includes both explicit edges AND implicit ``route_to`` targets
embedded in CONDITIONAL node configs so that both
:meth:`detect_cycles` and :meth:`validate_references` use an
identical, complete view of the graph topology.
Returns:
Mapping from node ID to list of reachable neighbour node IDs.
"""
node_ids = {node.id for node in self.nodes}
adj: dict[str, list[str]] = {node.id: [] for node in self.nodes}
for edge in self.edges:
adj[edge.from_node].append(edge.to_node)
for node in self.nodes:
if node.type == NodeType.CONDITIONAL and isinstance(node.config, dict):
for cond in node.config.get("conditions", []):
if isinstance(cond, dict) and "route_to" in cond:
target = str(cond["route_to"])
if target in node_ids:
adj[node.id].append(target)
return adj
def validate_references(self) -> None:
"""
Validate all node references in edges and entry/exit points.
@@ -585,18 +609,9 @@ class RouteDefinition(BaseModel):
raise ValueError(msg)
# Check all nodes are reachable from entry_node via BFS.
# Adjacency is built from both explicit edges and implicit routing
# targets embedded in conditional node configs (route_to keys).
adj: dict[str, list[str]] = {node.id: [] for node in self.nodes}
for edge in self.edges:
adj[edge.from_node].append(edge.to_node)
for node in self.nodes:
if node.type == NodeType.CONDITIONAL and isinstance(node.config, dict):
for cond in node.config.get("conditions", []):
if isinstance(cond, dict) and "route_to" in cond:
target = str(cond["route_to"])
if target in node_ids:
adj[node.id].append(target)
# Use _build_adjacency() to include both explicit edges and implicit
# route_to targets from CONDITIONAL nodes.
adj = self._build_adjacency()
reachable: set[str] = set()
queue: list[str] = [self.entry_node]
@@ -622,10 +637,17 @@ class RouteDefinition(BaseModel):
"""
Detect cycles in the graph using DFS.
Only explicit edges are considered for cycle detection conditional
``route_to`` routing targets are intentional back-references (e.g. retry
loops) and are not treated as cycles.
Returns:
List of node IDs involved in cycles (empty if acyclic)
"""
# Build adjacency list
# Build adjacency list from explicit edges only (not route_to targets).
# _build_adjacency() is used by validate_references() for reachability
# but is intentionally NOT used here to avoid false positive cycle
# detection from CONDITIONAL node retry patterns.
graph: dict[str, list[str]] = {node.id: [] for node in self.nodes}
for edge in self.edges:
graph[edge.from_node].append(edge.to_node)
@@ -675,6 +697,9 @@ class ActorConfigSchema(BaseModel):
type: Actor type (LLM, TOOL, or GRAPH)
description: What the actor does
version: Schema version (default: "1.0")
provider: LLM provider name (e.g. "openai", "anthropic"); required for
LLM and GRAPH actors, optional for TOOL actors which have no direct
LLM invocation.
model: LLM model name (required for LLM/GRAPH types)
system_prompt: System prompt for LLM actors
tools: List of tool references or inline definitions
@@ -685,9 +710,11 @@ class ActorConfigSchema(BaseModel):
env_vars: Environment variable mappings
Type-Specific Requirements:
ActorType.LLM: Requires model, optional system_prompt and tools
ActorType.TOOL: Requires tools list
ActorType.GRAPH: Requires model and route
ActorType.LLM: Requires ``provider`` and ``model``; optional
``system_prompt`` and ``tools``
ActorType.TOOL: Requires at least one entry in ``tools``; ``provider``
is optional (TOOL actors do not make direct LLM calls)
ActorType.GRAPH: Requires ``provider``, ``model``, and ``route``
Examples:
>>> # Simple LLM actor
@@ -714,6 +741,13 @@ class ActorConfigSchema(BaseModel):
description: str = Field(..., description="Actor description")
version: str = Field(default="1.0", description="Schema version")
# Provider is required for LLM and GRAPH actors (validated by
# validate_provider_required_for_llm_graph below), but optional for TOOL
# actors which have no direct LLM invocation and do not need a provider.
provider: str | None = Field(
default=None, description="LLM provider (e.g. openai, anthropic)"
)
# LLM configuration
model: str | None = Field(default=None, description="LLM model name")
system_prompt: str | None = Field(default=None, description="System prompt")
@@ -831,6 +865,21 @@ class ActorConfigSchema(BaseModel):
msg = "TOOL actors require at least one tool"
raise ValueError(msg)
# Validate string tool references: must follow "namespace/name" format.
# ToolDefinition instances are validated separately by their own field
# validators. Use strict two-part split to catch empty namespace or
# empty name (e.g. "/tool", "namespace/", or "a/b/c").
for tool_ref in self.tools:
if isinstance(tool_ref, str):
parts = tool_ref.split("/")
if len(parts) != 2 or not parts[0] or not parts[1]:
msg = (
f"tool '{tool_ref}' must be namespaced "
f"(namespace/name format, e.g. 'files/read_file'). "
f"Namespace and name must both be non-empty."
)
raise ValueError(msg)
# GRAPH actors require model and route
if self.type == ActorType.GRAPH:
if not self.model:
@@ -844,16 +893,25 @@ class ActorConfigSchema(BaseModel):
self.route.validate_references()
cycles = self.route.detect_cycles()
if cycles:
nodes_str = "".join(cycles)
msg = (
f"route: graph contains a cycle involving "
f"nodes: [{nodes_str}]. "
f"route: graph contains a cycle "
f"back-edge detected at node '{cycles[0]}'. "
f"Hint: remove or redirect edges to break the cycle."
)
raise ValueError(msg)
return self
# Runs after validate_type_requirements — provider check is secondary to
# type-specific field requirements.
@model_validator(mode="after")
def validate_provider_required_for_llm_graph(self) -> ActorConfigSchema:
"""Require provider for LLM and GRAPH actor types."""
if self.type in (ActorType.LLM, ActorType.GRAPH) and not self.provider:
msg = "LLM and GRAPH actors require 'provider' field"
raise ValueError(msg)
return self
@classmethod
def from_yaml_file(cls, file_path: str | Path) -> ActorConfigSchema:
"""
@@ -906,6 +964,42 @@ class ActorConfigSchema(BaseModel):
)
def is_v3_yaml(config_blob: dict[str, Any] | None) -> bool:
"""Detect if a config blob is v3 YAML.
A config blob is considered v3 YAML if it contains ANY ``type`` field
(any value let the schema validate whether the value is legal) or a
``version`` field whose string representation starts with ``"3"`` (e.g.
``"3"``, ``"3.0"``, ``"3.0.0"``).
Treating any ``type`` field as a v3 signal prevents configs with invalid
type values (e.g. ``type: robot``) from bypassing schema validation.
Args:
config_blob: The parsed YAML/JSON configuration dictionary.
Returns:
True if the config appears to be v3 YAML, False otherwise.
"""
if not isinstance(config_blob, dict):
return False
# Any non-null 'type' field = v3 YAML; let the schema validate the value.
# Explicitly exclude type: null so that misconfigured blobs with a null
# type key do not accidentally trigger v3 schema validation.
if "type" in config_blob and config_blob["type"] is not None:
return True
# Check for version field indicating v3 (future-proof: "3", "3.0", "3.0.0").
# Use exact equality for "3" and prefix check for "3." to avoid false
# positives from version strings like "30" or "300".
version = config_blob.get("version")
if version is None:
return False
version_str = str(version)
return version_str == "3" or version_str.startswith("3.")
__all__ = [
"ActorConfigSchema",
"ActorType",
@@ -923,4 +1017,5 @@ __all__ = [
"ToolParameter",
"ToolSourceRef",
"actor_role_warnings",
"is_v3_yaml", # Internal helper; not re-exported from actor/__init__.py
]
+54 -4
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from typing import Annotated, Any, cast
import click
import pydantic
import typer
import yaml
from rich.console import Console
@@ -13,7 +14,7 @@ from rich.panel import Panel
from rich.table import Table
from cleveragents.actor.config import ActorConfiguration
from cleveragents.actor.schema import actor_role_warnings
from cleveragents.actor.schema import ActorConfigSchema, actor_role_warnings, is_v3_yaml
from cleveragents.application.container import get_container
from cleveragents.cli.commands._resolve_actor import (
resolve_config_files as _resolve_config_files,
@@ -345,6 +346,32 @@ def _parse_option_overrides(option_values: list[str] | None) -> dict[str, Any]:
return overrides
def _validate_v3_config(source_path: Path, config_blob: dict[str, Any]) -> None:
"""Validate a v3 config blob using ActorConfigSchema.
Uses the already-parsed ``config_blob`` to avoid re-reading the file
(eliminates TOCTOU: the config the CLI parsed is the config that gets
validated, not a potentially-different file re-read at validation time).
Only ``pydantic.ValidationError`` is caught here because
``model_validate(dict)`` cannot raise ``yaml.YAMLError`` or ``OSError``
the YAML parsing has already happened before this function is called.
Args:
source_path: Path to the config file (used only in error messages).
config_blob: The parsed YAML/JSON configuration dictionary.
Raises:
typer.BadParameter: If schema validation fails.
"""
try:
ActorConfigSchema.model_validate(config_blob)
except pydantic.ValidationError as exc:
raise typer.BadParameter(
f"Invalid actor configuration in {source_path}: {exc}"
) from exc
def _canonicalize_actor_config(
*,
name: str,
@@ -570,6 +597,22 @@ def add(
assert loaded is not None, "unreachable: config is not None"
yaml_text, config_blob = loaded
# Validate v3 config via ActorConfigSchema if detected.
# This ensures v3 actors are fully validated (cycle detection, required
# fields, enum values) before the registry stores them.
# NOTE: Both the CLI (here) and the registry validate v3 configs
# independently. The CLI validation provides fast user-facing error
# messages; the registry validation is a defence-in-depth guard for
# programmatic callers that bypass the CLI layer.
if is_v3_yaml(config_blob):
_validate_v3_config(config, config_blob)
# NOTE: v3 TOOL actors without 'provider' will be rejected here by the legacy
# v2 canonicalization layer (ActorConfiguration.from_blob requires provider).
# This is a known limitation tracked in follow-up issue:
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
# Schema-level validation (above) already accepts provider-less TOOL actors
# correctly; the gap is only in the legacy from_blob() canonicalization.
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
name=name,
config_blob=config_blob,
@@ -607,9 +650,8 @@ def add(
pass # Actor does not exist yet — proceed with add
# Extract schema_version from the parsed config blob if present.
schema_version: str | None = config_blob.get("schema_version") # type: ignore[assignment]
if schema_version and not isinstance(schema_version, str):
schema_version = str(schema_version)
raw_version = config_blob.get("schema_version")
schema_version: str | None = str(raw_version) if raw_version is not None else None
try:
if registry:
@@ -713,6 +755,14 @@ def update(
if not option_overrides:
option_overrides = None
# Validate v3 YAML via ActorConfigSchema if a new config file was given.
# The `config is not None` guard is intentional: when no new --config file
# is provided, the existing registry blob is reused as-is (it was already
# validated at registration time). Only newly supplied config files need
# re-validation to catch any schema violations in the uploaded YAML.
if config is not None and is_v3_yaml(new_config):
_validate_v3_config(config, new_config)
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
name=name,
config_blob=new_config,