fix(actor): validate v3 YAML via ActorConfigSchema in agents actor add CLI #8636
@@ -7,6 +7,34 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Actor v3 YAML Schema Validation in CLI** (#5869): The `agents actor add --config`
|
||||
command now validates v3 YAML files using `ActorConfigSchema`, ensuring proper
|
||||
schema compliance including cycle detection for GRAPH actors, required field
|
||||
validation, and enum validation. v3 YAML is detected by the presence of ANY
|
||||
`type` field (any value — invalid type values are then rejected by schema
|
||||
validation) or a `version` field whose string value starts with `"3"` (e.g.
|
||||
`"3"`, `"3.0"`, `"3.0.0"`). Configs with `type: null` are not treated as v3.
|
||||
Invalid v3 actors are rejected with clear error messages before registration.
|
||||
|
||||
- **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
|
||||
`features/environment.py` now emits its non-assertion exception guard warning to
|
||||
both the structured logger and `stderr` via a new `_warning_with_stderr` helper.
|
||||
This makes the guard firing visible in standard Behave console output and CI log
|
||||
snippets where the structured logging sink may not be displayed. BDD infrastructure
|
||||
coverage added: a new scenario in `tdd_expected_fail_infrastructure.feature`
|
||||
asserts that the warning is emitted to stderr when a non-AssertionError exception
|
||||
is encountered in an `@tdd_expected_fail` scenario, and a second scenario asserts
|
||||
the warning is NOT emitted when the exception is an `AssertionError`. The
|
||||
`CONTRIBUTING.md` now documents that `@tdd_expected_fail` step definitions must
|
||||
signal expected failures via `AssertionError`.
|
||||
|
||||
- **Parallel Behave Runner Log Noise Reduction** (#8351): The parallel behave
|
||||
runner now suppresses captured stdout/stderr for passing worker chunks and
|
||||
only replays diagnostics for failed, errored, or crashed chunks. This makes
|
||||
failure output significantly easier to spot in CI and local runs. A worker
|
||||
crash (unhandled exception) is detected via an all-zero summary and the
|
||||
captured traceback is always surfaced.
|
||||
|
||||
- **Automation Profile Silent Fallback** (#8232): `_resolve_profile_for_plan` in
|
||||
`PlanLifecycleService` now raises a clear `ValidationError` when a plan's
|
||||
automation profile name is not a known built-in profile, instead of silently
|
||||
|
||||
@@ -19,5 +19,6 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed concurrency safety improvements, including thread-safe context tier management (issue #7547) for parallel plan execution.
|
||||
* HAL 9000 has contributed the plan concurrency race-condition fix (#7989): wired `LockService` into the plan lifecycle, guarding `execute_plan()` and `apply_plan()` with plan-level advisory locks and unique per-invocation owner identities to prevent silent concurrent state corruption.
|
||||
* HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading.
|
||||
* Rui Hu has contributed the v3 actor YAML schema validation fix (#5869): added `ActorConfigSchema` validation to the `agents actor add --config` CLI command, covering cycle detection, required field validation, and enum validation for v3 YAML actor definitions.
|
||||
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
|
||||
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
Feature: Actor add CLI validates v3 YAML via ActorConfigSchema
|
||||
As a CleverAgents v3 user
|
||||
I want `agents actor add --config` to validate v3 YAML files using ActorConfigSchema
|
||||
So that invalid v3 actor definitions (cycles, missing fields, etc.) are rejected at registration time
|
||||
|
||||
Background:
|
||||
Given an actor CLI runner
|
||||
And a temporary directory for test actor configs
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Valid v3 Actor Registration
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Register a valid v3 LLM actor via CLI
|
||||
Given a v3 LLM actor YAML file with name "local/test-llm"
|
||||
When I run the actor add command for "local/test-llm" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be registered with type "llm"
|
||||
|
||||
Scenario: Register a valid v3 TOOL actor via CLI
|
||||
Given a v3 TOOL actor YAML file with name "local/test-tool"
|
||||
When I run the actor add command for "local/test-tool" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be registered with type "tool"
|
||||
|
||||
Scenario: Register a valid v3 GRAPH actor via CLI
|
||||
Given a v3 GRAPH actor YAML file with name "local/test-graph"
|
||||
When I run the actor add command for "local/test-graph" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be registered with type "graph"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# v3 Schema Validation Failures
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
# NOTE: Only "cycle" is checked here (not specific node names like "node_a"
|
||||
# or "node_b") because detect_cycles() only appends the single back-edge
|
||||
# node, not the full cycle path. Node names appear in the error incidentally
|
||||
# via Pydantic's input dump; checking them would couple the test to Pydantic's
|
||||
# internal error formatting which may change across versions.
|
||||
Scenario: Reject v3 GRAPH actor with cycle in route
|
||||
Given a v3 GRAPH actor YAML file with a cycle in the route
|
||||
When I run the actor add command for "local/broken-graph" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "cycle"
|
||||
|
||||
Scenario: Reject v3 LLM actor without model field
|
||||
Given a v3 LLM actor YAML file without the model field
|
||||
When I run the actor add command for "local/no-model-llm" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "LLM actors require 'model' field"
|
||||
|
||||
Scenario: Reject v3 TOOL actor without tools field
|
||||
Given a v3 TOOL actor YAML file without the tools field
|
||||
When I run the actor add command for "local/no-tools-tool" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "TOOL actors require at least one tool"
|
||||
|
||||
Scenario: Reject v3 GRAPH actor without route field
|
||||
Given a v3 GRAPH actor YAML file without the route field
|
||||
When I run the actor add command for "local/no-route-graph" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "GRAPH actors require 'route' field"
|
||||
|
||||
Scenario: Reject v3 GRAPH actor without model field
|
||||
Given a v3 GRAPH actor YAML file without the model field
|
||||
When I run the actor add command for "local/no-model-graph" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "GRAPH actors require 'model' field"
|
||||
|
||||
Scenario: Reject v3 actor with invalid type enum
|
||||
Given a v3 actor YAML file with invalid type "invalid_type"
|
||||
When I run the actor add command for "local/invalid-type" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "Input should be 'llm', 'tool' or 'graph'"
|
||||
|
||||
Scenario: Reject v3 actor with invalid context_view enum
|
||||
Given a v3 LLM actor YAML file with invalid context_view "invalid_view"
|
||||
When I run the actor add command for "local/invalid-view" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "context_view"
|
||||
And v3actor the error message should contain "Input should be"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Definition of Done: Node ID format validation
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 GRAPH actor with invalid node ID format
|
||||
Given a v3 GRAPH actor YAML file with an invalid node ID
|
||||
When I run the actor add command for "local/bad-node-id" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "Node ID must be alphanumeric"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Definition of Done: Tool name namespace validation
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 TOOL actor with improperly namespaced tool name
|
||||
Given a v3 TOOL actor YAML file with an unnamespaced tool name
|
||||
When I run the actor add command for "local/bad-tool-name" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "must be namespaced"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Definition of Done: Unreachable node detection
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 GRAPH actor with unreachable node
|
||||
Given a v3 GRAPH actor YAML file with an unreachable node
|
||||
When I run the actor add command for "local/unreachable-node" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "unreachable"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# v3 Detection and Backward Compatibility
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Detect v3 YAML by type field (llm)
|
||||
Given a v3 LLM actor YAML file with name "local/detect-llm"
|
||||
When I run the actor add command for "local/detect-llm" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be validated via ActorConfigSchema
|
||||
|
||||
Scenario: Detect v3 YAML by type field (tool)
|
||||
Given a v3 TOOL actor YAML file with name "local/detect-tool"
|
||||
When I run the actor add command for "local/detect-tool" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be validated via ActorConfigSchema
|
||||
|
||||
Scenario: Detect v3 YAML by type field (graph)
|
||||
Given a v3 GRAPH actor YAML file with name "local/detect-graph"
|
||||
When I run the actor add command for "local/detect-graph" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be validated via ActorConfigSchema
|
||||
|
||||
# NOTE: The fixture includes both version and type fields so that
|
||||
# ActorConfigSchema.model_validate() succeeds. The version-only branch of
|
||||
# is_v3_yaml() is exercised by the "Detect v3 YAML by version" direct unit
|
||||
# scenarios below (using a config blob with only a version field).
|
||||
Scenario: Register v3 actor with both version and type fields
|
||||
Given a v3 actor YAML file with version "3.0"
|
||||
When I run the actor add command for "local/detect-version" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be validated via ActorConfigSchema
|
||||
|
||||
# ── Backward Compatibility: v2 Actors ─────────────────────────────────────
|
||||
|
||||
Scenario: Register a v2 actor without type field
|
||||
Given a v2 actor YAML file with only name, provider, and model fields
|
||||
When I run the actor add command for "local/v2-actor" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should NOT be validated via ActorConfigSchema
|
||||
|
||||
# ── Update Command v3 Validation ──────────────────────────────────────────
|
||||
|
||||
Scenario: Update a v3 actor with valid YAML succeeds
|
||||
Given a registered actor "local/existing-actor" already exists in the registry
|
||||
And a v3 LLM actor YAML file with name "local/existing-actor"
|
||||
When I run the actor update command for "local/existing-actor" with the prepared config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should be validated via ActorConfigSchema
|
||||
|
||||
Scenario: Update a v3 actor with invalid YAML (cycle) fails with error
|
||||
Given a registered actor "local/existing-graph" already exists in the registry
|
||||
And a v3 GRAPH actor YAML file with a cycle in the route
|
||||
When I run the actor update command for "local/existing-graph" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "cycle"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# is_v3_yaml Version Branch Edge Cases (Fix 2)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Detect v3 YAML by version "3" (short form)
|
||||
Given a config blob with only version field "3"
|
||||
When I call is_v3_yaml on the config blob
|
||||
Then is_v3_yaml should return True
|
||||
|
||||
Scenario: Do not detect v3 for version "30" (false positive guard)
|
||||
Given a config blob with only version field "30"
|
||||
When I call is_v3_yaml on the config blob
|
||||
Then is_v3_yaml should return False
|
||||
|
||||
Scenario: Do not detect v3 for null type field
|
||||
Given a config blob with a null type field
|
||||
When I call is_v3_yaml on the config blob
|
||||
Then is_v3_yaml should return False
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Provider-Required Validation (Fix 5)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 LLM actor without provider field
|
||||
Given a v3 LLM actor YAML file without the provider field named "local/no-provider-llm"
|
||||
When I run the actor add command for "local/no-provider-llm" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "provider"
|
||||
|
||||
Scenario: Reject v3 GRAPH actor without provider field
|
||||
Given a v3 GRAPH actor YAML file without the provider field named "local/no-provider-graph"
|
||||
When I run the actor add command for "local/no-provider-graph" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "provider"
|
||||
|
||||
# NOTE: CLI-level coverage for TOOL actors without provider is deferred.
|
||||
# v3 TOOL actors without 'provider' pass ActorConfigSchema validation (tested
|
||||
# below via direct schema call) but are rejected by the legacy v2
|
||||
# ActorConfiguration.from_blob() canonicalization layer that the CLI invokes.
|
||||
# This is a known limitation tracked in follow-up issue:
|
||||
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
|
||||
Scenario: Accept valid v3 TOOL actor without provider field (schema level)
|
||||
Given a valid v3 TOOL actor YAML without provider for schema validation
|
||||
When I validate the TOOL actor YAML blob directly via ActorConfigSchema
|
||||
Then the schema validation should succeed
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Registry-Level Direct Tests (Fix 6)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: ActorRegistry.add directly rejects v3 GRAPH actor with cycle
|
||||
Given a v3 GRAPH actor YAML with a cycle for direct registry testing named "local/registry-cycle-graph"
|
||||
When I call ActorRegistry.add directly with the YAML
|
||||
Then the registry should raise a ValidationError containing "cycle"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# M4: CLI-level test for type: null YAML (falls back to v2 handling)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: v3 actor with null type field falls back to v2 handling
|
||||
Given a v3 actor YAML file with null type field
|
||||
When I run the actor add command
|
||||
Then v3actor the command should succeed
|
||||
And the actor should NOT be validated via ActorConfigSchema
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# M5: Update without --config flag skips schema validation
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Update a registered v3 actor without --config flag skips schema validation
|
||||
Given a registered v3 LLM actor
|
||||
When I run the actor update command without a config file
|
||||
Then v3actor the command should succeed
|
||||
And the actor should NOT be validated via ActorConfigSchema
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# M6: CLI-level test for version "3" without type field
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 actor YAML with version "3" but no type field
|
||||
Given a v3 actor YAML file with version "3" but no type field
|
||||
When I run the actor add command
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "type"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# m5: Duplicate node IDs in GRAPH route
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 GRAPH actor YAML with duplicate node IDs
|
||||
Given a v3 GRAPH actor YAML file with duplicate node IDs
|
||||
When I run the actor add command for "local/dup-node-ids" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "Duplicate node IDs"
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# m6: Edge referencing non-existent node
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
Scenario: Reject v3 GRAPH actor YAML with edge referencing non-existent node
|
||||
Given a v3 GRAPH actor YAML file with an edge referencing a non-existent node
|
||||
When I run the actor add command for "local/bad-edge-ref" with the prepared config file
|
||||
Then v3actor the command should fail
|
||||
And v3actor the error message should contain "not found"
|
||||
@@ -71,6 +71,7 @@ Feature: Actor YAML examples validation
|
||||
type: llm
|
||||
description: Strategic planning and task decomposition
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
system_prompt: |
|
||||
You are a strategic planner. Break down goals into actionable steps.
|
||||
@@ -100,6 +101,7 @@ Feature: Actor YAML examples validation
|
||||
type: llm
|
||||
description: Code implementation and execution
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
system_prompt: "You are an expert software engineer."
|
||||
context_view: executor
|
||||
@@ -128,6 +130,7 @@ Feature: Actor YAML examples validation
|
||||
type: llm
|
||||
description: Code quality and standards review
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
system_prompt: "You are a senior code reviewer."
|
||||
context_view: reviewer
|
||||
@@ -197,6 +200,7 @@ Feature: Actor YAML examples validation
|
||||
type: llm
|
||||
description: Python code validation and linting
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
system_prompt: "You are a validation expert."
|
||||
context_view: reviewer
|
||||
@@ -225,6 +229,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Linear three-step review workflow
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -282,6 +287,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Workflow with conditional routing
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -335,6 +341,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Pipeline with tool execution
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -389,6 +396,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Hierarchical workflow with subgraphs
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -438,6 +446,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Workflow with retry logic
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -506,6 +515,7 @@ Feature: Actor YAML examples validation
|
||||
type: graph
|
||||
description: Multi-level planning and execution
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
|
||||
@@ -2,6 +2,7 @@ name: m2test/hierarchical-workflow
|
||||
type: graph
|
||||
description: M2 smoke test hierarchical actor with planner and executor nodes
|
||||
version: "1.0"
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
context_view: full
|
||||
memory:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -55,6 +55,7 @@ def _build_graph_config(
|
||||
name=name,
|
||||
type=ActorType.GRAPH,
|
||||
description="Coverage test graph actor",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
route=route,
|
||||
)
|
||||
@@ -360,6 +361,7 @@ def step_given_resolver_llm_type(context: Context, ref: str) -> None:
|
||||
name=ref,
|
||||
type=ActorType.LLM,
|
||||
description="An LLM actor",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
)
|
||||
context.actor_resolver = lambda name, _r=ref, _a=llm_actor: (
|
||||
|
||||
@@ -49,6 +49,7 @@ def _build_graph_config(
|
||||
name=name,
|
||||
type=ActorType.GRAPH,
|
||||
description="Test graph actor",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
route=route,
|
||||
)
|
||||
@@ -299,6 +300,7 @@ def step_given_llm_config(context: Context) -> None:
|
||||
name="assistants/simple",
|
||||
type=ActorType.LLM,
|
||||
description="Simple LLM",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
)
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ def _base_graph_yaml(
|
||||
"name": "local/hierarchy-test",
|
||||
"type": "graph",
|
||||
"description": "Test hierarchical actor",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": nodes,
|
||||
@@ -519,6 +520,7 @@ def step_hierarchy_loader_setup(context: Context) -> None:
|
||||
"name": "local/hierarchy-demo",
|
||||
"type": "graph",
|
||||
"description": "Hierarchical demo",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"skills": ["local/file-ops"],
|
||||
"lsp": ["local/pyright"],
|
||||
@@ -561,6 +563,7 @@ def step_hierarchy_loader_invalid(context: Context) -> None:
|
||||
"name": "local/bad-lsp",
|
||||
"type": "graph",
|
||||
"description": "Bad LSP binding",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": [
|
||||
|
||||
@@ -29,6 +29,7 @@ def _minimal_actor_yaml(
|
||||
'version: "1.0"',
|
||||
]
|
||||
if actor_type in ("llm", "graph"):
|
||||
lines.append("provider: openai")
|
||||
lines.append(f"model: {model}")
|
||||
if tools:
|
||||
lines.append("tools:")
|
||||
|
||||
@@ -9,7 +9,7 @@ from behave.runner import Context
|
||||
|
||||
from cleveragents.actor.registry import ActorRegistry
|
||||
from cleveragents.config.settings import ProviderDefaults
|
||||
from cleveragents.core.exceptions import ValidationError
|
||||
from cleveragents.core.exceptions import NotFoundError, ValidationError
|
||||
from cleveragents.domain.models.core.actor import Actor
|
||||
from cleveragents.providers.registry import ProviderInfo
|
||||
|
||||
@@ -74,7 +74,7 @@ class _StubActorService:
|
||||
|
||||
def get_actor(self, name: str) -> Actor:
|
||||
if name not in self.actors:
|
||||
raise ValueError(f"Actor '{name}' not found")
|
||||
raise NotFoundError(f"Actor '{name}' not found")
|
||||
return self.actors[name]
|
||||
|
||||
def list_actors(self) -> list[Actor]:
|
||||
@@ -82,7 +82,7 @@ class _StubActorService:
|
||||
|
||||
def remove_actor(self, name: str) -> None:
|
||||
if name not in self.actors:
|
||||
raise ValueError(f"Actor '{name}' not found")
|
||||
raise NotFoundError(f"Actor '{name}' not found")
|
||||
del self.actors[name]
|
||||
if self.default_actor_name == name:
|
||||
self.default_actor_name = None
|
||||
|
||||
@@ -24,6 +24,7 @@ _MINIMAL_LLM_YAML = """\
|
||||
name: assistants/simple
|
||||
type: llm
|
||||
description: A simple LLM actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
"""
|
||||
|
||||
@@ -31,6 +32,7 @@ _LLM_WITH_PROMPT_YAML = """\
|
||||
name: assistants/expert
|
||||
type: llm
|
||||
description: An expert assistant
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
system_prompt: "You are an expert Python developer"
|
||||
"""
|
||||
@@ -39,6 +41,7 @@ _LLM_WITH_TOOLS_YAML = """\
|
||||
name: assistants/helper
|
||||
type: llm
|
||||
description: Helper with tools
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
tools:
|
||||
- files/read_file
|
||||
@@ -49,6 +52,7 @@ _LLM_WITH_MEMORY_YAML = """\
|
||||
name: assistants/chatbot
|
||||
type: llm
|
||||
description: Chatbot with memory
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
memory:
|
||||
enabled: true
|
||||
@@ -60,6 +64,7 @@ _LLM_WITH_CONTEXT_YAML = """\
|
||||
name: assistants/analyzer
|
||||
type: llm
|
||||
description: Code analyzer
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
context:
|
||||
include_files:
|
||||
@@ -101,6 +106,7 @@ _GRAPH_MINIMAL_YAML = """\
|
||||
name: workflows/simple
|
||||
type: graph
|
||||
description: Simple workflow
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -120,6 +126,7 @@ _GRAPH_LINEAR_YAML = """\
|
||||
name: workflows/linear
|
||||
type: graph
|
||||
description: Linear workflow
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -155,6 +162,7 @@ _GRAPH_CONDITIONAL_YAML = """\
|
||||
name: workflows/conditional
|
||||
type: graph
|
||||
description: Workflow with conditionals
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -199,6 +207,7 @@ _GRAPH_SUBGRAPH_YAML = """\
|
||||
name: workflows/composed
|
||||
type: graph
|
||||
description: Workflow with subgraph
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -305,6 +314,7 @@ def step_given_actor_model_for_role_warning(context: Context) -> None:
|
||||
name="local/model-warning-actor",
|
||||
type="llm",
|
||||
description="Model-input warnings path",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
role_hint="estimation",
|
||||
context_view="executor",
|
||||
@@ -319,6 +329,7 @@ def step_given_actor_model_without_response_format(context: Context) -> None:
|
||||
name="local/model-warning-no-schema",
|
||||
type="llm",
|
||||
description="Model-input missing response_format",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
role_hint="estimation",
|
||||
context_view="strategist",
|
||||
@@ -732,6 +743,7 @@ def step_given_specific_node_id(context: Context, node_id: str) -> None:
|
||||
name: workflows/test
|
||||
type: graph
|
||||
description: Test workflow
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -755,6 +767,7 @@ def step_given_context_view(context: Context, view: str) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
context_view: {view}
|
||||
"""
|
||||
@@ -767,6 +780,7 @@ def step_given_env_vars(context: Context) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
env_vars:
|
||||
LOG_LEVEL: info
|
||||
@@ -787,6 +801,7 @@ def step_given_valid_actor_object(context: Context) -> None:
|
||||
name="test/actor",
|
||||
type="llm",
|
||||
description="Test actor",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
)
|
||||
|
||||
@@ -804,6 +819,7 @@ def step_given_edge_priorities(context: Context) -> None:
|
||||
name: workflows/priorities
|
||||
type: graph
|
||||
description: Workflow with priorities
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
route:
|
||||
nodes:
|
||||
@@ -836,6 +852,7 @@ def step_given_memory_disabled(context: Context) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
memory:
|
||||
enabled: false
|
||||
@@ -849,6 +866,7 @@ def step_given_max_messages(context: Context, count: int) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
memory:
|
||||
max_messages: {count}
|
||||
@@ -862,6 +880,7 @@ def step_given_max_tokens(context: Context, count: int) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
memory:
|
||||
max_tokens: {count}
|
||||
@@ -875,6 +894,7 @@ def step_given_summarize_old(context: Context) -> None:
|
||||
name: assistants/test
|
||||
type: llm
|
||||
description: Test actor
|
||||
provider: openai
|
||||
model: gpt-4
|
||||
memory:
|
||||
summarize_old: true
|
||||
|
||||
@@ -170,6 +170,7 @@ def step_pfg_estimation_actor_model_missing_response_format(context: Context) ->
|
||||
name="local/estimator-model",
|
||||
type="llm",
|
||||
description="Model-based estimation actor",
|
||||
provider="openai",
|
||||
model="gpt-4",
|
||||
role_hint="estimation",
|
||||
context_view="strategist",
|
||||
|
||||
@@ -0,0 +1,335 @@
|
||||
*** Settings ***
|
||||
Documentation Integration tests for actor add CLI v3 schema validation
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment With Database Isolation
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Variables ***
|
||||
${HELPER} ${CURDIR}/helper_actor_add_v3_schema_validation.py
|
||||
|
||||
*** Test Cases ***
|
||||
Register Valid v3 LLM Actor Via CLI
|
||||
[Documentation] Register a valid v3 LLM actor via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_llm.yaml
|
||||
Create File ${yaml_file} name: local/test-llm\ntype: llm\ndescription: Test LLM\nprovider: openai\nmodel: gpt-4\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/test-llm ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} actor-add-success
|
||||
|
||||
Register Valid v3 TOOL Actor Via CLI
|
||||
[Documentation] Register a valid v3 TOOL actor via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_tool.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/test-tool
|
||||
... type: tool
|
||||
... description: Test TOOL
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... tools:
|
||||
... ${SPACE}${SPACE}- files/read_file
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/test-tool ${yaml_file} cwd=${WORKSPACE}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} actor-add-success
|
||||
|
||||
Register Valid v3 GRAPH Actor Via CLI
|
||||
[Documentation] Register a valid v3 GRAPH actor via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}valid_graph.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/test-graph
|
||||
... type: graph
|
||||
... description: Test GRAPH
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges: []
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/test-graph ${yaml_file} cwd=${WORKSPACE}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} actor-add-success
|
||||
|
||||
Reject v3 GRAPH Actor With Cycle In Route
|
||||
[Documentation] Reject v3 GRAPH actor with cycle in route via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}broken_graph_cycle.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/broken-graph
|
||||
... type: graph
|
||||
... description: Graph with cycle
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node B
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Second node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_a
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/broken-graph ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} cycle
|
||||
|
||||
Reject v3 LLM Actor Without Model Field
|
||||
[Documentation] Reject v3 LLM actor without model field via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_llm_no_model.yaml
|
||||
Create File ${yaml_file} name: local/no-model-llm\ntype: llm\ndescription: LLM without model\nprovider: openai\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-llm ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} LLM actors require 'model' field
|
||||
|
||||
Reject v3 TOOL Actor Without Tools Field
|
||||
[Documentation] Reject v3 TOOL actor without tools field via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_tool_no_tools.yaml
|
||||
Create File ${yaml_file} name: local/no-tools-tool\ntype: tool\ndescription: TOOL without tools\nprovider: openai\nmodel: gpt-4\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/no-tools-tool ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} TOOL actors require at least one tool
|
||||
|
||||
Reject v3 GRAPH Actor Without Route Field
|
||||
[Documentation] Reject v3 GRAPH actor without route field via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_route.yaml
|
||||
Create File ${yaml_file} name: local/no-route-graph\ntype: graph\ndescription: GRAPH without route\nprovider: openai\nmodel: gpt-4\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/no-route-graph ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} GRAPH actors require 'route' field
|
||||
|
||||
Reject v3 GRAPH Actor Without Model Field
|
||||
[Documentation] Reject v3 GRAPH actor without model field via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_graph_no_model.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/no-model-graph
|
||||
... type: graph
|
||||
... description: GRAPH without model
|
||||
... provider: openai
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges: []
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/no-model-graph ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} GRAPH actors require 'model' field
|
||||
|
||||
Reject v3 Actor With Invalid Type Enum
|
||||
[Documentation] Reject v3 actor with invalid type enum via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_type.yaml
|
||||
Create File ${yaml_file} name: local/invalid-type\ntype: invalid_type\ndescription: Invalid type\nprovider: openai\nmodel: gpt-4\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-type ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} Input should be 'llm', 'tool' or 'graph'
|
||||
|
||||
Reject v3 Actor With Invalid Context View Enum
|
||||
[Documentation] Reject v3 actor with invalid context_view enum via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_context_view.yaml
|
||||
Create File ${yaml_file} name: local/invalid-view\ntype: llm\ndescription: Invalid context_view\nprovider: openai\nmodel: gpt-4\ncontext_view: invalid_view\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/invalid-view ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} context_view
|
||||
Should Contain ${result.stderr} Input should be
|
||||
|
||||
Reject v3 GRAPH Actor With Cycle Via Update Command
|
||||
[Documentation] Reject v3 GRAPH actor with cycle via agents actor update
|
||||
[Tags] slow
|
||||
# First register a valid v3 GRAPH actor so that the update command can
|
||||
# resolve it. The update command calls registry.get_actor(name) before
|
||||
# validating the new config — without this step the command aborts with
|
||||
# "Actor not found" instead of a cycle validation error.
|
||||
${valid_yaml_file}= Set Variable ${TEMPDIR}${/}update_valid_graph.yaml
|
||||
${valid_yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/update-test-graph
|
||||
... type: graph
|
||||
... description: Valid graph actor for update test setup
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges: []
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_a
|
||||
Create File ${valid_yaml_file} ${valid_yaml_content}
|
||||
${setup_result}= Run Process ${PYTHON} ${HELPER} add local/update-test-graph ${valid_yaml_file} cwd=${WORKSPACE}
|
||||
Log ${setup_result.stdout}
|
||||
Log ${setup_result.stderr}
|
||||
Should Be Equal As Integers ${setup_result.rc} 0
|
||||
# Now attempt the update with a cyclic YAML — should fail with cycle error
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}update_broken_cycle.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/update-test-graph
|
||||
... type: graph
|
||||
... description: Graph with cycle for update test
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node A
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: First node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Node B
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Second node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_a
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} update local/update-test-graph ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} cycle
|
||||
|
||||
Reject v3 GRAPH Actor With Invalid Node ID
|
||||
[Documentation] Reject v3 GRAPH actor with non-alphanumeric node ID via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}invalid_node_id.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/bad-node-id
|
||||
... type: graph
|
||||
... description: Graph with invalid node ID
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: "bad!node"
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Bad Node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Node with invalid ID
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges: []
|
||||
... ${SPACE}${SPACE}entry_node: "bad!node"
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- "bad!node"
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/bad-node-id ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} Node ID must be alphanumeric
|
||||
|
||||
Reject v3 TOOL Actor With Unnamespaced Tool Name
|
||||
[Documentation] Reject v3 TOOL actor with unnamespaced tool name via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}unnamespaced_tool.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/bad-tool-name
|
||||
... type: tool
|
||||
... description: TOOL with unnamespaced tool
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... tools:
|
||||
... ${SPACE}${SPACE}- read_file
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/bad-tool-name ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} must be namespaced
|
||||
|
||||
Reject v3 GRAPH Actor With Unreachable Node
|
||||
[Documentation] Reject v3 GRAPH actor with unreachable node via agents actor add
|
||||
[Tags] slow
|
||||
${yaml_file}= Set Variable ${TEMPDIR}${/}unreachable_node.yaml
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: local/unreachable-node
|
||||
... type: graph
|
||||
... description: Graph with unreachable node
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... route:
|
||||
... ${SPACE}${SPACE}nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Entry Node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Entry node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_b
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Exit Node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Exit node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- id: node_orphan
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}type: agent
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}name: Orphan Node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}description: Unreachable node
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}config: {}
|
||||
... ${SPACE}${SPACE}edges:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- from_node: node_a
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}to_node: node_b
|
||||
... ${SPACE}${SPACE}entry_node: node_a
|
||||
... ${SPACE}${SPACE}exit_nodes:
|
||||
... ${SPACE}${SPACE}${SPACE}${SPACE}- node_b
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${result}= Run Process ${PYTHON} ${HELPER} add local/unreachable-node ${yaml_file} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 1
|
||||
Should Contain ${result.stderr} unreachable
|
||||
@@ -21,7 +21,7 @@ Discover Actors From Custom Directory
|
||||
${tmp}= Set Variable ${TEMPDIR}${/}actor_loading_robot
|
||||
Run Keyword And Ignore Error Remove Directory ${tmp} recursive=True
|
||||
Create Directory ${tmp}
|
||||
Create File ${tmp}${/}test.yaml name: local/robot-test\ntype: llm\ndescription: Robot test actor\nversion: "1.0"\nmodel: gpt-4\n
|
||||
Create File ${tmp}${/}test.yaml name: local/robot-test\ntype: llm\ndescription: Robot test actor\nversion: "1.0"\nprovider: openai\nmodel: gpt-4\n
|
||||
${result}= Run Process ${PYTHON} ${HELPER} discover ${tmp} cwd=${WORKSPACE}
|
||||
Log ${result.stdout}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Helper script for actor add v3 schema validation robot tests.
|
||||
|
||||
Tests the agents actor add CLI command with v3 YAML files to ensure
|
||||
ActorConfigSchema validation is applied (cycle detection, required fields, etc.).
|
||||
|
||||
This helper delegates to the real CLI binary so that it exercises the actual
|
||||
command path — not internal helpers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def add_actor(actor_name: str, config_file: str) -> int:
|
||||
"""Test adding an actor via the real CLI binary.
|
||||
|
||||
Invokes ``agents actor add <actor_name> --config <config_file>`` as a
|
||||
subprocess so that all CLI-level validation (including v3 schema
|
||||
validation) runs for real.
|
||||
|
||||
Args:
|
||||
actor_name: The actor name (e.g., local/test-llm)
|
||||
config_file: Path to the YAML config file
|
||||
|
||||
Returns:
|
||||
0 on success, 1 on validation failure or timeout
|
||||
"""
|
||||
config_path = Path(config_file)
|
||||
if not config_path.exists():
|
||||
print(
|
||||
f"actor-add-error: Config file not found: {config_file}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["agents", "actor", "add", actor_name, "--config", config_file],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
)
|
||||
except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
|
||||
print(f"actor-add-error: command failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if result.returncode == 0:
|
||||
print("actor-add-success")
|
||||
return 0
|
||||
else:
|
||||
output = result.stdout + result.stderr
|
||||
print(f"actor-add-error: {output.strip()}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def update_actor(actor_name: str, config_file: str) -> int:
|
||||
"""Test updating an actor via the real CLI binary.
|
||||
|
||||
Invokes ``agents actor update <actor_name> --config <config_file>`` as a
|
||||
subprocess so that all CLI-level validation (including v3 schema
|
||||
validation) runs for real.
|
||||
|
||||
Args:
|
||||
actor_name: The actor name (e.g., local/test-llm)
|
||||
config_file: Path to the YAML config file
|
||||
|
||||
Returns:
|
||||
0 on success, 1 on validation failure or timeout
|
||||
"""
|
||||
config_path = Path(config_file)
|
||||
if not config_path.exists():
|
||||
print(
|
||||
f"actor-update-error: Config file not found: {config_file}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["agents", "actor", "update", actor_name, "--config", config_file],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
)
|
||||
except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
|
||||
print(f"actor-update-error: command failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if result.returncode == 0:
|
||||
print("actor-update-success")
|
||||
return 0
|
||||
else:
|
||||
output = result.stdout + result.stderr
|
||||
print(f"actor-update-error: {output.strip()}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def main() -> int:
|
||||
"""Main entry point."""
|
||||
if len(sys.argv) < 4:
|
||||
print(
|
||||
"Usage: helper_actor_add_v3_schema_validation.py"
|
||||
" <command> <actor_name> <config_file>"
|
||||
)
|
||||
return 1
|
||||
|
||||
command = sys.argv[1]
|
||||
actor_name = sys.argv[2]
|
||||
config_file = sys.argv[3]
|
||||
|
||||
if command == "add":
|
||||
return add_actor(actor_name, config_file)
|
||||
elif command == "update":
|
||||
return update_actor(actor_name, config_file)
|
||||
else:
|
||||
print(f"Unknown command: {command}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -61,6 +61,7 @@ def _parse_lsp_binding() -> int:
|
||||
"name": "local/lsp-test",
|
||||
"type": "graph",
|
||||
"description": "LSP binding test",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": [
|
||||
@@ -94,6 +95,7 @@ def _parse_tool_sources() -> int:
|
||||
"name": "local/ts-test",
|
||||
"type": "graph",
|
||||
"description": "Tool sources test",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": [
|
||||
@@ -125,6 +127,7 @@ def _parse_subgraph_ref() -> int:
|
||||
"name": "local/sg-test",
|
||||
"type": "graph",
|
||||
"description": "Subgraph ref test",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": [
|
||||
@@ -159,6 +162,7 @@ def _reject_bad_lsp() -> int:
|
||||
"name": "local/bad-lsp-test",
|
||||
"type": "graph",
|
||||
"description": "Bad LSP test",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4",
|
||||
"route": {
|
||||
"nodes": [
|
||||
|
||||
@@ -100,6 +100,7 @@ def _namespace_default() -> int:
|
||||
"type: llm\n"
|
||||
"description: Test namespace defaulting\n"
|
||||
'version: "1.0"\n'
|
||||
"provider: openai\n"
|
||||
"model: gpt-4\n"
|
||||
)
|
||||
(tmp / "bare.yaml").write_text(yaml_content)
|
||||
|
||||
@@ -119,6 +119,7 @@ def actor_yaml_create_load() -> None:
|
||||
"type: graph\n"
|
||||
"description: M2 E2E test actor\n"
|
||||
'version: "1.0"\n'
|
||||
"provider: openai\n"
|
||||
"model: gpt-4\n"
|
||||
"route:\n"
|
||||
" nodes:\n"
|
||||
@@ -299,6 +300,7 @@ def actor_yaml_parse_validate() -> None:
|
||||
"type: graph\n"
|
||||
"description: Valid actor\n"
|
||||
'version: "1.0"\n'
|
||||
"provider: openai\n"
|
||||
"model: gpt-4\n"
|
||||
"route:\n"
|
||||
" nodes:\n"
|
||||
@@ -327,6 +329,7 @@ def actor_yaml_parse_validate() -> None:
|
||||
"type: llm\n"
|
||||
"description: LLM actor\n"
|
||||
'version: "1.0"\n'
|
||||
"provider: openai\n"
|
||||
"model: gpt-4\n"
|
||||
)
|
||||
(tmp / "llm.yaml").write_text(llm_yaml)
|
||||
@@ -373,6 +376,7 @@ def actor_compile_stategraph() -> None:
|
||||
"type: llm\n"
|
||||
"description: LLM only actor\n"
|
||||
'version: "1.0"\n'
|
||||
"provider: openai\n"
|
||||
"model: gpt-4\n"
|
||||
)
|
||||
(tmp / "llm.yaml").write_text(llm_yaml)
|
||||
|
||||
@@ -5,10 +5,13 @@ from __future__ import annotations
|
||||
from dataclasses import asdict
|
||||
from typing import Any
|
||||
|
||||
import pydantic
|
||||
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
from cleveragents.actor.schema import ActorConfigSchema, is_v3_yaml
|
||||
from cleveragents.application.services.actor_service import ActorService
|
||||
from cleveragents.config.settings import Settings
|
||||
from cleveragents.core.exceptions import ValidationError
|
||||
from cleveragents.core.exceptions import NotFoundError, ValidationError
|
||||
from cleveragents.domain.models.core import Actor
|
||||
from cleveragents.providers.registry import (
|
||||
ProviderCapabilities,
|
||||
@@ -189,6 +192,11 @@ class ActorRegistry:
|
||||
The YAML is parsed into an ``ActorConfiguration``, validated, and
|
||||
persisted alongside the original *yaml_text* and *schema_version*.
|
||||
|
||||
For v3 YAML files (detected by any non-null ``type`` field or a
|
||||
``version`` starting with ``'3'``), the configuration is validated
|
||||
using ``ActorConfigSchema`` to ensure proper schema compliance,
|
||||
including cycle detection for GRAPH actors.
|
||||
|
||||
Args:
|
||||
yaml_text: The original actor YAML source.
|
||||
update: When ``True`` allow overwriting an existing actor.
|
||||
@@ -210,29 +218,42 @@ class ActorRegistry:
|
||||
if not name_raw:
|
||||
raise ValidationError("Actor YAML must include a 'name' field.")
|
||||
|
||||
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
|
||||
# This ensures v3 actors are validated against the full schema, including
|
||||
# cycle detection, required fields, and enum validation.
|
||||
blob_is_v3 = is_v3_yaml(blob)
|
||||
if blob_is_v3:
|
||||
try:
|
||||
ActorConfigSchema.model_validate(blob)
|
||||
except pydantic.ValidationError as exc:
|
||||
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
|
||||
|
||||
name = self._ensure_namespaced(name_raw)
|
||||
provider_raw = blob.get("provider") or blob.get("provider_type", "")
|
||||
model_raw = blob.get("model") or blob.get("model_id", "")
|
||||
|
||||
if not provider_raw or not model_raw:
|
||||
if not blob_is_v3 and (not provider_raw or not model_raw):
|
||||
raise ValidationError(
|
||||
"Actor YAML must include 'provider' and 'model' fields."
|
||||
)
|
||||
|
||||
provider = str(provider_raw)
|
||||
model = str(model_raw)
|
||||
# For v3 actors without provider/model (e.g. TOOL actors), provider_raw
|
||||
# and model_raw may be empty strings here. This is expected — the legacy
|
||||
# canonicalization path in upsert_actor() / from_blob() will reject
|
||||
# provider-less TOOL actors. See follow-up issue #9971.
|
||||
provider = str(provider_raw) if provider_raw else ""
|
||||
model = str(model_raw) if model_raw else ""
|
||||
|
||||
# Check for existing actor when not updating
|
||||
if not update:
|
||||
try:
|
||||
self._actor_service.get_actor(name)
|
||||
except NotFoundError:
|
||||
pass # Actor does not exist yet — proceed with add
|
||||
else:
|
||||
raise ValidationError(
|
||||
f"Actor '{name}' already exists. Pass update=True to overwrite."
|
||||
)
|
||||
except Exception as exc:
|
||||
if "already exists" in str(exc):
|
||||
raise
|
||||
# NotFoundError is expected for new actors
|
||||
|
||||
version = schema_version or self.DEFAULT_SCHEMA_VERSION
|
||||
config_blob: dict[str, Any] = dict(blob)
|
||||
@@ -273,9 +294,31 @@ class ActorRegistry:
|
||||
schema_version: str | None = None,
|
||||
compiled_metadata: dict[str, Any] | None = None,
|
||||
) -> Actor:
|
||||
"""Parse, validate, and persist an actor configuration."""
|
||||
"""Parse, validate, and persist an actor configuration.
|
||||
|
||||
For v3 YAML files (detected by any non-null ``type`` field or a
|
||||
``version`` starting with ``'3'``), the configuration is validated
|
||||
using ``ActorConfigSchema`` to ensure proper schema compliance,
|
||||
including cycle detection for GRAPH actors.
|
||||
"""
|
||||
|
||||
self.ensure_built_in_actors()
|
||||
|
||||
# ── Validate v3 YAML via ActorConfigSchema if detected ──────────────────
|
||||
# This ensures v3 actors are validated against the full schema, including
|
||||
# cycle detection, required fields, and enum validation.
|
||||
if config_blob and is_v3_yaml(config_blob):
|
||||
try:
|
||||
ActorConfigSchema.model_validate(config_blob)
|
||||
except pydantic.ValidationError as exc:
|
||||
raise ValidationError(f"Invalid v3 actor configuration: {exc}") from exc
|
||||
|
||||
# NOTE: v3 TOOL actors without 'provider' will be rejected here by the legacy
|
||||
# v2 canonicalization layer (ActorConfiguration.from_blob requires provider).
|
||||
# This is a known limitation tracked in follow-up issue:
|
||||
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
|
||||
# v3 schema validation (above) already accepts provider-less TOOL actors
|
||||
# correctly; the gap is only in the legacy from_blob() canonicalization.
|
||||
config = ActorConfiguration.from_blob(
|
||||
blob=config_blob,
|
||||
name=name,
|
||||
|
||||
@@ -544,6 +544,30 @@ class RouteDefinition(BaseModel):
|
||||
raise ValueError(msg)
|
||||
return v
|
||||
|
||||
def _build_adjacency(self) -> dict[str, list[str]]:
|
||||
"""Build the full adjacency list for the route graph.
|
||||
|
||||
Includes both explicit edges AND implicit ``route_to`` targets
|
||||
embedded in CONDITIONAL node configs so that both
|
||||
:meth:`detect_cycles` and :meth:`validate_references` use an
|
||||
identical, complete view of the graph topology.
|
||||
|
||||
Returns:
|
||||
Mapping from node ID to list of reachable neighbour node IDs.
|
||||
"""
|
||||
node_ids = {node.id for node in self.nodes}
|
||||
adj: dict[str, list[str]] = {node.id: [] for node in self.nodes}
|
||||
for edge in self.edges:
|
||||
adj[edge.from_node].append(edge.to_node)
|
||||
for node in self.nodes:
|
||||
if node.type == NodeType.CONDITIONAL and isinstance(node.config, dict):
|
||||
for cond in node.config.get("conditions", []):
|
||||
if isinstance(cond, dict) and "route_to" in cond:
|
||||
target = str(cond["route_to"])
|
||||
if target in node_ids:
|
||||
adj[node.id].append(target)
|
||||
return adj
|
||||
|
||||
def validate_references(self) -> None:
|
||||
"""
|
||||
Validate all node references in edges and entry/exit points.
|
||||
@@ -585,18 +609,9 @@ class RouteDefinition(BaseModel):
|
||||
raise ValueError(msg)
|
||||
|
||||
# Check all nodes are reachable from entry_node via BFS.
|
||||
# Adjacency is built from both explicit edges and implicit routing
|
||||
# targets embedded in conditional node configs (route_to keys).
|
||||
adj: dict[str, list[str]] = {node.id: [] for node in self.nodes}
|
||||
for edge in self.edges:
|
||||
adj[edge.from_node].append(edge.to_node)
|
||||
for node in self.nodes:
|
||||
if node.type == NodeType.CONDITIONAL and isinstance(node.config, dict):
|
||||
for cond in node.config.get("conditions", []):
|
||||
if isinstance(cond, dict) and "route_to" in cond:
|
||||
target = str(cond["route_to"])
|
||||
if target in node_ids:
|
||||
adj[node.id].append(target)
|
||||
# Use _build_adjacency() to include both explicit edges and implicit
|
||||
# route_to targets from CONDITIONAL nodes.
|
||||
adj = self._build_adjacency()
|
||||
|
||||
reachable: set[str] = set()
|
||||
queue: list[str] = [self.entry_node]
|
||||
@@ -622,10 +637,17 @@ class RouteDefinition(BaseModel):
|
||||
"""
|
||||
Detect cycles in the graph using DFS.
|
||||
|
||||
Only explicit edges are considered for cycle detection — conditional
|
||||
``route_to`` routing targets are intentional back-references (e.g. retry
|
||||
loops) and are not treated as cycles.
|
||||
|
||||
Returns:
|
||||
List of node IDs involved in cycles (empty if acyclic)
|
||||
"""
|
||||
# Build adjacency list
|
||||
# Build adjacency list from explicit edges only (not route_to targets).
|
||||
# _build_adjacency() is used by validate_references() for reachability
|
||||
# but is intentionally NOT used here to avoid false positive cycle
|
||||
# detection from CONDITIONAL node retry patterns.
|
||||
graph: dict[str, list[str]] = {node.id: [] for node in self.nodes}
|
||||
for edge in self.edges:
|
||||
graph[edge.from_node].append(edge.to_node)
|
||||
@@ -675,6 +697,9 @@ class ActorConfigSchema(BaseModel):
|
||||
type: Actor type (LLM, TOOL, or GRAPH)
|
||||
description: What the actor does
|
||||
version: Schema version (default: "1.0")
|
||||
provider: LLM provider name (e.g. "openai", "anthropic"); required for
|
||||
LLM and GRAPH actors, optional for TOOL actors which have no direct
|
||||
LLM invocation.
|
||||
model: LLM model name (required for LLM/GRAPH types)
|
||||
system_prompt: System prompt for LLM actors
|
||||
tools: List of tool references or inline definitions
|
||||
@@ -685,9 +710,11 @@ class ActorConfigSchema(BaseModel):
|
||||
env_vars: Environment variable mappings
|
||||
|
||||
Type-Specific Requirements:
|
||||
ActorType.LLM: Requires model, optional system_prompt and tools
|
||||
ActorType.TOOL: Requires tools list
|
||||
ActorType.GRAPH: Requires model and route
|
||||
ActorType.LLM: Requires ``provider`` and ``model``; optional
|
||||
``system_prompt`` and ``tools``
|
||||
ActorType.TOOL: Requires at least one entry in ``tools``; ``provider``
|
||||
is optional (TOOL actors do not make direct LLM calls)
|
||||
ActorType.GRAPH: Requires ``provider``, ``model``, and ``route``
|
||||
|
||||
Examples:
|
||||
>>> # Simple LLM actor
|
||||
@@ -714,6 +741,13 @@ class ActorConfigSchema(BaseModel):
|
||||
description: str = Field(..., description="Actor description")
|
||||
version: str = Field(default="1.0", description="Schema version")
|
||||
|
||||
# Provider is required for LLM and GRAPH actors (validated by
|
||||
# validate_provider_required_for_llm_graph below), but optional for TOOL
|
||||
# actors which have no direct LLM invocation and do not need a provider.
|
||||
provider: str | None = Field(
|
||||
default=None, description="LLM provider (e.g. openai, anthropic)"
|
||||
)
|
||||
|
||||
# LLM configuration
|
||||
model: str | None = Field(default=None, description="LLM model name")
|
||||
system_prompt: str | None = Field(default=None, description="System prompt")
|
||||
@@ -831,6 +865,21 @@ class ActorConfigSchema(BaseModel):
|
||||
msg = "TOOL actors require at least one tool"
|
||||
raise ValueError(msg)
|
||||
|
||||
# Validate string tool references: must follow "namespace/name" format.
|
||||
# ToolDefinition instances are validated separately by their own field
|
||||
# validators. Use strict two-part split to catch empty namespace or
|
||||
# empty name (e.g. "/tool", "namespace/", or "a/b/c").
|
||||
for tool_ref in self.tools:
|
||||
if isinstance(tool_ref, str):
|
||||
parts = tool_ref.split("/")
|
||||
if len(parts) != 2 or not parts[0] or not parts[1]:
|
||||
msg = (
|
||||
f"tool '{tool_ref}' must be namespaced "
|
||||
f"(namespace/name format, e.g. 'files/read_file'). "
|
||||
f"Namespace and name must both be non-empty."
|
||||
)
|
||||
raise ValueError(msg)
|
||||
|
||||
# GRAPH actors require model and route
|
||||
if self.type == ActorType.GRAPH:
|
||||
if not self.model:
|
||||
@@ -844,16 +893,25 @@ class ActorConfigSchema(BaseModel):
|
||||
self.route.validate_references()
|
||||
cycles = self.route.detect_cycles()
|
||||
if cycles:
|
||||
nodes_str = " → ".join(cycles)
|
||||
msg = (
|
||||
f"route: graph contains a cycle involving "
|
||||
f"nodes: [{nodes_str}]. "
|
||||
f"route: graph contains a cycle — "
|
||||
f"back-edge detected at node '{cycles[0]}'. "
|
||||
f"Hint: remove or redirect edges to break the cycle."
|
||||
)
|
||||
raise ValueError(msg)
|
||||
|
||||
return self
|
||||
|
||||
# Runs after validate_type_requirements — provider check is secondary to
|
||||
# type-specific field requirements.
|
||||
@model_validator(mode="after")
|
||||
def validate_provider_required_for_llm_graph(self) -> ActorConfigSchema:
|
||||
"""Require provider for LLM and GRAPH actor types."""
|
||||
if self.type in (ActorType.LLM, ActorType.GRAPH) and not self.provider:
|
||||
msg = "LLM and GRAPH actors require 'provider' field"
|
||||
raise ValueError(msg)
|
||||
return self
|
||||
|
||||
@classmethod
|
||||
def from_yaml_file(cls, file_path: str | Path) -> ActorConfigSchema:
|
||||
"""
|
||||
@@ -906,6 +964,42 @@ class ActorConfigSchema(BaseModel):
|
||||
)
|
||||
|
||||
|
||||
def is_v3_yaml(config_blob: dict[str, Any] | None) -> bool:
|
||||
"""Detect if a config blob is v3 YAML.
|
||||
|
||||
A config blob is considered v3 YAML if it contains ANY ``type`` field
|
||||
(any value — let the schema validate whether the value is legal) or a
|
||||
``version`` field whose string representation starts with ``"3"`` (e.g.
|
||||
``"3"``, ``"3.0"``, ``"3.0.0"``).
|
||||
|
||||
Treating any ``type`` field as a v3 signal prevents configs with invalid
|
||||
type values (e.g. ``type: robot``) from bypassing schema validation.
|
||||
|
||||
Args:
|
||||
config_blob: The parsed YAML/JSON configuration dictionary.
|
||||
|
||||
Returns:
|
||||
True if the config appears to be v3 YAML, False otherwise.
|
||||
"""
|
||||
if not isinstance(config_blob, dict):
|
||||
return False
|
||||
|
||||
# Any non-null 'type' field = v3 YAML; let the schema validate the value.
|
||||
# Explicitly exclude type: null so that misconfigured blobs with a null
|
||||
# type key do not accidentally trigger v3 schema validation.
|
||||
if "type" in config_blob and config_blob["type"] is not None:
|
||||
return True
|
||||
|
||||
# Check for version field indicating v3 (future-proof: "3", "3.0", "3.0.0").
|
||||
# Use exact equality for "3" and prefix check for "3." to avoid false
|
||||
# positives from version strings like "30" or "300".
|
||||
version = config_blob.get("version")
|
||||
if version is None:
|
||||
return False
|
||||
version_str = str(version)
|
||||
return version_str == "3" or version_str.startswith("3.")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ActorConfigSchema",
|
||||
"ActorType",
|
||||
@@ -923,4 +1017,5 @@ __all__ = [
|
||||
"ToolParameter",
|
||||
"ToolSourceRef",
|
||||
"actor_role_warnings",
|
||||
"is_v3_yaml", # Internal helper; not re-exported from actor/__init__.py
|
||||
]
|
||||
|
||||
@@ -6,6 +6,7 @@ from pathlib import Path
|
||||
from typing import Annotated, Any, cast
|
||||
|
||||
import click
|
||||
import pydantic
|
||||
import typer
|
||||
import yaml
|
||||
from rich.console import Console
|
||||
@@ -13,7 +14,7 @@ from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
|
||||
from cleveragents.actor.config import ActorConfiguration
|
||||
from cleveragents.actor.schema import actor_role_warnings
|
||||
from cleveragents.actor.schema import ActorConfigSchema, actor_role_warnings, is_v3_yaml
|
||||
from cleveragents.application.container import get_container
|
||||
from cleveragents.cli.commands._resolve_actor import (
|
||||
resolve_config_files as _resolve_config_files,
|
||||
@@ -345,6 +346,32 @@ def _parse_option_overrides(option_values: list[str] | None) -> dict[str, Any]:
|
||||
return overrides
|
||||
|
||||
|
||||
def _validate_v3_config(source_path: Path, config_blob: dict[str, Any]) -> None:
|
||||
"""Validate a v3 config blob using ActorConfigSchema.
|
||||
|
||||
Uses the already-parsed ``config_blob`` to avoid re-reading the file
|
||||
(eliminates TOCTOU: the config the CLI parsed is the config that gets
|
||||
validated, not a potentially-different file re-read at validation time).
|
||||
|
||||
Only ``pydantic.ValidationError`` is caught here because
|
||||
``model_validate(dict)`` cannot raise ``yaml.YAMLError`` or ``OSError``
|
||||
— the YAML parsing has already happened before this function is called.
|
||||
|
||||
Args:
|
||||
source_path: Path to the config file (used only in error messages).
|
||||
config_blob: The parsed YAML/JSON configuration dictionary.
|
||||
|
||||
Raises:
|
||||
typer.BadParameter: If schema validation fails.
|
||||
"""
|
||||
try:
|
||||
ActorConfigSchema.model_validate(config_blob)
|
||||
except pydantic.ValidationError as exc:
|
||||
raise typer.BadParameter(
|
||||
f"Invalid actor configuration in {source_path}: {exc}"
|
||||
) from exc
|
||||
|
||||
|
||||
def _canonicalize_actor_config(
|
||||
*,
|
||||
name: str,
|
||||
@@ -570,6 +597,22 @@ def add(
|
||||
assert loaded is not None, "unreachable: config is not None"
|
||||
yaml_text, config_blob = loaded
|
||||
|
||||
# Validate v3 config via ActorConfigSchema if detected.
|
||||
# This ensures v3 actors are fully validated (cycle detection, required
|
||||
# fields, enum values) before the registry stores them.
|
||||
# NOTE: Both the CLI (here) and the registry validate v3 configs
|
||||
# independently. The CLI validation provides fast user-facing error
|
||||
# messages; the registry validation is a defence-in-depth guard for
|
||||
# programmatic callers that bypass the CLI layer.
|
||||
if is_v3_yaml(config_blob):
|
||||
_validate_v3_config(config, config_blob)
|
||||
|
||||
# NOTE: v3 TOOL actors without 'provider' will be rejected here by the legacy
|
||||
# v2 canonicalization layer (ActorConfiguration.from_blob requires provider).
|
||||
# This is a known limitation tracked in follow-up issue:
|
||||
# https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9971
|
||||
# Schema-level validation (above) already accepts provider-less TOOL actors
|
||||
# correctly; the gap is only in the legacy from_blob() canonicalization.
|
||||
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
|
||||
name=name,
|
||||
config_blob=config_blob,
|
||||
@@ -607,9 +650,8 @@ def add(
|
||||
pass # Actor does not exist yet — proceed with add
|
||||
|
||||
# Extract schema_version from the parsed config blob if present.
|
||||
schema_version: str | None = config_blob.get("schema_version") # type: ignore[assignment]
|
||||
if schema_version and not isinstance(schema_version, str):
|
||||
schema_version = str(schema_version)
|
||||
raw_version = config_blob.get("schema_version")
|
||||
schema_version: str | None = str(raw_version) if raw_version is not None else None
|
||||
|
||||
try:
|
||||
if registry:
|
||||
@@ -713,6 +755,14 @@ def update(
|
||||
if not option_overrides:
|
||||
option_overrides = None
|
||||
|
||||
# Validate v3 YAML via ActorConfigSchema if a new config file was given.
|
||||
# The `config is not None` guard is intentional: when no new --config file
|
||||
# is provided, the existing registry blob is reused as-is (it was already
|
||||
# validated at registration time). Only newly supplied config files need
|
||||
# re-validation to catch any schema violations in the uploaded YAML.
|
||||
if config is not None and is_v3_yaml(new_config):
|
||||
_validate_v3_config(config, new_config)
|
||||
|
||||
resolved, canonical_blob, requires_confirmation = _canonicalize_actor_config(
|
||||
name=name,
|
||||
config_blob=new_config,
|
||||
|
||||
Reference in New Issue
Block a user