Files
cleveractors-core/features/steps/validate_dict_helpers.py
hurui200320 149feae15f
CI / quality (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 46s
CI / integration_tests (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 1m2s
CI / typecheck (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 1m5s
CI / unit_tests (pull_request) Successful in 3m39s
CI / coverage (pull_request) Failing after 3m35s
CI / status-check (pull_request) Failing after 3s
CI / lint (push) Successful in 40s
CI / typecheck (push) Successful in 1m4s
CI / security (push) Successful in 1m4s
CI / quality (push) Successful in 32s
CI / integration_tests (push) Successful in 51s
CI / build (push) Successful in 45s
CI / unit_tests (push) Successful in 3m38s
CI / coverage (push) Failing after 3m34s
CI / status-check (push) Failing after 3s
feat(validate_dict): expose public validate_dict(config_dict, platform_limits) API
Implement validate_dict() in cleveractors/validation.py and export it from
cleveractors/__init__.py as the first router-facing API (ADR-2024, Wave 1).

validate_dict(config_dict, platform_limits) -> dict[str, Any]:

- Validates that both 'agents' and 'routes' top-level keys are present.
- Validates each agent's type against the spec-defined vocabulary:
  {llm, tool, composite, chain, template_instance}.
- Validates LLM agent 'provider' values against platform_limits['allowed_providers']
  when present, or the default allowlist {openai, anthropic, google} when absent.
- Validates route/edge field names: 'from'/'to' are rejected in favour of
  'source'/'target' as required by the spec-conformant format (ADR-2025).
- Validates graph node types against the spec vocabulary:
  {start, end, agent, function, tool, conditional, subgraph, message_router}.
- Validates stream operator types against the spec vocabulary (all defined types).
- Enforces structural platform limits from platform_limits when present:
  max_graph_depth (DFS longest-path check), max_subgraph_depth (recursive
  subgraph-reference depth), max_total_nodes (total nodes across all graph routes).
- Returns config_dict unchanged when all checks pass (identity contract).
- Pure static validator: no file I/O, no env-var reads, no app construction.

Review fixes (Cycle 1):
- M3: Added isinstance(node_name, str) guard for graph node keys in
  _validate_graph_route, with BDD scenario for non-string node key.
- m1: Added # pragma: no branch to _count_total_nodes defensive guard in
  _limits.py (matching the identical guards in _subgraph_children).
- m2: Added # pragma: no branch to dead defensive continue branches in
  _validate_graph_route edge validation loop.
- m3: Added dedicated BDD scenario and step for dangling source node
  (src not in nodes_raw branch), plus renamed existing scenario from
  "source node" to "target node" for accuracy.
- m4: Deleted 4 lines of commented-out InlineYAMLJinja setup code in
  features/environment.py.
- n1: Added depth <= 0 ValueError guard in build_subgraph_chain helper.
- n2: Renamed scenario "Edge referencing a non-existent source node" to
  "Edge referencing a non-existent target node" (the step tested a target).
- n3: Changed > to >= in both _MAX_DFS_STEPS and _MAX_SUBGRAPH_STEPS guards
  to prevent off-by-one allowing one extra iteration beyond documented ceiling.
- n4: Added adjacency target deduplication in _compute_graph_depth to avoid
  wasting DFS step budget on parallel edges.
- Spec review: M1 (entry_point optional) and M2 (start/end implicit injection)
  were verified against the authoritative spec at docs/specification.md in the
  cleveragents-webapp repo (develop branch). The spec does NOT support either
  claim; entry_point is required and start/end must be explicitly declared.
  No changes needed for M1/M2.

Review fixes (Cycle 2):
- M1: Deduplicated _subgraph_children yields with a seen set to prevent
  duplicate subgraph references from consuming DFS steps redundantly and
  causing false "too complex" errors.
- M2: Changed adjacency value type from list[str] to set[str] in
  _compute_graph_depth for O(1) deduplication, reducing worst case
  from O(N^3) to O(N^2) for dense graphs.
- m1: Added BDD scenario for LLM agent without provider field failing
  when default "openai" is not in custom allowlist.
- m2: Added positive BDD scenarios for valid node types start, tool,
  conditional, message_router, function.
- m3: Added positive BDD scenarios for additional stream operator types
  filter, transform, reduce.
- m5: Added depth_cache memoization to _compute_subgraph_depth to avoid
  recomputing depths across graph routes sharing subgraph trees.
- m6: Moved cleveractors.validation._limits import from module level into
  after_scenario, guarded by hasattr.
- n1: Split validate_dict_routes_steps.py (was 473 lines) into
  validate_dict_routes_steps.py (generic route steps) and
  validate_dict_graph_route_steps.py (graph-specific steps).
- n2: Updated _MAX_DFS_STEPS module comment to accurately state O(N+E)
  complexity and that dense graphs can exhaust the ceiling quickly.

BDD Behave scenarios cover all requirements.
Robot Framework integration tests (robot/validate_dict.robot) cover API usage.

ISSUES CLOSED: #10
2026-06-05 09:23:44 +00:00

166 lines
5.5 KiB
Python

"""
Shared helper functions for validate_dict BDD step definitions.
This module provides reusable config builders, the ``call_and_capture``
invocation helper, and the ``minimal_config`` factory. It is imported
by the domain-specific step modules and is **not** itself a Behave step
file (it contains no @given/@when/@then decorators).
"""
from __future__ import annotations
import copy
from typing import Any
from behave.runner import Context
from cleveractors import validate_dict
from cleveractors.core.exceptions import ConfigurationError
# ── Minimal test configs ───────────────────────────────────────────────────
MINIMAL_CONFIG: dict[str, Any] = {
"agents": {
"echo": {
"type": "tool",
"config": {"tools": ["echo"]},
}
},
"routes": {
"main": {
"type": "stream",
"operators": [{"type": "map", "params": {"agent": "echo"}}],
"publications": ["__output__"],
}
},
}
def minimal_config() -> dict[str, Any]:
"""Return a deep copy of the minimal valid config."""
return copy.deepcopy(MINIMAL_CONFIG)
# ── Shared helper: call validate_dict and capture result/exception ──────
def call_and_capture(
context: Context,
config: Any,
limits: Any,
) -> None:
"""Call validate_dict and store the result or exception on context.
Accepts ``Any`` for *config* and *limits* so that test steps can
intentionally pass invalid types (e.g. ``None``) to exercise the
fail-fast argument guards in ``validate_dict``.
"""
context.validate_dict = validate_dict
context.ConfigurationError = ConfigurationError
try:
context.result = validate_dict(config, limits)
context.raised_exception = None
except Exception as exc:
context.raised_exception = exc
context.result = None
# ── Shared graph builders ────────────────────────────────────────────────
def build_linear_graph(depth: int) -> dict[str, Any]:
"""Build a config with a linear graph route of exactly 'depth' edges."""
config = minimal_config()
if depth <= 0:
config["routes"]["depth_graph"] = {
"type": "graph",
"entry_point": "end_node",
"nodes": {"end_node": {"type": "end"}},
"edges": [],
}
return config
nodes: dict[str, Any] = {}
edges: list[dict[str, Any]] = []
node_names = [f"node_{i}" for i in range(depth)]
for name in node_names:
nodes[name] = {"type": "agent", "agent": "echo"}
nodes["end"] = {"type": "end"}
for i in range(len(node_names) - 1):
edges.append({"source": node_names[i], "target": node_names[i + 1]})
if node_names:
edges.append({"source": node_names[-1], "target": "end"})
config["routes"]["depth_graph"] = {
"type": "graph",
"entry_point": node_names[0],
"nodes": nodes,
"edges": edges,
}
return config
def build_node_count_graph(count: int) -> dict[str, Any]:
"""Build a graph with exactly 'count' non-end nodes plus the end node."""
config = minimal_config()
nodes: dict[str, Any] = {}
edges: list[dict[str, Any]] = []
node_names = [f"node_{i}" for i in range(count)]
for name in node_names:
nodes[name] = {"type": "agent", "agent": "echo"}
nodes["end"] = {"type": "end"}
for i in range(len(node_names) - 1):
edges.append({"source": node_names[i], "target": node_names[i + 1]})
if node_names:
edges.append({"source": node_names[-1], "target": "end"})
config["routes"]["count_graph"] = {
"type": "graph",
"entry_point": node_names[0] if node_names else "end",
"nodes": nodes,
"edges": edges,
}
return config
def build_subgraph_chain(depth: int) -> dict[str, Any]:
"""Build a config with nested subgraph routes of 'depth' levels."""
if depth <= 0:
raise ValueError("depth must be >= 1")
config = minimal_config()
routes: dict[str, Any] = {}
for level in range(depth):
route_name = f"sub_level_{level}"
nodes: dict[str, Any] = {
"worker": {"type": "agent", "agent": "echo"},
"end": {"type": "end"},
}
edges_list: list[dict[str, Any]] = [{"source": "worker", "target": "end"}]
if level < depth - 1:
next_route = f"sub_level_{level + 1}"
nodes["sub_node"] = {"type": "subgraph", "subgraph": next_route}
edges_list = [
{"source": "worker", "target": "sub_node"},
{"source": "sub_node", "target": "end"},
]
routes[route_name] = {
"type": "graph",
"entry_point": "worker",
"nodes": nodes,
"edges": edges_list,
}
top_nodes: dict[str, Any] = {
"top_worker": {"type": "agent", "agent": "echo"},
"sub_entry": {"type": "subgraph", "subgraph": "sub_level_0"},
"end": {"type": "end"},
}
top_edges: list[dict[str, Any]] = [
{"source": "top_worker", "target": "sub_entry"},
{"source": "sub_entry", "target": "end"},
]
routes["top_graph"] = {
"type": "graph",
"entry_point": "top_worker",
"nodes": top_nodes,
"edges": top_edges,
}
config["routes"].update(routes)
return config