149feae15f
CI / quality (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 46s
CI / integration_tests (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 1m2s
CI / typecheck (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 1m5s
CI / unit_tests (pull_request) Successful in 3m39s
CI / coverage (pull_request) Failing after 3m35s
CI / status-check (pull_request) Failing after 3s
CI / lint (push) Successful in 40s
CI / typecheck (push) Successful in 1m4s
CI / security (push) Successful in 1m4s
CI / quality (push) Successful in 32s
CI / integration_tests (push) Successful in 51s
CI / build (push) Successful in 45s
CI / unit_tests (push) Successful in 3m38s
CI / coverage (push) Failing after 3m34s
CI / status-check (push) Failing after 3s
Implement validate_dict() in cleveractors/validation.py and export it from
cleveractors/__init__.py as the first router-facing API (ADR-2024, Wave 1).
validate_dict(config_dict, platform_limits) -> dict[str, Any]:
- Validates that both 'agents' and 'routes' top-level keys are present.
- Validates each agent's type against the spec-defined vocabulary:
{llm, tool, composite, chain, template_instance}.
- Validates LLM agent 'provider' values against platform_limits['allowed_providers']
when present, or the default allowlist {openai, anthropic, google} when absent.
- Validates route/edge field names: 'from'/'to' are rejected in favour of
'source'/'target' as required by the spec-conformant format (ADR-2025).
- Validates graph node types against the spec vocabulary:
{start, end, agent, function, tool, conditional, subgraph, message_router}.
- Validates stream operator types against the spec vocabulary (all defined types).
- Enforces structural platform limits from platform_limits when present:
max_graph_depth (DFS longest-path check), max_subgraph_depth (recursive
subgraph-reference depth), max_total_nodes (total nodes across all graph routes).
- Returns config_dict unchanged when all checks pass (identity contract).
- Pure static validator: no file I/O, no env-var reads, no app construction.
Review fixes (Cycle 1):
- M3: Added isinstance(node_name, str) guard for graph node keys in
_validate_graph_route, with BDD scenario for non-string node key.
- m1: Added # pragma: no branch to _count_total_nodes defensive guard in
_limits.py (matching the identical guards in _subgraph_children).
- m2: Added # pragma: no branch to dead defensive continue branches in
_validate_graph_route edge validation loop.
- m3: Added dedicated BDD scenario and step for dangling source node
(src not in nodes_raw branch), plus renamed existing scenario from
"source node" to "target node" for accuracy.
- m4: Deleted 4 lines of commented-out InlineYAMLJinja setup code in
features/environment.py.
- n1: Added depth <= 0 ValueError guard in build_subgraph_chain helper.
- n2: Renamed scenario "Edge referencing a non-existent source node" to
"Edge referencing a non-existent target node" (the step tested a target).
- n3: Changed > to >= in both _MAX_DFS_STEPS and _MAX_SUBGRAPH_STEPS guards
to prevent off-by-one allowing one extra iteration beyond documented ceiling.
- n4: Added adjacency target deduplication in _compute_graph_depth to avoid
wasting DFS step budget on parallel edges.
- Spec review: M1 (entry_point optional) and M2 (start/end implicit injection)
were verified against the authoritative spec at docs/specification.md in the
cleveragents-webapp repo (develop branch). The spec does NOT support either
claim; entry_point is required and start/end must be explicitly declared.
No changes needed for M1/M2.
Review fixes (Cycle 2):
- M1: Deduplicated _subgraph_children yields with a seen set to prevent
duplicate subgraph references from consuming DFS steps redundantly and
causing false "too complex" errors.
- M2: Changed adjacency value type from list[str] to set[str] in
_compute_graph_depth for O(1) deduplication, reducing worst case
from O(N^3) to O(N^2) for dense graphs.
- m1: Added BDD scenario for LLM agent without provider field failing
when default "openai" is not in custom allowlist.
- m2: Added positive BDD scenarios for valid node types start, tool,
conditional, message_router, function.
- m3: Added positive BDD scenarios for additional stream operator types
filter, transform, reduce.
- m5: Added depth_cache memoization to _compute_subgraph_depth to avoid
recomputing depths across graph routes sharing subgraph trees.
- m6: Moved cleveractors.validation._limits import from module level into
after_scenario, guarded by hasattr.
- n1: Split validate_dict_routes_steps.py (was 473 lines) into
validate_dict_routes_steps.py (generic route steps) and
validate_dict_graph_route_steps.py (graph-specific steps).
- n2: Updated _MAX_DFS_STEPS module comment to accurately state O(N+E)
complexity and that dense graphs can exhaust the ceiling quickly.
BDD Behave scenarios cover all requirements.
Robot Framework integration tests (robot/validate_dict.robot) cover API usage.
ISSUES CLOSED: #10
166 lines
5.5 KiB
Python
166 lines
5.5 KiB
Python
"""
|
|
Shared helper functions for validate_dict BDD step definitions.
|
|
|
|
This module provides reusable config builders, the ``call_and_capture``
|
|
invocation helper, and the ``minimal_config`` factory. It is imported
|
|
by the domain-specific step modules and is **not** itself a Behave step
|
|
file (it contains no @given/@when/@then decorators).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import copy
|
|
from typing import Any
|
|
|
|
from behave.runner import Context
|
|
|
|
from cleveractors import validate_dict
|
|
from cleveractors.core.exceptions import ConfigurationError
|
|
|
|
# ── Minimal test configs ───────────────────────────────────────────────────
|
|
|
|
MINIMAL_CONFIG: dict[str, Any] = {
|
|
"agents": {
|
|
"echo": {
|
|
"type": "tool",
|
|
"config": {"tools": ["echo"]},
|
|
}
|
|
},
|
|
"routes": {
|
|
"main": {
|
|
"type": "stream",
|
|
"operators": [{"type": "map", "params": {"agent": "echo"}}],
|
|
"publications": ["__output__"],
|
|
}
|
|
},
|
|
}
|
|
|
|
|
|
def minimal_config() -> dict[str, Any]:
|
|
"""Return a deep copy of the minimal valid config."""
|
|
return copy.deepcopy(MINIMAL_CONFIG)
|
|
|
|
|
|
# ── Shared helper: call validate_dict and capture result/exception ──────
|
|
|
|
|
|
def call_and_capture(
|
|
context: Context,
|
|
config: Any,
|
|
limits: Any,
|
|
) -> None:
|
|
"""Call validate_dict and store the result or exception on context.
|
|
|
|
Accepts ``Any`` for *config* and *limits* so that test steps can
|
|
intentionally pass invalid types (e.g. ``None``) to exercise the
|
|
fail-fast argument guards in ``validate_dict``.
|
|
"""
|
|
context.validate_dict = validate_dict
|
|
context.ConfigurationError = ConfigurationError
|
|
try:
|
|
context.result = validate_dict(config, limits)
|
|
context.raised_exception = None
|
|
except Exception as exc:
|
|
context.raised_exception = exc
|
|
context.result = None
|
|
|
|
|
|
# ── Shared graph builders ────────────────────────────────────────────────
|
|
|
|
|
|
def build_linear_graph(depth: int) -> dict[str, Any]:
|
|
"""Build a config with a linear graph route of exactly 'depth' edges."""
|
|
config = minimal_config()
|
|
if depth <= 0:
|
|
config["routes"]["depth_graph"] = {
|
|
"type": "graph",
|
|
"entry_point": "end_node",
|
|
"nodes": {"end_node": {"type": "end"}},
|
|
"edges": [],
|
|
}
|
|
return config
|
|
nodes: dict[str, Any] = {}
|
|
edges: list[dict[str, Any]] = []
|
|
node_names = [f"node_{i}" for i in range(depth)]
|
|
for name in node_names:
|
|
nodes[name] = {"type": "agent", "agent": "echo"}
|
|
nodes["end"] = {"type": "end"}
|
|
for i in range(len(node_names) - 1):
|
|
edges.append({"source": node_names[i], "target": node_names[i + 1]})
|
|
if node_names:
|
|
edges.append({"source": node_names[-1], "target": "end"})
|
|
config["routes"]["depth_graph"] = {
|
|
"type": "graph",
|
|
"entry_point": node_names[0],
|
|
"nodes": nodes,
|
|
"edges": edges,
|
|
}
|
|
return config
|
|
|
|
|
|
def build_node_count_graph(count: int) -> dict[str, Any]:
|
|
"""Build a graph with exactly 'count' non-end nodes plus the end node."""
|
|
config = minimal_config()
|
|
nodes: dict[str, Any] = {}
|
|
edges: list[dict[str, Any]] = []
|
|
node_names = [f"node_{i}" for i in range(count)]
|
|
for name in node_names:
|
|
nodes[name] = {"type": "agent", "agent": "echo"}
|
|
nodes["end"] = {"type": "end"}
|
|
for i in range(len(node_names) - 1):
|
|
edges.append({"source": node_names[i], "target": node_names[i + 1]})
|
|
if node_names:
|
|
edges.append({"source": node_names[-1], "target": "end"})
|
|
config["routes"]["count_graph"] = {
|
|
"type": "graph",
|
|
"entry_point": node_names[0] if node_names else "end",
|
|
"nodes": nodes,
|
|
"edges": edges,
|
|
}
|
|
return config
|
|
|
|
|
|
def build_subgraph_chain(depth: int) -> dict[str, Any]:
|
|
"""Build a config with nested subgraph routes of 'depth' levels."""
|
|
if depth <= 0:
|
|
raise ValueError("depth must be >= 1")
|
|
config = minimal_config()
|
|
routes: dict[str, Any] = {}
|
|
for level in range(depth):
|
|
route_name = f"sub_level_{level}"
|
|
nodes: dict[str, Any] = {
|
|
"worker": {"type": "agent", "agent": "echo"},
|
|
"end": {"type": "end"},
|
|
}
|
|
edges_list: list[dict[str, Any]] = [{"source": "worker", "target": "end"}]
|
|
if level < depth - 1:
|
|
next_route = f"sub_level_{level + 1}"
|
|
nodes["sub_node"] = {"type": "subgraph", "subgraph": next_route}
|
|
edges_list = [
|
|
{"source": "worker", "target": "sub_node"},
|
|
{"source": "sub_node", "target": "end"},
|
|
]
|
|
routes[route_name] = {
|
|
"type": "graph",
|
|
"entry_point": "worker",
|
|
"nodes": nodes,
|
|
"edges": edges_list,
|
|
}
|
|
top_nodes: dict[str, Any] = {
|
|
"top_worker": {"type": "agent", "agent": "echo"},
|
|
"sub_entry": {"type": "subgraph", "subgraph": "sub_level_0"},
|
|
"end": {"type": "end"},
|
|
}
|
|
top_edges: list[dict[str, Any]] = [
|
|
{"source": "top_worker", "target": "sub_entry"},
|
|
{"source": "sub_entry", "target": "end"},
|
|
]
|
|
routes["top_graph"] = {
|
|
"type": "graph",
|
|
"entry_point": "top_worker",
|
|
"nodes": top_nodes,
|
|
"edges": top_edges,
|
|
}
|
|
config["routes"].update(routes)
|
|
return config
|