Files
hurui200320 149feae15f
CI / quality (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 46s
CI / integration_tests (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 1m2s
CI / typecheck (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 1m5s
CI / unit_tests (pull_request) Successful in 3m39s
CI / coverage (pull_request) Failing after 3m35s
CI / status-check (pull_request) Failing after 3s
CI / lint (push) Successful in 40s
CI / typecheck (push) Successful in 1m4s
CI / security (push) Successful in 1m4s
CI / quality (push) Successful in 32s
CI / integration_tests (push) Successful in 51s
CI / build (push) Successful in 45s
CI / unit_tests (push) Successful in 3m38s
CI / coverage (push) Failing after 3m34s
CI / status-check (push) Failing after 3s
feat(validate_dict): expose public validate_dict(config_dict, platform_limits) API
Implement validate_dict() in cleveractors/validation.py and export it from
cleveractors/__init__.py as the first router-facing API (ADR-2024, Wave 1).

validate_dict(config_dict, platform_limits) -> dict[str, Any]:

- Validates that both 'agents' and 'routes' top-level keys are present.
- Validates each agent's type against the spec-defined vocabulary:
  {llm, tool, composite, chain, template_instance}.
- Validates LLM agent 'provider' values against platform_limits['allowed_providers']
  when present, or the default allowlist {openai, anthropic, google} when absent.
- Validates route/edge field names: 'from'/'to' are rejected in favour of
  'source'/'target' as required by the spec-conformant format (ADR-2025).
- Validates graph node types against the spec vocabulary:
  {start, end, agent, function, tool, conditional, subgraph, message_router}.
- Validates stream operator types against the spec vocabulary (all defined types).
- Enforces structural platform limits from platform_limits when present:
  max_graph_depth (DFS longest-path check), max_subgraph_depth (recursive
  subgraph-reference depth), max_total_nodes (total nodes across all graph routes).
- Returns config_dict unchanged when all checks pass (identity contract).
- Pure static validator: no file I/O, no env-var reads, no app construction.

Review fixes (Cycle 1):
- M3: Added isinstance(node_name, str) guard for graph node keys in
  _validate_graph_route, with BDD scenario for non-string node key.
- m1: Added # pragma: no branch to _count_total_nodes defensive guard in
  _limits.py (matching the identical guards in _subgraph_children).
- m2: Added # pragma: no branch to dead defensive continue branches in
  _validate_graph_route edge validation loop.
- m3: Added dedicated BDD scenario and step for dangling source node
  (src not in nodes_raw branch), plus renamed existing scenario from
  "source node" to "target node" for accuracy.
- m4: Deleted 4 lines of commented-out InlineYAMLJinja setup code in
  features/environment.py.
- n1: Added depth <= 0 ValueError guard in build_subgraph_chain helper.
- n2: Renamed scenario "Edge referencing a non-existent source node" to
  "Edge referencing a non-existent target node" (the step tested a target).
- n3: Changed > to >= in both _MAX_DFS_STEPS and _MAX_SUBGRAPH_STEPS guards
  to prevent off-by-one allowing one extra iteration beyond documented ceiling.
- n4: Added adjacency target deduplication in _compute_graph_depth to avoid
  wasting DFS step budget on parallel edges.
- Spec review: M1 (entry_point optional) and M2 (start/end implicit injection)
  were verified against the authoritative spec at docs/specification.md in the
  cleveragents-webapp repo (develop branch). The spec does NOT support either
  claim; entry_point is required and start/end must be explicitly declared.
  No changes needed for M1/M2.

Review fixes (Cycle 2):
- M1: Deduplicated _subgraph_children yields with a seen set to prevent
  duplicate subgraph references from consuming DFS steps redundantly and
  causing false "too complex" errors.
- M2: Changed adjacency value type from list[str] to set[str] in
  _compute_graph_depth for O(1) deduplication, reducing worst case
  from O(N^3) to O(N^2) for dense graphs.
- m1: Added BDD scenario for LLM agent without provider field failing
  when default "openai" is not in custom allowlist.
- m2: Added positive BDD scenarios for valid node types start, tool,
  conditional, message_router, function.
- m3: Added positive BDD scenarios for additional stream operator types
  filter, transform, reduce.
- m5: Added depth_cache memoization to _compute_subgraph_depth to avoid
  recomputing depths across graph routes sharing subgraph trees.
- m6: Moved cleveractors.validation._limits import from module level into
  after_scenario, guarded by hasattr.
- n1: Split validate_dict_routes_steps.py (was 473 lines) into
  validate_dict_routes_steps.py (generic route steps) and
  validate_dict_graph_route_steps.py (graph-specific steps).
- n2: Updated _MAX_DFS_STEPS module comment to accurately state O(N+E)
  complexity and that dense graphs can exhaust the ceiling quickly.

BDD Behave scenarios cover all requirements.
Robot Framework integration tests (robot/validate_dict.robot) cover API usage.

ISSUES CLOSED: #10
2026-06-05 09:23:44 +00:00

386 lines
14 KiB
Python

"""
Shared Then-step definitions for validate_dict BDD feature.
These assertion steps are used across all scenarios in validate_dict.feature.
"""
from __future__ import annotations
from behave import then
from behave.runner import Context
@then("no exception is raised")
def step_no_exception(context: Context) -> None:
if context.raised_exception is not None:
raise AssertionError(
f"Expected no exception, but got: {context.raised_exception!r}"
)
@then("a ConfigurationError is raised")
def step_configuration_error_raised(context: Context) -> None:
assert context.raised_exception is not None, (
"Expected a ConfigurationError to be raised, but none was."
)
assert isinstance(context.raised_exception, context.ConfigurationError), (
f"Expected ConfigurationError, got {type(context.raised_exception).__name__}: "
f"{context.raised_exception!r}"
)
@then('the error message mentions "agents"')
def step_error_mentions_agents(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "agents" in msg, (
f"Expected error message to mention 'agents', got: {context.raised_exception!r}"
)
@then('the error message mentions "routes"')
def step_error_mentions_routes(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "routes" in msg, (
f"Expected error message to mention 'routes', got: {context.raised_exception!r}"
)
@then("the error message mentions agent type validation")
def step_error_mentions_agent_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "unknown agent type" in msg, (
f"Expected error message to mention unknown agent type, got: {context.raised_exception!r}"
)
@then("the error message mentions provider validation")
def step_error_mentions_provider(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "provider" in msg and "allowed-providers" in msg, (
f"Expected error message to mention provider and allowed-providers, got: {context.raised_exception!r}"
)
@then("the error message mentions edge field names")
def step_error_mentions_edge_fields(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert (
"'from' is not accepted" in msg
or "'to' is not accepted" in msg
or ("source" in msg and "target" in msg)
), (
f"Expected error message to mention edge field names, got: {context.raised_exception!r}"
)
@then("the error message mentions node type validation")
def step_error_mentions_node_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "unknown node type" in msg, (
f"Expected error message to mention unknown node type, got: {context.raised_exception!r}"
)
@then("the error message mentions operator type validation")
def step_error_mentions_operator(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "unknown operator type" in msg, (
f"Expected error message to mention unknown operator type, got: {context.raised_exception!r}"
)
@then("the error message mentions graph depth")
def step_error_mentions_depth(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "max_graph_depth" in msg, (
f"Expected error message to mention max_graph_depth, got: {context.raised_exception!r}"
)
@then("the error message mentions total nodes")
def step_error_mentions_total_nodes(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "max_total_nodes" in msg, (
f"Expected error message to mention max_total_nodes, got: {context.raised_exception!r}"
)
@then("the error message mentions subgraph depth")
def step_error_mentions_subgraph_depth(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "max_subgraph_depth" in msg, (
f"Expected error message to mention max_subgraph_depth, got: {context.raised_exception!r}"
)
@then("the function returns the config dict unchanged")
def step_returns_dict_unchanged(context: Context) -> None:
assert context.result == context.config_dict, (
f"Expected the returned dict to equal the input dict.\n"
f"Input: {context.config_dict!r}\n"
f"Output: {context.result!r}"
)
@then("the returned dict is the same object as the input dict")
def step_returns_same_object(context: Context) -> None:
assert context.result is context.config_dict, (
"Expected the returned dict to be the same object (identity) as the input dict, "
"but validate_dict returned a different object."
)
@then("the error message mentions source and target fields")
def step_error_mentions_source_target(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "source" in msg and "target" in msg, (
f"Expected error message to mention source and target fields, got: {context.raised_exception!r}"
)
@then('the error message mentions "nodes"')
def step_error_mentions_nodes(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "nodes" in msg, (
f"Expected error message to mention 'nodes', got: {context.raised_exception!r}"
)
@then('the error message mentions "edges"')
def step_error_mentions_edges(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "edges" in msg, (
f"Expected error message to mention 'edges', got: {context.raised_exception!r}"
)
@then("the error message mentions missing type")
def step_error_mentions_missing_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "missing" in msg and "type" in msg, (
f"Expected error message to mention missing type, got: {context.raised_exception!r}"
)
@then("the error message mentions platform_limits type validation")
def step_error_mentions_limits_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "platform_limits" in msg, (
f"Expected error message to mention platform_limits, got: {context.raised_exception!r}"
)
@then("the error message mentions allowed_providers type validation")
def step_error_mentions_allowed_providers(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "allowed_providers" in msg, (
f"Expected error message to mention allowed_providers, got: {context.raised_exception!r}"
)
@then("the error message mentions non-string provider")
def step_error_mentions_non_string_provider(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "provider" in msg and "string" in msg, (
f"Expected error message to mention non-string provider, got: {context.raised_exception!r}"
)
@then("the error message mentions non-string node type")
def step_error_mentions_non_string_node_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "node type" in msg and "string" in msg, (
f"Expected error message to mention non-string node type, got: {context.raised_exception!r}"
)
@then("the error message mentions entry_point not found")
def step_error_mentions_entry_point(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "entry_point" in msg and "does not match any declared node" in msg, (
f"Expected error message to mention entry_point not found, got: {context.raised_exception!r}"
)
@then("the error message mentions subgraph reference")
def step_error_mentions_subgraph_reference(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "subgraph" in msg, (
f"Expected error message to mention subgraph, got: {context.raised_exception!r}"
)
@then('the error message mentions "type"')
def step_error_mentions_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "type" in msg, (
f"Expected error message to mention 'type', got: {context.raised_exception!r}"
)
@then("the error message mentions non-string agent type")
def step_error_mentions_non_string_agent_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "type" in msg and "string" in msg and "agent" in msg, (
f"Expected error message to mention non-string agent type, got: {context.raised_exception!r}"
)
@then("the error message mentions non-string route type")
def step_error_mentions_non_string_route_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "type" in msg and "string" in msg and "route" in msg, (
f"Expected error message to mention non-string route type, got: {context.raised_exception!r}"
)
@then('the error message mentions "entry_point"')
def step_error_mentions_entry_point_keyword(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "entry_point" in msg, (
f"Expected error message to mention 'entry_point', got: {context.raised_exception!r}"
)
@then("the error message mentions non-string operator type")
def step_error_mentions_non_string_op_type(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "operator type" in msg and "string" in msg, (
f"Expected error message to mention non-string operator type, got: {context.raised_exception!r}"
)
@then("the error message mentions config_dict and dict")
def step_error_mentions_config_dict_is_dict(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "config_dict" in msg and "dict" in msg, (
f"Expected error message to mention config_dict and dict, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions platform_limits and dict")
def step_error_mentions_platform_limits_is_dict(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "platform_limits" in msg and "dict" in msg, (
f"Expected error message to mention platform_limits and dict, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions edge must be a mapping")
def step_error_mentions_edge_mapping(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "edge" in msg and "mapping" in msg, (
f"Expected error message to mention edge mapping, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions operator must be a mapping")
def step_error_mentions_operator_mapping(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "operator" in msg and "mapping" in msg, (
f"Expected error message to mention operator mapping, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions too complex to validate depth")
def step_error_mentions_too_complex_depth(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "too complex to validate" in msg and "depth" in msg, (
f"Expected error message to mention 'too complex to validate depth', "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions too complex to validate")
def step_error_mentions_too_complex(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "too complex to validate" in msg, (
f"Expected error message to mention 'too complex to validate', "
f"got: {context.raised_exception!r}"
)
@then('the error message mentions "must be >= 0"')
def step_error_mentions_must_be_ge_zero(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "must be >= 0" in msg, (
f"Expected error message to mention 'must be >= 0', "
f"got: {context.raised_exception!r}"
)
@then('the error message mentions "must be an integer"')
def step_error_mentions_must_be_integer(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "must be an integer" in msg, (
f"Expected error message to mention 'must be an integer', "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions edge endpoint not declared")
def step_error_mentions_edge_endpoint_not_declared(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "not declared in 'nodes'" in msg, (
f"Expected error message to mention edge endpoint not declared, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions config must be a mapping")
def step_error_mentions_config_mapping(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "config" in msg and "mapping" in msg, (
f"Expected error message to mention config mapping, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions reserved agent name")
def step_error_mentions_reserved_agent(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "reserved" in msg and "agent" in msg, (
f"Expected error message to mention reserved agent name, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions reserved route name")
def step_error_mentions_reserved_route(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "reserved" in msg and "route" in msg, (
f"Expected error message to mention reserved route name, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions agent key must be a string")
def step_error_mentions_agent_key_string(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "agent key" in msg and "string" in msg, (
f"Expected error message to mention agent key must be a string, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions route key must be a string")
def step_error_mentions_route_key_string(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "route key" in msg and "string" in msg, (
f"Expected error message to mention route key must be a string, "
f"got: {context.raised_exception!r}"
)
@then("the error message mentions node key must be a string")
def step_error_mentions_node_key_string(context: Context) -> None:
msg = str(context.raised_exception).lower()
assert "node key" in msg and "string" in msg, (
f"Expected error message to mention node key must be a string, "
f"got: {context.raised_exception!r}"
)