Files
hurui200320 149feae15f
CI / quality (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 46s
CI / integration_tests (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 1m2s
CI / typecheck (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 1m5s
CI / unit_tests (pull_request) Successful in 3m39s
CI / coverage (pull_request) Failing after 3m35s
CI / status-check (pull_request) Failing after 3s
CI / lint (push) Successful in 40s
CI / typecheck (push) Successful in 1m4s
CI / security (push) Successful in 1m4s
CI / quality (push) Successful in 32s
CI / integration_tests (push) Successful in 51s
CI / build (push) Successful in 45s
CI / unit_tests (push) Successful in 3m38s
CI / coverage (push) Failing after 3m34s
CI / status-check (push) Failing after 3s
feat(validate_dict): expose public validate_dict(config_dict, platform_limits) API
Implement validate_dict() in cleveractors/validation.py and export it from
cleveractors/__init__.py as the first router-facing API (ADR-2024, Wave 1).

validate_dict(config_dict, platform_limits) -> dict[str, Any]:

- Validates that both 'agents' and 'routes' top-level keys are present.
- Validates each agent's type against the spec-defined vocabulary:
  {llm, tool, composite, chain, template_instance}.
- Validates LLM agent 'provider' values against platform_limits['allowed_providers']
  when present, or the default allowlist {openai, anthropic, google} when absent.
- Validates route/edge field names: 'from'/'to' are rejected in favour of
  'source'/'target' as required by the spec-conformant format (ADR-2025).
- Validates graph node types against the spec vocabulary:
  {start, end, agent, function, tool, conditional, subgraph, message_router}.
- Validates stream operator types against the spec vocabulary (all defined types).
- Enforces structural platform limits from platform_limits when present:
  max_graph_depth (DFS longest-path check), max_subgraph_depth (recursive
  subgraph-reference depth), max_total_nodes (total nodes across all graph routes).
- Returns config_dict unchanged when all checks pass (identity contract).
- Pure static validator: no file I/O, no env-var reads, no app construction.

Review fixes (Cycle 1):
- M3: Added isinstance(node_name, str) guard for graph node keys in
  _validate_graph_route, with BDD scenario for non-string node key.
- m1: Added # pragma: no branch to _count_total_nodes defensive guard in
  _limits.py (matching the identical guards in _subgraph_children).
- m2: Added # pragma: no branch to dead defensive continue branches in
  _validate_graph_route edge validation loop.
- m3: Added dedicated BDD scenario and step for dangling source node
  (src not in nodes_raw branch), plus renamed existing scenario from
  "source node" to "target node" for accuracy.
- m4: Deleted 4 lines of commented-out InlineYAMLJinja setup code in
  features/environment.py.
- n1: Added depth <= 0 ValueError guard in build_subgraph_chain helper.
- n2: Renamed scenario "Edge referencing a non-existent source node" to
  "Edge referencing a non-existent target node" (the step tested a target).
- n3: Changed > to >= in both _MAX_DFS_STEPS and _MAX_SUBGRAPH_STEPS guards
  to prevent off-by-one allowing one extra iteration beyond documented ceiling.
- n4: Added adjacency target deduplication in _compute_graph_depth to avoid
  wasting DFS step budget on parallel edges.
- Spec review: M1 (entry_point optional) and M2 (start/end implicit injection)
  were verified against the authoritative spec at docs/specification.md in the
  cleveragents-webapp repo (develop branch). The spec does NOT support either
  claim; entry_point is required and start/end must be explicitly declared.
  No changes needed for M1/M2.

Review fixes (Cycle 2):
- M1: Deduplicated _subgraph_children yields with a seen set to prevent
  duplicate subgraph references from consuming DFS steps redundantly and
  causing false "too complex" errors.
- M2: Changed adjacency value type from list[str] to set[str] in
  _compute_graph_depth for O(1) deduplication, reducing worst case
  from O(N^3) to O(N^2) for dense graphs.
- m1: Added BDD scenario for LLM agent without provider field failing
  when default "openai" is not in custom allowlist.
- m2: Added positive BDD scenarios for valid node types start, tool,
  conditional, message_router, function.
- m3: Added positive BDD scenarios for additional stream operator types
  filter, transform, reduce.
- m5: Added depth_cache memoization to _compute_subgraph_depth to avoid
  recomputing depths across graph routes sharing subgraph trees.
- m6: Moved cleveractors.validation._limits import from module level into
  after_scenario, guarded by hasattr.
- n1: Split validate_dict_routes_steps.py (was 473 lines) into
  validate_dict_routes_steps.py (generic route steps) and
  validate_dict_graph_route_steps.py (graph-specific steps).
- n2: Updated _MAX_DFS_STEPS module comment to accurately state O(N+E)
  complexity and that dense graphs can exhaust the ceiling quickly.

BDD Behave scenarios cover all requirements.
Robot Framework integration tests (robot/validate_dict.robot) cover API usage.

ISSUES CLOSED: #10
2026-06-05 09:23:44 +00:00

878 lines
45 KiB
Gherkin

Feature: validate_dict public API — spec-conformant Actor Configuration validation
As a router platform integrating cleveractors-core
I want to call validate_dict(config_dict, platform_limits) to validate actor YAML dicts
So that only spec-conformant configs are accepted before they are stored in the database
Background:
Given the validate_dict function is imported from cleveractors
# ── Top-level key presence ─────────────────────────────────────────────────
Scenario: Valid minimal config returns the dict unchanged
Given a minimal valid config dict with agents and routes
When I call validate_dict with empty platform_limits
Then the function returns the config dict unchanged
And no exception is raised
Scenario: Config missing "agents" key raises ConfigurationError
Given a config dict without an "agents" key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "agents"
Scenario: Config missing "routes" key raises ConfigurationError
Given a config dict without a "routes" key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "routes"
# ── Agent type validation ──────────────────────────────────────────────────
Scenario: Config with a valid llm agent type passes validation
Given a config dict with an agent of type "llm"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Config with a valid tool agent type passes validation
Given a config dict with an agent of type "tool"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Config with a valid composite agent type passes validation
Given a config dict with an agent of type "composite"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Config with a valid chain agent type passes validation
Given a config dict with an agent of type "chain"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Config with a valid template_instance agent type passes validation
Given a config dict with an agent of type "template_instance"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Config with an unknown agent type raises ConfigurationError
Given a config dict with an agent of type "unknown_type"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions agent type validation
# ── LLM provider validation ────────────────────────────────────────────────
Scenario: LLM agent with openai provider passes default allowlist validation
Given a config dict with an llm agent using provider "openai"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: LLM agent with anthropic provider passes default allowlist validation
Given a config dict with an llm agent using provider "anthropic"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: LLM agent with google provider passes default allowlist validation
Given a config dict with an llm agent using provider "google"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: LLM agent with unknown provider and no allowlist raises ConfigurationError
Given a config dict with an llm agent using provider "unknown_provider"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions provider validation
Scenario: LLM agent provider matches custom allowed_providers list
Given a config dict with an llm agent using provider "groq"
When I call validate_dict with platform_limits containing allowed_providers "groq,openai"
Then no exception is raised
Scenario: LLM agent provider not in custom allowed_providers raises ConfigurationError
Given a config dict with an llm agent using provider "anthropic"
When I call validate_dict with platform_limits containing allowed_providers "openai"
Then a ConfigurationError is raised
And the error message mentions provider validation
Scenario: LLM agent without explicit provider uses default openai and passes
Given a config dict with an llm agent without a provider field
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Route / edge / node / operator validation ──────────────────────────────
Scenario: Graph route with source/target edge fields passes validation
Given a config dict with a graph route using source/target edges
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route edge using "from" field raises ConfigurationError
Given a config dict with a graph route edge using "from" instead of "source"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions edge field names
Scenario: Graph route edge using "to" field raises ConfigurationError
Given a config dict with a graph route edge using "to" instead of "target"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions edge field names
Scenario: Graph route with valid node type "agent" passes validation
Given a config dict with a graph route containing a node of type "agent"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with invalid node type raises ConfigurationError
Given a config dict with a graph route containing a node of type "invalid_node"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions node type validation
Scenario: Stream route with valid operator type "map" passes validation
Given a config dict with a stream route containing a "map" operator
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Stream route with invalid operator type raises ConfigurationError
Given a config dict with a stream route containing an invalid operator type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions operator type validation
# ── Structural limit enforcement ───────────────────────────────────────────
Scenario: Graph within max_graph_depth limit passes validation
Given a config dict with a graph route of 4 edges
When I call validate_dict with platform_limits max_graph_depth of 5
Then no exception is raised
Scenario: Graph exceeding max_graph_depth raises ConfigurationError
Given a config dict with a graph route of 4 edges
When I call validate_dict with platform_limits max_graph_depth of 3
Then a ConfigurationError is raised
And the error message mentions graph depth
Scenario: Actor within max_total_nodes limit passes validation
Given a config dict with a graph route containing 3 non-end nodes
When I call validate_dict with platform_limits max_total_nodes of 10
Then no exception is raised
Scenario: Actor exceeding max_total_nodes raises ConfigurationError
Given a config dict with a graph route containing 5 non-end nodes
When I call validate_dict with platform_limits max_total_nodes of 3
Then a ConfigurationError is raised
And the error message mentions total nodes
Scenario: Nested subgraph within max_subgraph_depth limit passes validation
Given a config dict with nested subgraphs of depth 2
When I call validate_dict with platform_limits max_subgraph_depth of 3
Then no exception is raised
Scenario: Nested subgraph exceeding max_subgraph_depth raises ConfigurationError
Given a config dict with nested subgraphs of depth 4
When I call validate_dict with platform_limits max_subgraph_depth of 3
Then a ConfigurationError is raised
And the error message mentions subgraph depth
# ── Return value contract ──────────────────────────────────────────────────
Scenario: Valid config is returned unchanged (identity contract)
Given a fully valid spec-conformant config dict
When I call validate_dict with empty platform_limits
Then the returned dict is the same object as the input dict
# ── Callable without file/env/app side effects ────────────────────────────
Scenario: Bridge route type passes validation
Given a config dict with a valid bridge route
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Case-insensitive validation ────────────────────────────────────────────
Scenario: LLM agent with provider "OpenAI" passes default allowlist
Given a config dict with an llm agent using provider "OpenAI" in mixed case
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with node type "Agent" passes validation
Given a config dict with a graph route containing a node of type "Agent" in mixed case
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Boundary-exact structural limit tests ──────────────────────────────────
Scenario: Graph at exact max_graph_depth limit passes validation
Given a config dict with a graph route of exactly 5 edges for boundary test
When I call validate_dict with platform_limits max_graph_depth of 5
Then no exception is raised
Scenario: Actor at exact max_total_nodes limit passes validation
Given a config dict with a graph route containing exactly 5 non-end nodes for boundary test
When I call validate_dict with platform_limits max_total_nodes of 6
Then no exception is raised
Scenario: Nested subgraph at exact max_subgraph_depth limit passes validation
Given a config dict with nested subgraphs of exactly 3 levels for boundary test
When I call validate_dict with platform_limits max_subgraph_depth of 3
Then no exception is raised
# ── Zero-value boundary tests ──────────────────────────────────────────────
Scenario: max_graph_depth=0 raises ConfigurationError when graph has depth 1
Given a config dict with a graph route of 1 edge
When I call validate_dict with platform_limits max_graph_depth of 0
Then a ConfigurationError is raised
And the error message mentions graph depth
Scenario: max_total_nodes=0 raises ConfigurationError when config has any nodes
Given a config dict with a graph route containing 1 non-end node
When I call validate_dict with platform_limits max_total_nodes of 0
Then a ConfigurationError is raised
And the error message mentions total nodes
Scenario: max_subgraph_depth=0 raises ConfigurationError when config has a subgraph
Given a config dict with nested subgraphs of depth 1
When I call validate_dict with platform_limits max_subgraph_depth of 0
Then a ConfigurationError is raised
And the error message mentions subgraph depth
# ── Empty collection boundary tests ────────────────────────────────────────
Scenario: Empty agents dict with valid routes passes validation
Given a config dict with empty agents and valid routes
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Empty routes dict with valid agents passes validation
Given a config dict with valid agents and empty routes
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Empty allowed_providers list raises ConfigurationError for LLM agent
Given a config dict with an llm agent using provider "openai"
When I call validate_dict with platform_limits containing allowed_providers ""
Then a ConfigurationError is raised
And the error message mentions provider validation
# ── Negative platform limits rejected ──────────────────────────────────────
Scenario: Negative max_graph_depth raises ConfigurationError
Given a config dict with a graph route of 1 edge
When I call validate_dict with platform_limits max_graph_depth of -1
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: Negative max_total_nodes raises ConfigurationError
Given a config dict with a graph route containing 1 non-end node
When I call validate_dict with platform_limits max_total_nodes of -1
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: Negative max_subgraph_depth raises ConfigurationError
Given a config dict with nested subgraphs of depth 1
When I call validate_dict with platform_limits max_subgraph_depth of -1
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
# ── Crash-bug regression tests ─────────────────────────────────────────────
Scenario: Non-string LLM provider raises ConfigurationError instead of crashing
Given a config dict with an llm agent using a non-string provider value
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions non-string provider
Scenario: Non-string node type raises ConfigurationError instead of crashing
Given a config dict with a graph route containing a non-string node type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions non-string node type
Scenario: Edge missing source and target fields raises ConfigurationError
Given a config dict with a graph route where an edge lacks source and target
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions source and target fields
Scenario: Graph route missing nodes key raises ConfigurationError
Given a config dict with a graph route missing the nodes key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "nodes"
Scenario: Graph route missing edges key raises ConfigurationError
Given a config dict with a graph route missing the edges key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "edges"
Scenario: Stream operator missing type field raises ConfigurationError
Given a config dict with a stream route containing an operator missing type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions missing type
Scenario: Bare string as allowed_providers raises ConfigurationError
Given a config dict with platform_limits allowed_providers as a bare string "openai"
When I call validate_dict with platform_limits containing the bare string
Then a ConfigurationError is raised
And the error message mentions allowed_providers type validation
Scenario: Non-int max_graph_depth raises ConfigurationError
Given a config dict with platform_limits containing a non-int "max_graph_depth" value
When I call validate_dict with platform_limits containing a non-int limit value
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: agents section that is not a dict raises ConfigurationError
Given a config dict where agents is a list instead of a mapping
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: routes section that is not a dict raises ConfigurationError
Given a config dict where routes is a list instead of a mapping
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Agent definition that is not a dict raises ConfigurationError
Given a config dict where an agent definition is a string instead of a mapping
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Agent with agent_template key but no type skips type validation
Given a config dict with an agent that has an agent_template key instead of type
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Agent with no type and no template key raises ConfigurationError
Given a config dict with an agent that has no type and no template key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Agent with template key but no type skips type validation
Given a config dict with an agent that has a template key instead of type
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: LLM agent with non-dict config raises ConfigurationError
Given a config dict with an llm agent that has a non-dict config
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions config must be a mapping
Scenario: Route definition that is not a dict raises ConfigurationError
Given a config dict where a route definition is a string instead of a mapping
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Route with unknown type raises ConfigurationError
Given a config dict with a route of unknown type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Graph route with non-list edges raises ConfigurationError
Given a config dict with a graph route where edges is not a list
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "edges"
Scenario: Graph route with non-dict edge entry raises ConfigurationError
Given a config dict with a graph route where one edge is a string
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions edge must be a mapping
Scenario: Graph route with non-dict node entry raises ConfigurationError
Given a config dict with a graph route where one node is a string
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Graph route with non-dict nodes section raises ConfigurationError
Given a config dict with a graph route where nodes is not a dict
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "nodes"
Scenario: Stream route with non-list operators raises ConfigurationError
Given a config dict with a stream route where operators is not a list
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Stream route with non-dict operator entry raises ConfigurationError
Given a config dict with a stream route where one operator is a string
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions operator must be a mapping
Scenario: Stream operator with empty type string raises ConfigurationError
Given a config dict with a stream route where one operator has an empty type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions operator type validation
Scenario: Route with non-dict value raises ConfigurationError before structural limits
Given a config dict where a route value is not a dict for structural limits
When I call validate_dict with platform_limits max_graph_depth of 5
Then a ConfigurationError is raised
Scenario: Route with non-dict value raises ConfigurationError before total nodes count
Given a config dict with a non-dict route value for total nodes counting
When I call validate_dict with platform_limits max_total_nodes of 100
Then a ConfigurationError is raised
Scenario: Graph depth raises ConfigurationError when nodes field is not a dict
Given a config dict with a graph route that has non-dict nodes for depth calculation
When I call validate_dict with platform_limits max_graph_depth of 5
Then a ConfigurationError is raised
And the error message mentions "nodes"
Scenario: Graph depth check rejects disconnected entry_point
Given a config dict with a graph route whose entry_point does not match any node
When I call validate_dict with platform_limits max_graph_depth of 5
Then a ConfigurationError is raised
And the error message mentions entry_point not found
Scenario: Graph depth limit handles cycles gracefully
Given a config dict with a graph route containing a cycle
When I call validate_dict with platform_limits max_graph_depth of 5
Then no exception is raised
Scenario: Graph depth calculation raises ConfigurationError for non-dict edge entries
Given a config dict with a graph route where edges list contains a non-dict entry
When I call validate_dict with platform_limits max_graph_depth of 5
Then a ConfigurationError is raised
And the error message mentions edge must be a mapping
Scenario: Subgraph depth cycle guard prevents infinite recursion
Given a config dict with two routes that reference each other as subgraphs
When I call validate_dict with platform_limits max_subgraph_depth of 10
Then no exception is raised
Scenario: Subgraph depth raises ConfigurationError for non-existent subgraph reference
Given a config dict with a subgraph node referencing a missing route
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
Scenario: Graph route with non-dict node definitions raises ConfigurationError before subgraph depth check
Given a config dict with a graph route containing non-dict node values
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
Scenario: Graph route with non-dict nodes field raises ConfigurationError before subgraph depth check
Given a config dict with a graph route that has a non-dict nodes field and subgraph limit
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
And the error message mentions "nodes"
# ── Strengthened error message assertions ──────────────────────────────────
Scenario: Non-string agent type raises ConfigurationError with specific message
Given a config dict with an agent that has a non-string type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions non-string agent type
Scenario: Non-string route type raises ConfigurationError with specific message
Given a config dict with a route that has a non-string type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions non-string route type
Scenario: Subgraph node with non-string reference raises ConfigurationError
Given a config dict with a subgraph node referencing a missing route via a list
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
# ── Cycle 3 fixes: entry_point, edge type, subgraph validation ────────────
Scenario: Graph route missing entry_point raises ConfigurationError
Given a config dict with a graph route that has no entry_point field
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "entry_point"
Scenario: Graph route with non-string entry_point raises ConfigurationError in route validation
Given a config dict with a graph route whose entry_point is not a string
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "entry_point"
Scenario: Graph route edge with non-string source raises ConfigurationError
Given a config dict with a graph route where an edge source is a list
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions source and target fields
Scenario: Graph route node with empty string subgraph reference raises ConfigurationError
Given a config dict with a graph route containing a subgraph node with empty reference
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions subgraph reference
Scenario: Graph route node with non-string subgraph reference raises ConfigurationError
Given a config dict with a graph route containing a subgraph node with list reference
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions subgraph reference
Scenario: Route missing type field entirely raises ConfigurationError
Given a config dict with a route that has no type field
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "type"
# ── Cycle 4 fixes: empty-string source/target, empty entry_point ──────────
Scenario: Graph route edge with empty string source raises ConfigurationError
Given a config dict with a graph route where an edge has an empty source
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions source and target fields
Scenario: Graph route edge with empty string target raises ConfigurationError
Given a config dict with a graph route where an edge has an empty target
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions source and target fields
Scenario: Graph route with empty entry_point string raises ConfigurationError
Given a config dict with a graph route that has an empty entry_point string
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "entry_point"
# ── Cycle 4: subgraph pointing to non-graph route ─────────────────────────
Scenario: Subgraph node referencing a stream route raises ConfigurationError
Given a config dict with a subgraph node referencing a stream route
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
Scenario: Subgraph node referencing a bridge route raises ConfigurationError
Given a config dict with a subgraph node referencing a bridge route
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then a ConfigurationError is raised
# ── Coverage gap: non-string stream operator type ─────────────────────────
Scenario: Stream operator with non-string type raises ConfigurationError
Given a config dict with a stream route where one operator has a non-string type
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions non-string operator type
# ── Coverage gap: non-string element in allowed_providers ──────────────────
Scenario: Allowed_providers list with a non-string element raises ConfigurationError
Given a config dict with llm agent and allowed_providers containing a non-string element
When I call validate_dict with platform_limits containing the mixed-type list
Then a ConfigurationError is raised
And the error message mentions allowed_providers type validation
# ── Coverage gap: invalid types on max_total_nodes / max_subgraph_depth ────
Scenario: Boolean True as max_total_nodes raises ConfigurationError
Given a config dict with a graph route containing 3 non-end nodes
When I call validate_dict with platform_limits containing bool max_total_nodes
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: Non-int string as max_total_nodes raises ConfigurationError
Given a config dict with a graph route containing 3 non-end nodes
When I call validate_dict with platform_limits containing string max_total_nodes
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: Boolean True as max_subgraph_depth raises ConfigurationError
Given a config dict with nested subgraphs of depth 2
When I call validate_dict with platform_limits containing bool max_subgraph_depth
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
Scenario: Non-int string as max_subgraph_depth raises ConfigurationError
Given a config dict with nested subgraphs of depth 2
When I call validate_dict with platform_limits containing string max_subgraph_depth
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
# ── Crash-bug regression: non-string entry_point ────────────────────────────
Scenario: Non-string entry_point raises ConfigurationError instead of crashing
Given a config dict with a graph route whose entry_point is not a string
When I call validate_dict with platform_limits max_graph_depth of 5
Then a ConfigurationError is raised
# ── Crash-bug regression: allowed_providers as dict ─────────────────────────
Scenario: Dict as allowed_providers raises ConfigurationError
Given a config dict with platform_limits containing a dict for allowed_providers
When I call validate_dict with platform_limits containing a dict as allowed_providers
Then a ConfigurationError is raised
And the error message mentions allowed_providers type validation
# ── Crash-bug regression: boolean platform_limits ───────────────────────────
Scenario: Boolean True as max_graph_depth raises ConfigurationError
Given a config dict with platform_limits containing a bool for max_graph_depth
When I call validate_dict with platform_limits containing a bool as max_graph_depth
Then a ConfigurationError is raised
And the error message mentions platform_limits type validation
# ── Cycle 5: None argument guards ─────────────────────────────────────────
Scenario: validate_dict with None config_dict raises ConfigurationError
When I call validate_dict with config_dict set to None and empty platform_limits
Then a ConfigurationError is raised
And the error message mentions config_dict and dict
Scenario: validate_dict with None platform_limits raises ConfigurationError
Given a minimal valid config dict with agents and routes
When I call validate_dict with platform_limits set to None
Then a ConfigurationError is raised
And the error message mentions platform_limits and dict
# ── Cycle 5: missing node type key ────────────────────────────────────────
Scenario: Graph node missing type field produces clear error message
Given a config dict with a graph route where a node is missing the type key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions missing type
# ── Cycle 5: ConfigurationError exported from top-level package ───────────
Scenario: ConfigurationError is importable from the top-level cleveractors package
When I import ConfigurationError from the cleveractors package
Then the import succeeds and ConfigurationError is a class
Scenario: ConfigurationError is listed in cleveractors.__all__
When I check the cleveractors package __all__
Then ConfigurationError is listed in the __all__
# ── Cycle 5: non-dict config_dict and platform_limits ──────────────────────
Scenario: validate_dict with list config_dict raises ConfigurationError
When I call validate_dict with config_dict set to an empty list and empty platform_limits
Then a ConfigurationError is raised
And the error message mentions config_dict and dict
Scenario: validate_dict with string platform_limits raises ConfigurationError
Given a minimal valid config dict with agents and routes
When I call validate_dict with platform_limits set to a string
Then a ConfigurationError is raised
And the error message mentions platform_limits and dict
# ── Cycle 7: DFS step guard branches ──────────────────────────────────────
Scenario: DFS step guard in _compute_graph_depth raises ConfigurationError
Given I monkey-patch _MAX_DFS_STEPS to 3
And a config dict with a graph route of 10 edges
When I call validate_dict with platform_limits max_graph_depth of 100
Then a ConfigurationError is raised
And the error message mentions too complex to validate depth
Scenario: Subgraph step guard in _compute_subgraph_depth raises ConfigurationError
Given I monkey-patch _MAX_SUBGRAPH_STEPS to 3
And a config dict with nested subgraphs of depth 4
When I call validate_dict with platform_limits max_subgraph_depth of 10
Then a ConfigurationError is raised
And the error message mentions too complex to validate
# ── Cycle 7: Stronger platform_limits error assertions ────────────────────
Scenario: Negative limit error message mentions "must be >= 0"
Given a config dict with a graph route containing 1 non-end node
When I call validate_dict with platform_limits max_total_nodes of -1
Then a ConfigurationError is raised
And the error message mentions "must be >= 0"
Scenario: Type error limit message mentions "must be an integer"
Given a config dict with a graph route containing 3 non-end nodes
When I call validate_dict with platform_limits containing string max_total_nodes
Then a ConfigurationError is raised
And the error message mentions "must be an integer"
# ── Cycle 8: Subgraph reference validation without max_subgraph_depth ─────
Scenario: Subgraph referencing missing route raises error without max_subgraph_depth
Given a config dict with a subgraph node referencing a missing route without max_subgraph_depth
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Subgraph referencing stream route raises error without max_subgraph_depth
Given a config dict with a subgraph node referencing a stream route without max_subgraph_depth
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
Scenario: Subgraph referencing bridge route raises error without max_subgraph_depth
Given a config dict with a subgraph node referencing a bridge route without max_subgraph_depth
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
# ── Cycle 8: Edge endpoint validation against declared nodes ─────────────
Scenario: Edge referencing a non-existent target node raises ConfigurationError
Given a config dict with a graph route where an edge references a non-existent node
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions edge endpoint not declared
Scenario: Edge referencing a non-existent source node raises ConfigurationError
Given a config dict with a graph route where an edge source references a non-existent node
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions edge endpoint not declared
# ── Cycle 8: LLM provider whitespace stripping ───────────────────────────
Scenario: LLM agent provider with surrounding whitespace passes validation
Given a config dict with an llm agent using provider " openai " with whitespace
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Review fix: reserved agent names (§4.2) ─────────────────────────────
Scenario: Agent name beginning with double underscore raises ConfigurationError
Given a config dict with an agent named "__internal_agent"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions reserved agent name
# ── Review fix: reserved route names (§5.1) ─────────────────────────────
Scenario: Route name beginning with double underscore raises ConfigurationError
Given a config dict with a route named "__input__"
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions reserved route name
# ── Review fix: route type explicitly set to None ──────────────────────
Scenario: Route with type set to None raises ConfigurationError
Given a config dict with a route whose type is null
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions "type"
# ── Review fix: allowed_providers as set and frozenset ──────────────────
Scenario: Allowed_providers as a Python set passes validation
Given a config dict with an llm agent using provider "anthropic"
When I call validate_dict with an allowed_providers set containing anthropic and openai
Then no exception is raised
Scenario: Allowed_providers as a Python frozenset passes validation
Given a config dict with an llm agent using provider "anthropic"
When I call validate_dict with an allowed_providers frozenset containing anthropic and openai
Then no exception is raised
# ── Review fix: max_total_nodes aggregation across multiple graph routes ──
Scenario: max_total_nodes aggregates across multiple graph routes
Given a config dict with two graph routes of 3 nodes each
When I call validate_dict with platform_limits max_total_nodes of 5
Then a ConfigurationError is raised
And the error message mentions total nodes
Scenario: max_total_nodes at exact limit across multiple graph routes
Given a config dict with two graph routes of 3 nodes each
When I call validate_dict with platform_limits max_total_nodes of 6
Then no exception is raised
# ── Review fix: allowed_providers as a tuple ────────────────────────────
Scenario: Allowed_providers as a Python tuple passes validation
Given a config dict with an llm agent using provider "anthropic"
When I call validate_dict with an allowed_providers tuple containing anthropic and openai
Then no exception is raised
# ── Review fix: empty operators list on stream route ───────────────────
Scenario: Stream route with empty operators list passes validation
Given a config dict with a stream route containing no operators
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Review fix: non-string agent key guard ─────────────────────────────
Scenario: Non-string key in agents dict raises ConfigurationError
Given a config dict with a non-string agent key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions agent key must be a string
# ── Review fix: non-string route key guard ─────────────────────────────
Scenario: Non-string key in routes dict raises ConfigurationError
Given a config dict with a non-string route key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions route key must be a string
# ── Review fix: non-string graph node key guard ──────────────────────
Scenario: Non-string key in graph route nodes raises ConfigurationError
Given a config dict with a graph route that has a non-string node key
When I call validate_dict with empty platform_limits
Then a ConfigurationError is raised
And the error message mentions node key must be a string
# ── Review Cycle 2: LLM default provider exclusion test ──────────────
Scenario: LLM agent without provider fails when default openai not in allowed_providers
Given a config dict with an llm agent without a provider field
When I call validate_dict with platform_limits containing allowed_providers "anthropic,google"
Then a ConfigurationError is raised
And the error message mentions provider validation
# ── Review Cycle 2: positive valid node type coverage ─────────────────
Scenario: Graph route with valid node type "start" passes validation
Given a config dict with a graph route containing a node of type "start"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with valid node type "tool" passes validation
Given a config dict with a graph route containing a node of type "tool"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with valid node type "conditional" passes validation
Given a config dict with a graph route containing a node of type "conditional"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with valid node type "message_router" passes validation
Given a config dict with a graph route containing a node of type "message_router"
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Graph route with valid node type "function" passes validation
Given a config dict with a graph route containing a node of type "function"
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Review Cycle 2: positive valid operator type coverage ─────────────
Scenario: Stream route with valid operator type "filter" passes validation
Given a config dict with a stream route containing a "filter" operator
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Stream route with valid operator type "transform" passes validation
Given a config dict with a stream route containing a "transform" operator
When I call validate_dict with empty platform_limits
Then no exception is raised
Scenario: Stream route with valid operator type "reduce" passes validation
Given a config dict with a stream route containing a "reduce" operator
When I call validate_dict with empty platform_limits
Then no exception is raised
# ── Review Cycle 2: duplicate subgraph reference de-duplication ───────
Scenario: Route with duplicate subgraph references does not cause false too-complex error
Given a config dict with a graph route containing two subgraph nodes referencing the same route
When I call validate_dict with platform_limits max_subgraph_depth of 5
Then no exception is raised