CoreRasurae
8f986c1e31
Additional BDD scenarios covering registry resolver errors, cache TTL expiry, runtime dispatch normalization, template base edge cases, validation actor coverage gaps, and YAML Jinja loader deferred rendering.
180 lines
9.2 KiB
Gherkin
180 lines
9.2 KiB
Gherkin
Feature: Runtime Executor API
|
|
As a developer
|
|
I want the router-facing Executor API to correctly create executors and dispatch to LLM, graph, tool, and multi-actor agents
|
|
So that the CleverThis router can invoke actors via a stable interface
|
|
|
|
Background:
|
|
Given the runtime test environment is initialized
|
|
|
|
Scenario: create_executor constructs an Executor with all parameters
|
|
Given a valid actor config dict with type "llm"
|
|
And credentials dict with openai provider
|
|
And limits dict with max_depth 5
|
|
And pricing dict with per_token_cost 0.01
|
|
When I call create_executor
|
|
Then an Executor instance should be returned
|
|
And the executor config should match the config dict
|
|
And the executor credentials should match the credentials dict
|
|
And the executor limits should match the limits dict
|
|
And the executor pricing should match the pricing dict
|
|
|
|
Scenario: create_executor with None limits and pricing defaults to empty dicts
|
|
Given a valid actor config dict with type "llm"
|
|
And credentials dict with openai provider
|
|
When I call create_executor with None limits and pricing
|
|
Then an Executor instance should be returned
|
|
And the executor limits should be an empty dictionary
|
|
And the executor pricing should be an empty dictionary
|
|
|
|
Scenario: execute dispatches to LLM agent when type is "llm"
|
|
Given a valid actor config dict with type "llm" and openai provider
|
|
And credentials dict with openai api key
|
|
When I execute the actor with message "Hello"
|
|
Then the execution should return an ActorResult
|
|
And the ActorResult response should be non-empty
|
|
And the ActorResult should have token usage tracked
|
|
|
|
Scenario: execute dispatches to graph agent when type is "graph"
|
|
Given a valid actor config dict with type "graph" and route definition
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "Hello graph"
|
|
Then the execution should return an ActorResult
|
|
And the ActorResult should have at least one node usage entry
|
|
|
|
Scenario: execute dispatches to tool agent when type is "tool"
|
|
Given a valid actor config dict with type "tool" and tools list
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "echo test"
|
|
Then the execution should return an ActorResult
|
|
And the ActorResult should have zero prompt tokens for tool agents
|
|
|
|
Scenario: execute dispatches to multi_actor when type is "multi_actor"
|
|
Given a multi-actor config dict with multiple sub-actors
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "Hello multi"
|
|
Then the execution should return an ActorResult
|
|
And the node usage IDs should be prefixed with the default actor name
|
|
|
|
Scenario: execute raises ConfigurationError for unknown actor type
|
|
Given a valid actor config dict with type "unknown_type"
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "test"
|
|
Then a ConfigurationError should be raised for runtime
|
|
And the error message should mention the unknown actor type
|
|
|
|
Scenario: _execute_llm handles execution failure gracefully
|
|
Given a valid actor config dict with type "llm" and openai provider
|
|
And credentials dict with openai api key
|
|
And the LLM agent is configured to fail
|
|
When I execute the actor with message "Hello"
|
|
Then an ExecutionError should be raised for runtime with the original cause suppressed
|
|
|
|
Scenario: _execute_graph builds PureGraphConfig from route definition
|
|
Given a valid actor config dict with type "graph" and full route definition
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "Hello graph world"
|
|
Then the execution should return an ActorResult
|
|
|
|
Scenario: _execute_multi_actor raises ConfigurationError with no actors
|
|
Given a multi-actor config dict with empty actors
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "test"
|
|
Then a ConfigurationError should be raised for runtime about no actors
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# n3: create_executor importability from cleveractors package root
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: create_executor is importable from cleveractors package root and in __all__
|
|
Given I import create_executor from the cleveractors package
|
|
Then create_executor should be callable and listed in __all__
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# m3: _execute_multi_actor default_actor fallback
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_multi_actor falls back to first actor when default_actor is not in actors
|
|
Given a multi-actor config dict with default_actor pointing to a non-existent actor
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "Hello fallback"
|
|
Then the execution should return an ActorResult
|
|
And the ActorResult response should be non-empty
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# cc6: _execute_graph with empty nodes_cfg
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_graph handles empty nodes_cfg gracefully
|
|
Given a valid actor config dict with type "graph" and empty route nodes
|
|
And credentials dict with openai provider
|
|
When I execute the actor with message "Hello empty graph"
|
|
Then the execution should return an ActorResult
|
|
And the ActorResult should have at least one node usage entry
|
|
And the ActorResult should have token usage tracked
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# m2: _execute_graph validation branches
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_graph raises ConfigurationError for node missing id key
|
|
Given an Executor for a graph actor with a node missing an id key
|
|
When I execute the graph actor for validation test with message "test"
|
|
Then a ConfigurationError should be raised about invalid graph configuration
|
|
|
|
Scenario: _execute_graph raises ConfigurationError for duplicate node IDs
|
|
Given an Executor for a graph actor with duplicate node IDs
|
|
When I execute the graph actor for validation test with message "test"
|
|
Then a ConfigurationError should be raised about invalid graph configuration
|
|
|
|
Scenario: _execute_graph raises ConfigurationError for invalid edge definition
|
|
Given an Executor for a graph actor with an invalid edge definition
|
|
When I execute the graph actor for validation test with message "test"
|
|
Then a ConfigurationError should be raised about invalid graph configuration
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# m5: float()/int() conversion error paths in _execute_llm
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_llm raises ConfigurationError for non-float temperature
|
|
Given a valid actor config dict with type "llm"
|
|
And the temperature is set to a non-float value "not_a_number"
|
|
And credentials dict with openai api key
|
|
When I execute the actor with message "test"
|
|
Then a ConfigurationError should be raised for runtime
|
|
|
|
Scenario: _execute_llm raises ConfigurationError for non-int max_tokens
|
|
Given a valid actor config dict with type "llm"
|
|
And the max_tokens is set to a non-int value "also_not_a_number"
|
|
And credentials dict with openai api key
|
|
When I execute the actor with message "test"
|
|
Then a ConfigurationError should be raised for runtime
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# M5: cleanup error paths in _execute_llm and _execute_graph
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_llm logs warning when agent cleanup raises RuntimeError
|
|
Given an Executor for an LLM actor with openai credentials dict
|
|
And I arrange for the LLM agent cleanup to raise RuntimeError
|
|
When I execute the LLM actor with cleanup error test
|
|
Then a warning log from runtime_dispatch should be emitted about cleanup failure
|
|
|
|
Scenario: _execute_graph logs warning when agent cleanup raises RuntimeError
|
|
Given an Executor for a graph actor with openai credentials dict
|
|
And I arrange for one graph agent's cleanup to raise RuntimeError
|
|
When I execute the graph actor with cleanup error test
|
|
Then a warning log should be emitted about cleanup failure
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# M1: _execute_graph with actors key (v2.0 convention) regression test
|
|
# ---------------------------------------------------------------------------
|
|
|
|
Scenario: _execute_graph resolves agents from actors key when agents key is absent
|
|
Given an Executor for a graph actor using v2.0 actors key instead of agents key
|
|
When I execute the graph actor for validation test with message "test actors key"
|
|
Then the execution should return an ActorResult
|
|
|
|
Scenario: runtime_types backward-compatibility shim is importable
|
|
When I import from cleaveractors.runtime_types
|
|
Then ActorResult and NodeUsage should be importable
|