Files
cleveractors-core/features/nodes_coverage_gaps.feature
CoreRasurae ceb577e405
CI / lint (pull_request) Successful in 46s
CI / typecheck (pull_request) Successful in 51s
CI / security (pull_request) Successful in 50s
CI / quality (pull_request) Successful in 34s
CI / unit_tests (pull_request) Successful in 3m7s
CI / integration_tests (pull_request) Successful in 1m8s
CI / build (pull_request) Successful in 34s
CI / coverage (pull_request) Successful in 3m8s
CI / status-check (pull_request) Successful in 3s
CI / lint (push) Successful in 46s
CI / typecheck (push) Successful in 51s
CI / security (push) Successful in 51s
CI / quality (push) Successful in 34s
CI / unit_tests (push) Successful in 3m7s
CI / integration_tests (push) Successful in 1m8s
CI / build (push) Successful in 35s
CI / coverage (push) Successful in 3m9s
CI / status-check (push) Successful in 3s
fix(nodes): propagate ExecutionError from retry mechanism instead of swallowing it
When all LLM communication retries are exhausted, call_with_retry() raises
ExecutionError(kind=\"timeout\"). Two broad except Exception handlers in
Node._execute_agent() and Node.execute() systematically caught this,
converting the timeout into an error string in graph state or — in the
streaming path — an empty response.

Added targeted except ExecutionError as e handlers that check
e.kind and re-raise. Non-timeout ExecutionError instances
(tool errors, etc.) continue to be caught and converted to error strings
for graceful graph continuation.

Extend _is_http_comms_error() in retry.py to recognize provider SDK
connection errors (APIConnectionError, APITimeoutError) by class name,
so they are retried with exponential backoff instead of propagating
immediately.

Add safety net in process_message() to wrap non-retried connection
errors as ExecutionError(kind='timeout') for correct propagation
through the node layer (issue #71).

Two new BDD scenarios verify:
- ExecutionError(kind=\"timeout\") propagates with correct kind/reason
- Non-timeout ExecutionError is still caught gracefully

ISSUES CLOSED: #71
2026-07-07 13:47:02 +01:00

179 lines
10 KiB
Gherkin

Feature: Node Execution Without Event Loop, MESSAGE_ROUTER, and History Truncation
As a developer
I want nodes to execute without a running event loop, handle MESSAGE_ROUTER and conditionals, and truncate conversation history
So that nodes function correctly across thread boundaries and maintain bounded conversation memory
Scenario: Execute method handles no running event loop
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute with RuntimeError for get_running_loop (nodes_gaps)
Then the execute method should handle the no-loop fallback (nodes_gaps)
Scenario: Execute method dispatches to MESSAGE_ROUTER branch
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute with MESSAGE_ROUTER node type (nodes_gaps)
Then the execute method should call the message router branch (nodes_gaps)
Scenario: ToolAgent receives current_message from metadata
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with ToolAgent and current_message (nodes_gaps)
Then the ToolAgent should receive the current_message input (nodes_gaps)
Scenario: ToolAgent falls back to last message when no current_message
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with ToolAgent and no current_message (nodes_gaps)
Then the ToolAgent should fall back to last message content (nodes_gaps)
Scenario: LLM agent receives current_message as None
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with current_message set to None (nodes_gaps)
Then the agent should use the last user message as fallback (nodes_gaps)
Scenario: LLM agent handles no user messages in state
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with no user messages (nodes_gaps)
Then the agent should use the last message content as input (nodes_gaps)
Scenario: History truncation sets _history_truncated flag
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise history truncation with many messages (nodes_gaps)
Then the context should include _history_truncated and original length (nodes_gaps)
Scenario: Metadata and nested context are passed into agent context
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise agent context construction with metadata (nodes_gaps)
Then the context should contain metadata keys and flattened nested context (nodes_gaps)
Scenario: Long agent response is logged with truncation debug
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise an agent that returns a long response (nodes_gaps)
Then the debug log should include the truncated response info (nodes_gaps)
Scenario: Agent throws an exception during execution
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with a failing agent (nodes_gaps)
Then the error should be caught and an error response returned (nodes_gaps)
Scenario: ExecutionError with non-empty kind propagates through _execute_agent
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with an agent raising ExecutionError with kind "timeout" (nodes_gaps)
Then ExecutionError with non-empty kind should propagate to the caller (nodes_gaps)
Scenario: ExecutionError with kind "cost" also propagates (future-proof)
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with an agent raising ExecutionError with kind "cost" (nodes_gaps)
Then ExecutionError with non-empty kind should propagate to the caller (nodes_gaps)
Scenario: ExecutionError with empty kind is still caught by _execute_agent
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with an agent raising ExecutionError with empty kind (nodes_gaps)
Then the ExecutionError with empty kind should be caught and converted to error string (nodes_gaps)
Scenario: _is_http_comms_error recognizes APIConnectionError as retryable
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I check _is_http_comms_error against APIConnectionError (nodes_gaps)
Then it should return True for APIConnectionError (nodes_gaps)
Scenario: _is_http_comms_error recognizes APITimeoutError as retryable
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I check _is_http_comms_error against APITimeoutError (nodes_gaps)
Then it should return True for APITimeoutError (nodes_gaps)
Scenario: process_message wraps APIConnectionError as ExecutionError with kind=timeout
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise process_message with an agent raising APIConnectionError (nodes_gaps)
Then the resulting ExecutionError should have kind "timeout" (nodes_gaps)
Scenario: Context diff tracks added and removed keys
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise execute_agent with context-mutating agent (nodes_gaps)
Then state updates should reflect added and removed context keys (nodes_gaps)
Scenario: Dynamic router returns next_node in result
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise dynamic_router with a next_node result (nodes_gaps)
Then the function should return next_node in metadata (nodes_gaps)
Scenario: Dynamic router without router instance logs error
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise dynamic_router with no router instance (nodes_gaps)
Then the function should log an error and return empty dict (nodes_gaps)
Scenario: Dynamic router handles dict and non-dict messages
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise dynamic_router with various message types (nodes_gaps)
Then text extraction should work for dict and non-dict messages (nodes_gaps)
Scenario: Dynamic router handles empty messages state
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise dynamic_router with empty messages (nodes_gaps)
Then the dynamic router message content should default to empty string (nodes_gaps)
Scenario: Conditional message_count with lt and unknown operator
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise message_count conditional with lt operator (nodes_gaps)
Then condition_result should be True when count is less than threshold (nodes_gaps)
Scenario: Conditional content_contains evaluates correctly
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise content_contains conditional (nodes_gaps)
Then it should return True when text is found and False when empty (nodes_gaps)
Scenario: Conditional content_not_contains evaluates correctly
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise content_not_contains conditional (nodes_gaps)
Then it should return False when text is found and True when empty (nodes_gaps)
Scenario: Conditional content_starts_with evaluates correctly
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise content_starts_with conditional (nodes_gaps)
Then it should return True when prefix matches and False when empty (nodes_gaps)
Scenario: Custom conditional with no callable function defaults to True
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise custom conditional without a callable function (nodes_gaps)
Then the condition result should default to True (nodes_gaps)
Scenario: Message router caches router instance and routes messages
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise message router with rules (nodes_gaps)
Then the router should cache the instance and route messages (nodes_gaps)
Scenario: Message router handles no rules configured
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise message router with no rules (nodes_gaps)
Then it should log a warning and return empty dict (nodes_gaps)
Scenario: Message router handles empty messages state
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise message router with empty messages (nodes_gaps)
Then the message router should handle empty state gracefully (nodes_gaps)
Scenario: Message router returns result without next_node
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I exercise message router with a rule not matching (nodes_gaps)
Then it should return the result state without next_node branching (nodes_gaps)
Scenario: Evaluate edge condition with nested event loop
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I evaluate edge condition inside a running event loop (nodes_gaps)
Then it should use ThreadPoolExecutor to run condition (nodes_gaps)
Scenario: Evaluate edge condition with no running loop uses get_event_loop
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I evaluate edge condition with no running event loop (nodes_gaps)
Then it should use get_event_loop to run the condition (nodes_gaps)
Scenario: Evaluate edge condition with closed event loop
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I evaluate edge condition with a closed event loop (nodes_gaps)
Then it should create a new event loop (nodes_gaps)
Scenario: Evaluate edge condition with get_event_loop raising RuntimeError
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I evaluate edge condition and get_event_loop raises (nodes_gaps)
Then it should create a new event loop in the except handler (nodes_gaps)
Scenario: Evaluate edge condition with non-bool condition result
Given a fresh nodes coverage gaps test context (nodes_gaps)
When I evaluate edge condition returning a non-bool value (nodes_gaps)
Then it should coerce the result via bool conversion (nodes_gaps)