cleveragents-core/features/actor_runtime.feature

Feature: Tool-Calling Actor Runtime
  As a developer
  I want a tool-calling runtime for execution actors
  So that actors can invoke tools through an LLM-driven loop with safety limits

  # ---- Single Tool Call ----

  Scenario: Tool-call loop with single tool call
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests one tool call then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Use the echo tool"
    Then the run result content should be "Final answer after tool call"
    And the run result should have 1 tool call records
    And the run result iterations should be 2
    And the run result should not be terminated by limit

  # ---- Multiple Sequential Tool Calls ----

  Scenario: Tool-call loop with multiple sequential tool calls
    Given a tool registry with an echo tool
    And a tool registry with an adder tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests two sequential tool calls then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Use both tools"
    Then the run result content should be "Done after two tool calls"
    And the run result should have 2 tool call records
    And the run result iterations should be 3
    And the run result should not be terminated by limit

  # ---- Max Iteration Safety ----

  Scenario: Max-iteration safety termination
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that always requests tool calls
    And a tool-calling runtime with max iterations 3
    When I run the tool loop with prompt "Keep calling tools"
    Then the run result should be terminated by limit
    And the run result iterations should be 3
    And the run result should have 3 tool call records

  # ---- Tool Not Found Error ----

  Scenario: Tool call error handling for tool not found
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests a nonexistent tool then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Call missing tool"
    Then the run result should have 1 tool call records
    And the first tool call record should have success False
    And the first tool call record should have an error message

  # ---- Tool Execution Error ----

  Scenario: Tool call error handling for execution error
    Given a tool registry with a failing tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests the failing tool then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Call failing tool"
    Then the run result should have 1 tool call records
    And the first tool call record should have success False
    And the first tool call record error should contain "RuntimeError"

  # ---- Metadata Capture ----

  Scenario: Tool metadata capture verification
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests one tool call then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Echo something"
    Then the first tool call record tool name should be "test/echo"
    And the first tool call record inputs should not be empty
    And the first tool call record output should not be empty
    And the first tool call record duration should be greater than 0
    And the first tool call record iteration should be 1

  # ---- Empty Tool List ----

  Scenario: Empty tool list handling
    Given an empty actor tool registry
    And a tool runner for the actor registry
    And a mock LLM caller that responds immediately without tool calls
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "No tools available"
    Then the run result content should be "No tools needed"
    And the run result should have 0 tool call records
    And the run result iterations should be 1

  # ---- Sandbox Root Threading ----

  Scenario: Sandbox root threading into tool inputs
    Given a tool registry with an input-capturing tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests the capturing tool then responds
    And a tool-calling runtime with the mock LLM caller
    And an actor context with sandbox root "/tmp/sandbox"
    When I run the tool loop with prompt "Use sandbox" and the actor context
    Then the captured tool inputs should contain sandbox root "/tmp/sandbox"

  # ---- Resource Binding Threading ----

  Scenario: Resource bindings threading into tool inputs
    Given a tool registry with an input-capturing tool
    And a tool runner for the actor registry
    And a mock LLM caller that requests the capturing tool then responds
    And a tool-calling runtime with the mock LLM caller
    And an actor context with resource bindings
    When I run the tool loop with prompt "Use resources" and the actor context
    Then the captured tool inputs should contain resource bindings

  # ---- Constructor Validation ----

  Scenario: Runtime rejects invalid max iterations
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that responds immediately without tool calls
    Then creating a runtime with max iterations 0 should raise ValueError

  Scenario: Runtime rejects empty prompt
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that responds immediately without tool calls
    And a tool-calling runtime with the mock LLM caller
    Then running the tool loop with empty prompt should raise ValueError

  # ---- Actor Context ----

  Scenario: ToolActorContext construction and properties
    Given an actor context with plan "plan-abc" phase "execute"
    Then the actor context plan id should be "plan-abc"
    And the actor context phase should be "execute"
    And the actor context tool call history should be empty

  Scenario: ToolActorContext records tool calls
    Given an actor context with plan "plan-rec"
    When I record a tool call with name "test/echo" and success True
    Then the actor context tool call history should have 1 records

  Scenario: ToolActorContext rejects empty plan id
    Then creating a tool actor context with empty plan id should raise ValueError

  Scenario: ToolActorContext summary includes metadata
    Given an actor context with plan "plan-sum" sandbox "/tmp/s"
    Then the actor context summary should contain plan id "plan-sum"
    And the actor context summary should contain sandbox root "/tmp/s"

  # ---- Router Integration ----

  Scenario: Tool-call loop with router integration
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a tool call router with plan id "plan-r"
    And a mock LLM caller that requests one tool call then responds
    And a tool-calling runtime with the mock LLM caller and router
    When I run the tool loop with prompt "Use echo with router"
    Then the run result content should be "Final answer after tool call"
    And the run result should have 1 tool call records

  # ---- ToolCallRecord Model ----

  Scenario: ToolCallRecord model fields
    Given a tool call record with name "test/tool" and duration 42.5
    Then the tool call record tool name should be "test/tool"
    And the tool call record duration should be 42.5

  # ---- Default Context ----

  Scenario: Tool loop creates default context when none provided
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that responds immediately without tool calls
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "No context" and no context
    Then the run result content should be "No tools needed"
    And the run result should have 0 tool call records

  # ---- Actor Context Additional Properties ----

  Scenario: ToolActorContext exposes all property accessors
    Given an actor context with all optional fields
    Then the actor context automation profile should be "auto-profile"
    And the actor context project resources should have key "repo"
    And the actor context metadata should have key "version"
    And the actor context clear history should empty the history

  Scenario: ToolActorContext rejects non-ToolCallRecord in record_tool_call
    Given an actor context with plan "plan-typecheck"
    Then recording a non-ToolCallRecord should raise TypeError

  # ---- Runtime Constructor Type Validation ----

  Scenario: Runtime rejects invalid registry type
    Then creating a runtime with a non-ToolRegistry should raise TypeError

  Scenario: Runtime rejects invalid runner type
    Given a tool registry with an echo tool
    Then creating a runtime with a non-ToolRunner should raise TypeError

  # ---- Runtime Property Accessors ----

  Scenario: Runtime exposes max_iterations and provider_format properties
    Given a tool registry with an echo tool
    And a tool runner for the actor registry
    And a mock LLM caller that responds immediately without tool calls
    And a tool-calling runtime with max iterations 5
    Then the runtime max iterations should be 5
    And the runtime provider format should not be None

  # ---- Generic Exception in Tool Execution ----

  Scenario: Tool call handles unexpected generic exception
    Given a tool registry with a tool that raises a generic exception
    And a tool runner for the actor registry
    And a mock LLM caller that requests the generic-error tool then responds
    And a tool-calling runtime with the mock LLM caller
    When I run the tool loop with prompt "Trigger generic error"
    Then the run result should have 1 tool call records
    And the first tool call record should have success False
    And the first tool call record error should contain "KeyError"