features/llm_agent_stream_tool_calls.feature

@@ -0,0 +1,48 @@
+Feature: LLMAgent.stream_message() with tool-call support (issue #67)
+
+  # ── Tools configured: single-chunk output ────────────────────────────────
+
+  Scenario: stream_message yields one non-empty chunk when tools configured
+    Given an LLMAgent with tools configured and mock that does one tool round
+    When I call stream_message on the tool-configured agent
+    Then the yielded tokens should contain exactly one non-empty string
+    And _last_token_usage should have non-zero accumulated counts
+
+  Scenario: stream_message last_result token counts equal accumulated totals from tool loop
+    Given an LLMAgent with tools configured and two-round tool mock
+    When I call stream_message on the tool-configured agent
+    Then the accumulated prompt tokens from stream_message should equal two rounds sum
+    And the accumulated completion tokens from stream_message should equal two rounds sum
+
+  Scenario: stream_message on tool actor yields final answer text as single chunk
+    Given an LLMAgent with tools configured and mock returning "Tool loop final answer"
+    When I call stream_message on the tool-configured agent
+    Then the single yielded chunk should be "Tool loop final answer"
+
+  # ── No-tools configured: real astream() path unchanged ───────────────────
+
+  Scenario: stream_message on no-tools actor still uses real astream token-by-token
+    Given an LLMAgent with no tools and a mock astream yielding "tok1", "tok2", "tok3"
+    When I call stream_message on the no-tools agent
+    Then the no-tools stream yielded tokens tok1 tok2 tok3
+    And _last_token_usage should be (0, 0) for no-usage mock
+
+  Scenario: stream_message on no-tools actor captures token counts from astream final chunk
+    Given an LLMAgent with no tools and mock astream with usage prompt=7 completion=13
+    When I call stream_message on the no-tools agent
+    Then _last_token_usage for no-tools stream should be (7, 13)
+
+  # ── Billing integrity on exception ───────────────────────────────────────
+
+  Scenario: stream_message billing integrity preserved on tool loop exception
+    Given an LLMAgent with tools configured and mock that completes one round then errors
+    When I attempt stream_message on the tool-configured agent
+    Then an ExecutionError should be raised from stream_message
+    And _last_token_usage should have the partial accumulated counts
+
+  # ── Memory update for tool path ───────────────────────────────────────────
+
+  Scenario: stream_message with tools and memory_enabled updates conversation history
+    Given an LLMAgent with tools and memory_enabled and mock returning "Memory answer"
+    When I call stream_message on the tool-configured agent
+    Then the conversation memory should contain the tool answer

features/llm_agent_tool_loop.feature

@@ -0,0 +1,97 @@
+Feature: LLMAgent._execute_tool_loop() — shared tool-call orchestration helper
+
+  # ── Basic single tool round ───────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop returns result for single tool round
+    Given I have an LLMAgent with tool loop test config for tool "echo"
+    And I set a mock that makes one tool call then returns a final answer "Final answer from loop"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result final_response content should be "Final answer from loop"
+    And the loop result accumulated_prompt should be greater than 0
+    And the loop result accumulated_completion should be greater than 0
+    And the loop result budget_exhausted should be false
+    And the loop result synthesis_was_run should be false
+
+  # ── Multi-round (two consecutive tool-call rounds) ────────────────────────
+
+  Scenario: _execute_tool_loop accumulates tokens across two tool-call rounds
+    Given I have an LLMAgent with tool loop test config for tool "echo"
+    And I set a mock that makes two tool calls then returns a final answer "Final answer two rounds"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result final_response content should be "Final answer two rounds"
+    And the loop result accumulated_prompt should be at least 30
+    And the loop result accumulated_completion should be at least 15
+
+  # ── tool_max_rounds exhaustion ────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop handles tool_max_rounds exhaustion
+    Given I have an LLMAgent with tool loop test config for tool "echo" and tool_max_rounds 1
+    And I set a mock that always returns tool_calls never a final answer
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result should contain some content from the synthesis prompt
+    And the loop result budget_exhausted should be false
+
+  # ── Token accumulation spanning all rounds ────────────────────────────────
+
+  Scenario: _execute_tool_loop token counts span all ainvoke rounds
+    Given I have an LLMAgent with tool loop test config for tool "echo"
+    And I set a mock that makes two tool calls then returns a final answer "Accumulated answer"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result accumulated_prompt should equal the sum of all rounds prompt tokens
+    And the loop result accumulated_completion should equal the sum of all rounds completion tokens
+
+  # ── Token-budget exhaustion ───────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop triggers synthesis when token budget exhausted
+    Given I have an LLMAgent with tool loop test config for tool "echo" and tiny token_budget_percent
+    And I set a mock that triggers budget exhaustion then returns "Budget exhausted answer"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result budget_exhausted should be true
+    And the loop result final_response content should be "Budget exhausted answer"
+
+  # ── Stuck-model synthesis ─────────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop runs synthesis when model is stuck in tool-only mode
+    Given I have an LLMAgent with tool loop test config for tool "echo" and tool_max_rounds 2
+    And I set a mock that returns empty content tool calls then "Synthesis answer"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result synthesis_was_run should be true
+    And the loop result final_response content should be "Synthesis answer"
+
+  # ── Tool-output pruning ───────────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop runs pruning pass and includes its tokens in accumulated counts
+    Given I have an LLMAgent with tool loop test config for tool "file_read" and pruning enabled
+    And I set a mock that makes one file_read tool call then returns "Pruned answer"
+    And I set a mock pruning model that adds extra tokens
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result final_response content should be "Pruned answer"
+    And the loop result accumulated_prompt should include pruning tokens
+
+  # ── Tool dispatch error ───────────────────────────────────────────────────
+
+  Scenario: _execute_tool_loop captures tool dispatch ExecutionError as ToolMessage
+    Given I have an LLMAgent with tool loop test config for tool "echo"
+    And I set a mock that calls a nonexistent tool then returns "After error answer"
+    When I run _execute_tool_loop with an initial messages list
+    Then the loop result final_response content should be "After error answer"
+    And the messages list should contain a ToolMessage with not available content
+
+  # ── Exception path: partial token counts preserved ────────────────────────
+
+  Scenario: _execute_tool_loop wraps exception in _ToolLoopError with partial counts
+    Given I have an LLMAgent with tool loop test config for tool "echo"
+    And I set a mock that returns one tool round then raises a RuntimeError
+    When I attempt to run _execute_tool_loop with an initial messages list
+    Then a _ToolLoopError should be raised
+    And the _ToolLoopError any_invocation_made should be true
+    And the _ToolLoopError accumulated_prompt should be greater than 0
+
+  # ── Exception before any invocation ──────────────────────────────────────
+
+  Scenario: _ToolLoopError has any_invocation_made false when error before first ainvoke
+    Given I have an LLMAgent with tool loop test config for tool "echo" and bad tool_max_rounds
+    When I attempt to run _execute_tool_loop with an initial messages list
+    Then a _ToolLoopError should be raised
+    And the _ToolLoopError any_invocation_made should be false
+    And the _ToolLoopError cause should be a ConfigurationError

features/steps/llm_agent_stream_tool_calls_steps.py

@@ -0,0 +1,379 @@
+"""Step definitions for LLMAgent.stream_message() with tool-call support tests."""
+
+from __future__ import annotations
+
+import asyncio
+from typing import Any
+from unittest.mock import Mock
+
+from behave import given, then, when
+from langchain_core.messages import AIMessage
+
+from cleveractors.agents.llm import LLMAgent, last_token_usage_var
+from cleveractors.core.exceptions import ExecutionError
+from cleveractors.templates.renderer import TemplateRenderer
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _make_agent(config: dict[str, Any]) -> LLMAgent:
+    renderer = Mock(spec=TemplateRenderer)
+    renderer.render_string.return_value = "system prompt"
+    return LLMAgent(
+        name="stream_tool_agent",
+        config=config,
+        template_renderer=renderer,
+    )
+
+
+def _make_ai_message(
+    content: str,
+    tool_calls: list[dict[str, Any]] | None = None,
+    prompt: int = 10,
+    completion: int = 5,
+) -> AIMessage:
+    return AIMessage(
+        content=content,
+        usage_metadata={
+            "input_tokens": prompt,
+            "output_tokens": completion,
+            "total_tokens": prompt + completion,
+        },
+        tool_calls=tool_calls or [],
+    )
+
+
+async def _collect_stream_message(agent: LLMAgent, message: str = "test") -> list[str]:
+    tokens: list[str] = []
+    async for token in agent.stream_message(message):
+        tokens.append(token)
+    return tokens
+
+
+# ---------------------------------------------------------------------------
+# Given steps
+# ---------------------------------------------------------------------------
+
+
+@given("an LLMAgent with tools configured and mock that does one tool round")
+def step_tool_stream_one_round(context):
+    call_counter = [0]
+    context._stream_tool_round_prompts = [15, 20]
+    context._stream_tool_round_completions = [5, 10]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {"id": "call_001", "name": "echo", "args": {"message": "hi"}}
+                ],
+                prompt=15,
+                completion=5,
+            )
+        return _make_ai_message(content="Final stream answer", prompt=20, completion=10)
+
+    context.stream_tool_agent_config = {
+        "name": "stream_tool_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": "echo"}],
+        "tool_max_rounds": 5,
+    }
+    context.stream_tool_mock_ainvoke = _ainvoke
+
+
+@given("an LLMAgent with tools configured and two-round tool mock")
+def step_tool_stream_two_rounds(context):
+    call_counter = [0]
+    context._stream_two_round_prompts = [15, 15, 20]
+    context._stream_two_round_completions = [5, 5, 10]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] <= 2:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": f"call_{call_counter[0]:03d}",
+                        "name": "echo",
+                        "args": {"message": "input"},
+                    }
+                ],
+                prompt=15,
+                completion=5,
+            )
+        return _make_ai_message(content="Two round answer", prompt=20, completion=10)
+
+    context.stream_tool_agent_config = {
+        "name": "stream_tool_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": "echo"}],
+        "tool_max_rounds": 5,
+    }
+    context.stream_tool_mock_ainvoke = _ainvoke
+
+
+@given('an LLMAgent with tools configured and mock returning "{final_text}"')
+def step_tool_stream_mock_final(context, final_text):
+    async def _ainvoke(messages, **kwargs):
+        return _make_ai_message(content=final_text, prompt=10, completion=5)
+
+    context.stream_tool_agent_config = {
+        "name": "stream_tool_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": "echo"}],
+        "tool_max_rounds": 1,
+    }
+    context.stream_tool_mock_ainvoke = _ainvoke
+
+
+@given('an LLMAgent with no tools and a mock astream yielding "tok1", "tok2", "tok3"')
+def step_no_tools_astream_three_tokens(context):
+    from types import SimpleNamespace
+
+    async def _astream(messages, **kwargs):
+        for t in ["tok1", "tok2", "tok3"]:
+            chunk = SimpleNamespace(content=t)
+            yield chunk
+
+    context.stream_no_tools_config = {
+        "name": "stream_no_tools_agent",
+        "provider": "openai",
+        "api_key": "test",
+    }
+    context.stream_no_tools_mock_astream = _astream
+
+
+@given("an LLMAgent with no tools and mock astream with usage prompt=7 completion=13")
+def step_no_tools_astream_with_usage(context):
+    async def _astream(messages, **kwargs):
+        yield AIMessage(
+            content="answer",
+            usage_metadata={
+                "input_tokens": 7,
+                "output_tokens": 13,
+                "total_tokens": 20,
+            },
+        )
+
+    context.stream_no_tools_config = {
+        "name": "stream_no_tools_agent",
+        "provider": "openai",
+        "api_key": "test",
+    }
+    context.stream_no_tools_mock_astream = _astream
+
+
+@given(
+    "an LLMAgent with tools configured and mock that completes one round then errors"
+)
+def step_tool_stream_one_round_then_error(context):
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {"id": "call_001", "name": "echo", "args": {"message": "hi"}}
+                ],
+                prompt=20,
+                completion=8,
+            )
+        raise RuntimeError("Simulated failure after first tool round")
+
+    context.stream_tool_agent_config = {
+        "name": "stream_tool_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": "echo"}],
+        "tool_max_rounds": 5,
+    }
+    context.stream_tool_mock_ainvoke = _ainvoke
+    context._stream_tool_partial_prompt = 20
+    context._stream_tool_partial_completion = 8
+
+
+@given('an LLMAgent with tools and memory_enabled and mock returning "{final_text}"')
+def step_tool_stream_memory_enabled(context, final_text):
+    async def _ainvoke(messages, **kwargs):
+        return _make_ai_message(content=final_text, prompt=12, completion=6)
+
+    context.stream_tool_agent_config = {
+        "name": "stream_tool_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": "echo"}],
+        "tool_max_rounds": 1,
+        "memory_enabled": True,
+    }
+    context.stream_tool_mock_ainvoke = _ainvoke
+    context._stream_tool_memory_answer = final_text
+
+
+# ---------------------------------------------------------------------------
+# When steps
+# ---------------------------------------------------------------------------
+
+
+@when("I call stream_message on the tool-configured agent")
+def step_call_stream_message_tool(context):
+    agent = _make_agent(context.stream_tool_agent_config)
+    mock_model = Mock()
+    mock_model.temperature = 0.7
+    mock_model.ainvoke = context.stream_tool_mock_ainvoke
+    agent.chat_model = mock_model
+
+    context.stream_tool_tokens = asyncio.run(_collect_stream_message(agent))
+    context.stream_tool_agent = agent
+    context.stream_tool_error = None
+
+
+@when("I call stream_message on the no-tools agent")
+def step_call_stream_message_no_tools(context):
+    agent = _make_agent(context.stream_no_tools_config)
+    mock_model = Mock()
+    mock_model.temperature = 0.7
+    mock_model.astream = context.stream_no_tools_mock_astream
+    agent.chat_model = mock_model
+
+    context.stream_no_tools_tokens = asyncio.run(_collect_stream_message(agent))
+    context.stream_no_tools_agent = agent
+
+
+@when("I attempt stream_message on the tool-configured agent")
+def step_attempt_stream_message_tool(context):
+    agent = _make_agent(context.stream_tool_agent_config)
+    mock_model = Mock()
+    mock_model.temperature = 0.7
+    mock_model.ainvoke = context.stream_tool_mock_ainvoke
+    agent.chat_model = mock_model
+
+    context.stream_tool_agent = agent
+    context.stream_tool_tokens = []
+    context.stream_tool_error = None
+
+    async def _run():
+        async for token in agent.stream_message("test"):
+            context.stream_tool_tokens.append(token)
+
+    try:
+        asyncio.run(_run())
+    except ExecutionError as e:
+        context.stream_tool_error = e
+
+
+# ---------------------------------------------------------------------------
+# Then steps
+# ---------------------------------------------------------------------------
+
+
+@then("the yielded tokens should contain exactly one non-empty string")
+def step_one_non_empty_token(context):
+    tokens = context.stream_tool_tokens
+    non_empty = [t for t in tokens if t.strip()]
+    assert len(non_empty) == 1, (
+        f"Expected exactly 1 non-empty token, got {len(non_empty)}: {tokens}"
+    )
+
+
+@then("_last_token_usage should have non-zero accumulated counts")
+def step_last_token_usage_non_zero(context):
+    agent = context.stream_tool_agent
+    prompt, completion = agent._last_token_usage
+    assert prompt > 0 or completion > 0, (
+        f"Expected non-zero token usage, got ({prompt}, {completion})"
+    )
+
+
+@then("the accumulated prompt tokens from stream_message should equal two rounds sum")
+def step_stream_prompt_equals_two_rounds(context):
+    agent = context.stream_tool_agent
+    prompt, _ = agent._last_token_usage
+    expected = sum(context._stream_two_round_prompts)
+    assert prompt == expected, f"Expected prompt={expected}, got {prompt}"
+
+
+@then(
+    "the accumulated completion tokens from stream_message should equal two rounds sum"
+)
+def step_stream_completion_equals_two_rounds(context):
+    agent = context.stream_tool_agent
+    _, completion = agent._last_token_usage
+    expected = sum(context._stream_two_round_completions)
+    assert completion == expected, f"Expected completion={expected}, got {completion}"
+
+
+@then('the single yielded chunk should be "{expected}"')
+def step_single_chunk_equals(context, expected):
+    tokens = context.stream_tool_tokens
+    combined = "".join(tokens)
+    assert combined == expected, (
+        f"Expected combined tokens={expected!r}, got {combined!r}"
+    )
+
+
+@then("the no-tools stream yielded tokens tok1 tok2 tok3")
+def step_three_tokens(context):
+    assert context.stream_no_tools_tokens == ["tok1", "tok2", "tok3"], (
+        f"Got {context.stream_no_tools_tokens}"
+    )
+
+
+@then("_last_token_usage should be (0, 0) for no-usage mock")
+def step_no_usage_zero(context):
+    agent = context.stream_no_tools_agent
+    assert agent._last_token_usage == (0, 0), (
+        f"Expected (0,0), got {agent._last_token_usage}"
+    )
+
+
+@then("_last_token_usage for no-tools stream should be (7, 13)")
+def step_no_tools_token_counts(context):
+    agent = context.stream_no_tools_agent
+    assert agent._last_token_usage == (7, 13), (
+        f"Expected (7,13), got {agent._last_token_usage}"
+    )
+
+
+@then("an ExecutionError should be raised from stream_message")
+def step_execution_error_raised(context):
+    assert isinstance(context.stream_tool_error, ExecutionError), (
+        f"Expected ExecutionError, got {type(context.stream_tool_error)}"
+    )
+
+
+@then("_last_token_usage should have the partial accumulated counts")
+def step_partial_accumulated_counts(context):
+    agent = context.stream_tool_agent
+    prompt, completion = agent._last_token_usage
+    expected_prompt = context._stream_tool_partial_prompt
+    expected_completion = context._stream_tool_partial_completion
+    assert prompt == expected_prompt and completion == expected_completion, (
+        f"Expected ({expected_prompt},{expected_completion}), got ({prompt},{completion})"
+    )
+
+
+@then("the conversation memory should contain the tool answer")
+def step_memory_contains_answer(context):
+    agent = context.stream_tool_agent
+
+    async def _check_memory():
+        history = await agent.get_memory("conversation_history", [])
+        return history
+
+    history = asyncio.run(_check_memory())
+    answer = context._stream_tool_memory_answer
+    found = any(
+        msg.get("role") == "assistant" and msg.get("content") == answer
+        for msg in history
+    )
+    assert found, f"Expected assistant message with {answer!r} in memory, got {history}"

features/steps/llm_agent_tool_loop_steps.py

@@ -0,0 +1,601 @@
+"""Step definitions for LLMAgent._execute_tool_loop() BDD tests."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from typing import Any
+from unittest.mock import Mock, patch
+
+from behave import given, then, when
+from langchain_core.messages import AIMessage, ToolMessage
+
+from cleveractors.agents.llm import LLMAgent, _ToolLoopError
+from cleveractors.core.exceptions import ConfigurationError
+from cleveractors.templates.renderer import TemplateRenderer
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _make_agent(config: dict[str, Any]) -> LLMAgent:
+    """Create an LLMAgent with a mock TemplateRenderer."""
+    renderer = Mock(spec=TemplateRenderer)
+    renderer.render_string.return_value = "system prompt"
+    return LLMAgent(
+        name="tool_loop_agent",
+        config=config,
+        template_renderer=renderer,
+    )
+
+
+def _make_ai_message(
+    content: str,
+    tool_calls: list[dict[str, Any]] | None = None,
+    prompt: int = 10,
+    completion: int = 5,
+) -> AIMessage:
+    """Return an AIMessage with token usage metadata."""
+    return AIMessage(
+        content=content,
+        usage_metadata={
+            "input_tokens": prompt,
+            "output_tokens": completion,
+            "total_tokens": prompt + completion,
+        },
+        tool_calls=tool_calls or [],
+    )
+
+
+def _initial_messages() -> list[Any]:
+    """Return a minimal messages list (system + user)."""
+    from langchain_core.messages import HumanMessage, SystemMessage
+
+    return [
+        SystemMessage(content="You are a helpful assistant."),
+        HumanMessage(content="Test message"),
+    ]
+
+
+# ---------------------------------------------------------------------------
+# Given steps
+# ---------------------------------------------------------------------------
+
+
+@given('I have an LLMAgent with tool loop test config for tool "{tool_name}"')
+def step_basic_tool_loop_config(context, tool_name):
+    context.tool_loop_config = {
+        "name": "tool_loop_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": tool_name}],
+    }
+    context.tool_loop_tool_name = tool_name
+
+
+@given(
+    'I have an LLMAgent with tool loop test config for tool "{tool_name}" and '
+    "tool_max_rounds {n:d}"
+)
+def step_tool_loop_config_with_max_rounds(context, tool_name, n):
+    context.tool_loop_config = {
+        "name": "tool_loop_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": tool_name}],
+        "tool_max_rounds": n,
+    }
+    context.tool_loop_tool_name = tool_name
+
+
+@given(
+    'I have an LLMAgent with tool loop test config for tool "{tool_name}" and '
+    "tiny token_budget_percent"
+)
+def step_tool_loop_config_tiny_budget(context, tool_name):
+    context.tool_loop_config = {
+        "name": "tool_loop_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": tool_name}],
+        "token_budget_percent": 0.000001,
+        "tool_max_rounds": 5,
+    }
+    context.tool_loop_tool_name = tool_name
+
+
+@given(
+    'I have an LLMAgent with tool loop test config for tool "{tool_name}" and '
+    "pruning enabled"
+)
+def step_tool_loop_config_pruning_enabled(context, tool_name):
+    context.tool_loop_config = {
+        "name": "tool_loop_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": tool_name}],
+        "allow_tool_output_pruning": True,
+        "pruning_threshold": 1,
+        "pruning_tool_filter": [tool_name],
+        "unsafe_mode": True,
+    }
+    context.tool_loop_tool_name = tool_name
+
+
+@given(
+    'I have an LLMAgent with tool loop test config for tool "{tool_name}" and '
+    "bad tool_max_rounds"
+)
+def step_tool_loop_config_bad_max_rounds(context, tool_name):
+    context.tool_loop_config = {
+        "name": "tool_loop_agent",
+        "provider": "openai",
+        "api_key": "test",
+        "tools": [{"name": tool_name}],
+        "tool_max_rounds": "not_a_number",
+    }
+    context.tool_loop_tool_name = tool_name
+
+
+@given(
+    'I set a mock that makes one tool call then returns a final answer "{final_text}"'
+)
+def step_mock_one_tool_call_then_final(context, final_text):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+    context._tool_loop_round_prompts = []
+    context._tool_loop_round_completions = []
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            msg = _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": "call_001",
+                        "name": tool_name,
+                        "args": {"message": "tool input"},
+                    }
+                ],
+                prompt=15,
+                completion=5,
+            )
+            context._tool_loop_round_prompts.append(15)
+            context._tool_loop_round_completions.append(5)
+            return msg
+        msg = _make_ai_message(content=final_text, prompt=20, completion=10)
+        context._tool_loop_round_prompts.append(20)
+        context._tool_loop_round_completions.append(10)
+        return msg
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+    context.tool_loop_config.setdefault("tool_max_rounds", 5)
+
+
+@given(
+    'I set a mock that makes two tool calls then returns a final answer "{final_text}"'
+)
+def step_mock_two_tool_calls_then_final(context, final_text):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+    context._tool_loop_round_prompts = []
+    context._tool_loop_round_completions = []
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] <= 2:
+            msg = _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": f"call_{call_counter[0]:03d}",
+                        "name": tool_name,
+                        "args": {"message": "tool input"},
+                    }
+                ],
+                prompt=15,
+                completion=5,
+            )
+            context._tool_loop_round_prompts.append(15)
+            context._tool_loop_round_completions.append(5)
+            return msg
+        msg = _make_ai_message(content=final_text, prompt=20, completion=10)
+        context._tool_loop_round_prompts.append(20)
+        context._tool_loop_round_completions.append(10)
+        return msg
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+    context.tool_loop_config.setdefault("tool_max_rounds", 5)
+
+
+@given("I set a mock that always returns tool_calls never a final answer")
+def step_mock_always_tool_calls(context):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        return _make_ai_message(
+            content="",
+            tool_calls=[
+                {
+                    "id": f"call_{call_counter[0]:03d}",
+                    "name": tool_name,
+                    "args": {"message": "always calling"},
+                }
+            ],
+            prompt=10,
+            completion=5,
+        )
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+
+
+@given('I set a mock that triggers budget exhaustion then returns "{final_text}"')
+def step_mock_budget_exhaustion(context, final_text):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            # First call: return tool calls so something accumulates
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": "call_001",
+                        "name": tool_name,
+                        "args": {"message": "input"},
+                    }
+                ],
+                prompt=30,
+                completion=20,
+            )
+        # Synthesis call after budget exhausted
+        return _make_ai_message(content=final_text, prompt=10, completion=5)
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+
+
+@given('I set a mock that returns empty content tool calls then "{synthesis_answer}"')
+def step_mock_stuck_then_synthesis(context, synthesis_answer):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        # Check if this is the post-loop synthesis call (no tools in kwargs)
+        if not kwargs.get("tools"):
+            return _make_ai_message(content=synthesis_answer, prompt=12, completion=8)
+        # During the loop: return tool_calls with empty content
+        return _make_ai_message(
+            content="",
+            tool_calls=[
+                {
+                    "id": f"call_{call_counter[0]:03d}",
+                    "name": tool_name,
+                    "args": {"message": "loop call"},
+                }
+            ],
+            prompt=10,
+            completion=5,
+        )
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+
+
+@given('I set a mock that makes one file_read tool call then returns "{final_text}"')
+def step_mock_file_read_then_final(context, final_text):
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": "call_001",
+                        "name": "file_read",
+                        "args": {"path": "/tmp/big_file.txt"},
+                    }
+                ],
+                prompt=15,
+                completion=5,
+            )
+        return _make_ai_message(content=final_text, prompt=18, completion=7)
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+
+
+@given("I set a mock pruning model that adds extra tokens")
+def step_mock_pruning_model(context):
+    from langchain_core.messages import AIMessage as LC_AIMessage
+
+    context.tool_loop_pruning_response = LC_AIMessage(
+        content="[PRUNE_OUTPUT_START]\npruned content\n[PRUNE_OUTPUT_END]",
+        usage_metadata={
+            "input_tokens": 25,
+            "output_tokens": 10,
+            "total_tokens": 35,
+        },
+    )
+    context.tool_loop_extra_pruning_prompt = 25
+    context.tool_loop_extra_pruning_completion = 10
+
+
+@given('I set a mock that calls a nonexistent tool then returns "{final_text}"')
+def step_mock_nonexistent_tool(context, final_text):
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": "call_001",
+                        "name": "nonexistent_tool_xyz",
+                        "args": {},
+                    }
+                ],
+                prompt=12,
+                completion=6,
+            )
+        return _make_ai_message(content=final_text, prompt=10, completion=4)
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+    context.tool_loop_config.setdefault("tool_max_rounds", 5)
+
+
+@given("I set a mock that returns one tool round then raises a RuntimeError")
+def step_mock_one_round_then_error(context):
+    tool_name = getattr(context, "tool_loop_tool_name", "echo")
+    call_counter = [0]
+
+    async def _ainvoke(messages, **kwargs):
+        call_counter[0] += 1
+        if call_counter[0] == 1:
+            return _make_ai_message(
+                content="",
+                tool_calls=[
+                    {
+                        "id": "call_001",
+                        "name": tool_name,
+                        "args": {"message": "input"},
+                    }
+                ],
+                prompt=20,
+                completion=8,
+            )
+        raise RuntimeError("Simulated LLM failure after first round")
+
+    context.tool_loop_mock_ainvoke = _ainvoke
+    context.tool_loop_config.setdefault("tool_max_rounds", 5)
+
+
+# ---------------------------------------------------------------------------
+# When steps
+# ---------------------------------------------------------------------------
+
+
+@when("I run _execute_tool_loop with an initial messages list")
+def step_run_execute_tool_loop(context):
+    agent = _make_agent(context.tool_loop_config)
+
+    # Patch build_chat_model to return a mock with custom ainvoke
+    ainvoke_fn = getattr(context, "tool_loop_mock_ainvoke", None)
+    if ainvoke_fn is not None:
+        mock_model = Mock()
+        mock_model.temperature = 0.7
+        mock_model.ainvoke = ainvoke_fn
+        agent.chat_model = mock_model
+
+    # If pruning model expected, also patch _run_pruning_pass
+    if hasattr(context, "tool_loop_pruning_response"):
+        pruning_resp = context.tool_loop_pruning_response
+
+        async def _mock_pruning_pass(
+            tool_name, raw_output, task_context, prune_context=None
+        ):
+            return (
+                str(pruning_resp.content),
+                context.tool_loop_extra_pruning_prompt,
+                context.tool_loop_extra_pruning_completion,
+            )
+
+        agent._run_pruning_pass = _mock_pruning_pass  # type: ignore[method-assign]
+
+    msgs = _initial_messages()
+    context.tool_loop_messages = msgs
+
+    try:
+        context.tool_loop_result = asyncio.run(agent._execute_tool_loop(msgs))
+        context.tool_loop_error = None
+    except _ToolLoopError as e:
+        context.tool_loop_result = None
+        context.tool_loop_error = e
+    except Exception as e:
+        context.tool_loop_result = None
+        context.tool_loop_error = e
+
+
+@when("I attempt to run _execute_tool_loop with an initial messages list")
+def step_attempt_run_execute_tool_loop(context):
+    """Same as run but expects an error."""
+    step_run_execute_tool_loop(context)
+
+
+# ---------------------------------------------------------------------------
+# Then steps
+# ---------------------------------------------------------------------------
+
+
+@then('the loop result final_response content should be "{expected}"')
+def step_loop_result_content(context, expected):
+    assert context.tool_loop_result is not None, (
+        f"Expected loop result but got error: {context.tool_loop_error}"
+    )
+    actual = str(context.tool_loop_result.final_response.content)
+    assert actual == expected, f"Expected {expected!r}, got {actual!r}"
+
+
+@then("the loop result accumulated_prompt should be greater than 0")
+def step_loop_result_prompt_gt_zero(context):
+    assert context.tool_loop_result is not None
+    assert context.tool_loop_result.accumulated_prompt > 0, (
+        f"Expected prompt > 0, got {context.tool_loop_result.accumulated_prompt}"
+    )
+
+
+@then("the loop result accumulated_completion should be greater than 0")
+def step_loop_result_completion_gt_zero(context):
+    assert context.tool_loop_result is not None
+    assert context.tool_loop_result.accumulated_completion > 0, (
+        f"Expected completion > 0, got {context.tool_loop_result.accumulated_completion}"
+    )
+
+
+@then("the loop result budget_exhausted should be false")
+def step_loop_result_budget_not_exhausted(context):
+    assert context.tool_loop_result is not None
+    assert not context.tool_loop_result.budget_exhausted, (
+        "Expected budget_exhausted=False"
+    )
+
+
+@then("the loop result synthesis_was_run should be false")
+def step_loop_result_synthesis_not_run(context):
+    assert context.tool_loop_result is not None
+    assert not context.tool_loop_result.synthesis_was_run, (
+        "Expected synthesis_was_run=False"
+    )
+
+
+@then("the loop result accumulated_prompt should be at least {n:d}")
+def step_loop_result_prompt_at_least(context, n):
+    assert context.tool_loop_result is not None
+    actual = context.tool_loop_result.accumulated_prompt
+    assert actual >= n, f"Expected accumulated_prompt >= {n}, got {actual}"
+
+
+@then("the loop result accumulated_completion should be at least {n:d}")
+def step_loop_result_completion_at_least(context, n):
+    assert context.tool_loop_result is not None
+    actual = context.tool_loop_result.accumulated_completion
+    assert actual >= n, f"Expected accumulated_completion >= {n}, got {actual}"
+
+
+@then(
+    "the loop result accumulated_prompt should equal the sum of all rounds prompt tokens"
+)
+def step_loop_result_prompt_sum(context):
+    assert context.tool_loop_result is not None
+    expected = sum(context._tool_loop_round_prompts)
+    actual = context.tool_loop_result.accumulated_prompt
+    assert actual == expected, (
+        f"Expected accumulated_prompt={expected} (sum of rounds {context._tool_loop_round_prompts}), "
+        f"got {actual}"
+    )
+
+
+@then(
+    "the loop result accumulated_completion should equal the sum of all rounds completion tokens"
+)
+def step_loop_result_completion_sum(context):
+    assert context.tool_loop_result is not None
+    expected = sum(context._tool_loop_round_completions)
+    actual = context.tool_loop_result.accumulated_completion
+    assert actual == expected, (
+        f"Expected accumulated_completion={expected} (sum of rounds {context._tool_loop_round_completions}), "
+        f"got {actual}"
+    )
+
+
+@then("the loop result budget_exhausted should be true")
+def step_loop_result_budget_exhausted(context):
+    assert context.tool_loop_result is not None
+    assert context.tool_loop_result.budget_exhausted, "Expected budget_exhausted=True"
+
+
+@then("the loop result synthesis_was_run should be true")
+def step_loop_result_synthesis_run(context):
+    assert context.tool_loop_result is not None
+    assert context.tool_loop_result.synthesis_was_run, "Expected synthesis_was_run=True"
+
+
+@then("the loop result should contain some content from the synthesis prompt")
+def step_loop_result_synthesis_content(context):
+    assert context.tool_loop_result is not None
+    # After max_rounds exhaustion, synthesis runs; result content may be empty or
+    # contain synthesis output.  We just check that the loop completed normally.
+    assert context.tool_loop_result.final_response is not None, (
+        "Expected final_response to be set"
+    )
+
+
+@then("the loop result accumulated_prompt should include pruning tokens")
+def step_loop_result_includes_pruning_tokens(context):
+    assert context.tool_loop_result is not None
+    expected_min = context.tool_loop_extra_pruning_prompt
+    actual = context.tool_loop_result.accumulated_prompt
+    assert actual >= expected_min, (
+        f"Expected accumulated_prompt to include pruning tokens ({expected_min}), "
+        f"got {actual}"
+    )
+
+
+@then("the messages list should contain a ToolMessage with not available content")
+def step_messages_contain_not_available(context):
+    msgs = context.tool_loop_messages
+    found = any(
+        isinstance(m, ToolMessage) and "is not available" in str(m.content)
+        for m in msgs
+    )
+    assert found, (
+        f"Expected a ToolMessage with 'is not available' in {[type(m).__name__ for m in msgs]}"
+    )
+
+
+@then("a _ToolLoopError should be raised")
+def step_tool_loop_error_raised(context):
+    assert isinstance(context.tool_loop_error, _ToolLoopError), (
+        f"Expected _ToolLoopError, got {type(context.tool_loop_error).__name__}: "
+        f"{context.tool_loop_error}"
+    )
+
+
+@then("the _ToolLoopError any_invocation_made should be true")
+def step_tool_loop_error_any_invocation_true(context):
+    assert isinstance(context.tool_loop_error, _ToolLoopError)
+    assert context.tool_loop_error.any_invocation_made, (
+        "Expected any_invocation_made=True"
+    )
+
+
+@then("the _ToolLoopError any_invocation_made should be false")
+def step_tool_loop_error_any_invocation_false(context):
+    assert isinstance(context.tool_loop_error, _ToolLoopError)
+    assert not context.tool_loop_error.any_invocation_made, (
+        "Expected any_invocation_made=False"
+    )
+
+
+@then("the _ToolLoopError accumulated_prompt should be greater than 0")
+def step_tool_loop_error_prompt_gt_zero(context):
+    assert isinstance(context.tool_loop_error, _ToolLoopError)
+    assert context.tool_loop_error.accumulated_prompt > 0, (
+        f"Expected accumulated_prompt > 0, got {context.tool_loop_error.accumulated_prompt}"
+    )
+
+
+@then("the _ToolLoopError cause should be a ConfigurationError")
+def step_tool_loop_error_cause_config_error(context):
+    assert isinstance(context.tool_loop_error, _ToolLoopError)
+    assert isinstance(context.tool_loop_error.cause, ConfigurationError), (
+        f"Expected cause=ConfigurationError, got {type(context.tool_loop_error.cause).__name__}"
+    )

robot/ToolCallingTestLib.py

@@ -25,6 +25,7 @@ class ToolCallingTestLib:
         self._last_result: ActorResult | None = None
         self._mock_ainvoke_calls: list[dict[str, Any]] = []
         self._patches: list[Any] = []
+        self._multi_round_expected_prompt: int = 0
 
     def _teardown_patches(self) -> None:
         for p in self._patches:
@@ -198,3 +199,119 @@ class ToolCallingTestLib:
         assert len(self._last_result.response.strip()) > 0, (
             "Expected non-empty response after tool calls"
         )
+
+    # ------------------------------------------------------------------
+    # New keywords for issue #67 (stream_message tool-loop integration)
+    # ------------------------------------------------------------------
+
+    def stream_result_response_is_non_empty(self) -> None:  # noqa: D401
+        """Verify that the stream response is non empty."""
+        assert self._last_result is not None
+        assert len(self._last_result.response.strip()) > 0, (
+            f"Expected non-empty stream response, got: {self._last_result.response!r}"
+        )
+
+    def stream_result_prompt_tokens_greater_than_zero(self) -> None:
+        """Verify stream result has non-zero accumulated prompt tokens."""
+        assert self._last_result is not None
+        assert self._last_result.prompt_tokens > 0, (
+            f"Expected stream prompt_tokens > 0, got {self._last_result.prompt_tokens}"
+        )
+
+    def stream_result_response_contains(self, text: str) -> None:
+        """Verify the stream response contains *text*."""
+        assert self._last_result is not None
+        assert text in self._last_result.response, (
+            f"Expected {text!r} in stream response, got: {self._last_result.response[:200]!r}"
+        )
+
+    def stream_result_prompt_tokens_equals_multi_round_sum(self) -> None:
+        """Verify accumulated prompt tokens equal the sum across all mock rounds."""
+        assert self._last_result is not None
+        expected = self._multi_round_expected_prompt
+        actual = self._last_result.prompt_tokens
+        assert actual == expected, (
+            f"Expected prompt_tokens={expected} (sum of all rounds), got {actual}"
+        )
+
+    def create_executor_with_multi_round_tool_calling_agent(
+        self, tool_name: str = "echo", rounds: int = 2
+    ) -> None:
+        """Create an Executor whose mock model returns tool_calls for *rounds* rounds
+        then a final answer.  Tracks per-round token counts for assertion.
+        """
+        self._teardown_patches()
+        self._mock_ainvoke_calls = []
+
+        # Per-round prompt tokens: 50 per tool round + 30 for final answer
+        tool_round_prompt = 50
+        final_prompt = 30
+        tool_round_completion = 10
+        final_completion = 15
+        total_rounds = int(rounds)
+
+        self._multi_round_expected_prompt = (
+            tool_round_prompt * total_rounds + final_prompt
+        )
+
+        final_response_text = "Multi round final answer"
+
+        def _build_mock_model(*args, **kwargs):  # type: ignore[no-untyped-def]
+            mock_model = MagicMock()
+            mock_model.temperature = 0.7
+
+            async def _mock_ainvoke(messages, **invoke_kwargs):  # type: ignore[no-untyped-def]
+                self._mock_ainvoke_calls.append({"kwargs": invoke_kwargs})
+                call_count = len(self._mock_ainvoke_calls)
+                if call_count <= total_rounds and invoke_kwargs.get("tools"):
+                    return AIMessage(
+                        content="",
+                        usage_metadata={
+                            "input_tokens": tool_round_prompt,
+                            "output_tokens": tool_round_completion,
+                            "total_tokens": tool_round_prompt + tool_round_completion,
+                        },
+                        tool_calls=[
+                            {
+                                "id": f"call_{call_count:03d}",
+                                "name": tool_name,
+                                "args": {"message": f"round_{call_count}"},
+                            }
+                        ],
+                    )
+                return AIMessage(
+                    content=final_response_text,
+                    usage_metadata={
+                        "input_tokens": final_prompt,
+                        "output_tokens": final_completion,
+                        "total_tokens": final_prompt + final_completion,
+                    },
+                )
+
+            mock_model.ainvoke = _mock_ainvoke
+            mock_model.astream = MagicMock()  # not called in tool path
+            return mock_model
+
+        patcher = patch(
+            "cleveractors.agents.llm.build_chat_model",
+            side_effect=_build_mock_model,
+        )
+        patcher.start()
+        self._patches.append(patcher)
+
+        config = {
+            "type": "llm",
+            "name": "multi_round_tool_agent",
+            "provider": "openai",
+            "model": "gpt-3.5-turbo",
+            "config": {
+                "tools": [{"name": tool_name}],
+            },
+        }
+
+        self._executor = create_executor(
+            config_dict=config,
+            credentials={"openai": {"api_key": "mock-key"}},
+            limits={},
+            pricing={},
+        )

robot/llm_tool_calling.robot

@@ -41,12 +41,24 @@ Tool Calling Produces Non-Empty Final Response
     Tool Call Result Is In Response
     Result Prompt Tokens Greater Than Zero
 
-Streaming Path Works With Tool-Configured Agent
-    [Documentation]    execute_stream produces a valid result with a tool-configured
-    ...    agent. Tool calling is not exercised in the streaming path — tools are
-    ...    not passed to the model during stream_message. This test verifies the
-    ...    streaming path itself works and does not crash with a tool-configured agent.
+Streaming Path Exercises Tool Loop Via Execute Stream
+    [Documentation]    execute_stream on a tool-declaring single-LLM actor exercises
+    ...    the _execute_tool_loop() path (issue #67): the tool-call loop runs via
+    ...    ainvoke(), the final response is yielded as a single content chunk, and
+    ...    executor.last_result reports accumulated token counts covering all rounds.
     Create Executor With Tool Calling Agent    tool_name=echo    message=from_stream
     Execute Stream With Message    Tell me about yourself
     Result Is Valid Actor Result
     Result Has Nodes
+    Stream Result Response Is Non Empty
+    Stream Result Prompt Tokens Greater Than Zero
+
+Streaming Path Accumulates Tokens Across Two Tool Rounds
+    [Documentation]    When the mock LLM returns tool_calls for two rounds before the
+    ...    final answer, execute_stream yields the final answer and the accumulated
+    ...    prompt_tokens equals the sum across all three invocations.
+    Create Executor With Multi Round Tool Calling Agent    tool_name=echo    rounds=2
+    Execute Stream With Message    Multi round test
+    Result Is Valid Actor Result
+    Stream Result Response Contains    Multi round final answer
+    Stream Result Prompt Tokens Equals Multi Round Sum