feat(agents): surface timeout budget in tool schemas and errors
CI / lint (pull_request) Successful in 1m5s
CI / quality (pull_request) Successful in 1m49s
CI / typecheck (pull_request) Successful in 2m2s
CI / build (pull_request) Successful in 1m39s
CI / unit_tests (pull_request) Successful in 5m12s
CI / integration_tests (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
CI / security (pull_request) Has been cancelled
CI / benchmark (pull_request) Has been cancelled

The `shell` and `http_request` built-in tools are the only two whose
execution actually honors the tool agent's `timeout` config field
(ToolAgent._execute_shell_command, ToolAgent._http_request_tool), but
their LLM-facing schemas (_BUILTIN_TOOL_SCHEMAS in llm_tools.py) never
told the model a timeout applied, and neither tool's timeout error
named the config field that controls it.

- _BUILTIN_TOOL_SCHEMAS["shell"] and ["http_request"] descriptions now
  state that execution is bounded by the agent's configured `timeout`
  (seconds), matching the existing precedent of documenting `file_read`'s
  `max_chars` behavior in its schema.
- _execute_shell_command's timeout ExecutionError now names `timeout`
  as the adjustable config field, in addition to the elapsed seconds it
  already reported.
- _http_request_tool now catches asyncio.TimeoutError distinctly from
  the generic `except Exception`, so a request timeout produces a
  message naming the config field instead of the previous generic
  "HTTP request failed" text (which gave no indication a timeout was
  the cause). Non-timeout HTTP failures are unaffected — same
  exception handling, same message shape as before.

No change to enforcement mechanics: `self.timeout` remains the sole
config field (default 1s, Actor Configuration Standard §4.5). This is
schema/message content only, within the extension precedent already
established by ADR-2030 (D-1 schema registry, D-4/D-6 error-quality
improvements) — no new ADR was needed for this change.

Tests: extended features/tool_agent.feature (updated the two existing
shell/http_request timeout scenarios to also assert the new config-field
hint) and features/llm_tools_coverage.feature (new scenarios asserting
both schema descriptions mention the timeout budget). Full suite:
141 features / 2782 scenarios / 13172 steps passed; coverage 96.8%
(>=96.5% threshold). Robot integration_tests: 27/27 suites passed.
ASV benchmark_regression: no significant change, as expected for a
messaging-only change.

ISSUES CLOSED: #82
This commit is contained in:
2026-07-30 17:29:38 +00:00
parent b8177be061
commit c04b7ba7eb
6 changed files with 35 additions and 5 deletions
+1
View File
@@ -168,6 +168,7 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm
### Changed
- **Timeout Awareness Surfaced to the LLM for `shell` and `http_request` Tools (issue #82)** (`agents/llm_tools.py`, `agents/tool.py`): The LLM-facing tool-calling schemas for `shell` and `http_request` now state that execution is bounded by the agent's configured `timeout` (seconds) — previously only `file_read`'s `max_chars` guidance was documented in the schema, leaving the model unaware that these two tools could be killed/aborted mid-call. When a timeout is actually triggered, `ToolAgent._execute_shell_command` and `ToolAgent._http_request_tool` now name `timeout` as the adjustable agent config field in the raised `ExecutionError`, in addition to the elapsed seconds already reported. `_http_request_tool` also now catches the timeout condition distinctly from other request failures (connection errors, invalid URLs, etc.), which previously all collapsed into one generic "HTTP request failed" message with no indication a timeout occurred. No change to the underlying `timeout` enforcement mechanics (§4.5) — this is schema/message content only. BDD: scenarios in `features/tool_agent.feature` and `features/llm_tools_coverage.feature`.
- **Real LangChain `usage_metadata` token extraction**: `LLMAgent.process_message()` now reads token counts directly from LangChain response metadata instead of estimating via tiktoken or a 4-chars/token heuristic. `ActorResult.prompt_tokens`/`completion_tokens` now reflect actual LLM billing data.
- **`ActorResult`/`NodeUsage` moved to canonical `cleveractors.result` module**: Both types are now defined in `src/cleveractors/result.py` (ADR-2027 §3.1) and re-exported from `cleveractors.runtime` for backward compatibility. `cleveractors.__init__` updated to import from `result` instead of `runtime`.
- **RxPy Architecture**: Converted routing system from synchronous to reactive (RxPy) streams using Observable/Subject patterns for asynchronous message flow.
+13 -1
View File
@@ -31,4 +31,16 @@ Feature: LLM Tools Coverage
Scenario: normalize_tool_entry rejects non-str non-dict type
Given an llm_tools coverage test environment
When I call normalize_tool_entry with integer 42
Then a ConfigurationError mentioning Invalid tool config entry is raised
Then a ConfigurationError mentioning Invalid tool config entry is raised
Scenario: shell tool schema description mentions the configurable timeout
Given an llm_tools coverage test environment
When I call normalize_tool_entry with string "shell"
Then the result description should mention "timeout"
And the result description should mention "config"
Scenario: http_request tool schema description mentions the configurable timeout
Given an llm_tools coverage test environment
When I call normalize_tool_entry with string "http_request"
Then the result description should mention "timeout"
And the result description should mention "config"
@@ -73,3 +73,10 @@ def step_lt_same_dict(context):
def step_lt_config_error(context, text):
assert context.error is not None
assert text in context.error
@then('the result description should mention "{text}"')
def step_lt_description_mentions(context, text):
assert context.result is not None
description = context.result["function"]["description"]
assert text in description, f"Expected '{text}' in description: {description!r}"
+3 -1
View File
@@ -242,6 +242,7 @@ Feature: ToolAgent Functionality
When I create the ToolAgent
And I process a message with the ToolAgent: "sleep 1"
Then tool execution should fail with message containing "timed out after 0.1 seconds"
And tool execution should fail with message containing "'timeout' config value"
Scenario: Tool agent shell command with non-zero exit code
Given I am running in unsafe mode
@@ -536,7 +537,8 @@ Feature: ToolAgent Functionality
"""
{"tool": "http_request", "args": {"url": "https://httpbin.org/delay/1", "method": "GET"}}
"""
Then tool execution should fail with message containing "HTTP request failed"
Then tool execution should fail with message containing "timed out after 0.001 seconds"
And tool execution should fail with message containing "'timeout' config value"
Scenario: Tool agent http_request tool with invalid URL
Given a ToolAgent is configured with name "http_invalid_agent" and config
+2 -2
View File
@@ -70,7 +70,7 @@ _BUILTIN_TOOL_SCHEMAS: dict[str, dict[str, Any]] = {
},
},
"http_request": {
"description": "Make an HTTP request and return the response.",
"description": "Make an HTTP request and return the response. Execution is bounded by the agent's configured `timeout` (seconds, default 1); requests that exceed it are aborted and return a timeout error — increase `timeout` in the tool agent config for slow endpoints.",
"parameters": {
"type": "object",
"properties": {
@@ -208,7 +208,7 @@ _BUILTIN_TOOL_SCHEMAS: dict[str, dict[str, Any]] = {
},
},
"shell": {
"description": "Execute a shell command and return stdout. Requires allow_shell. Use for: find, grep, cat. Prefer file_read for reading files (has max_chars to prevent context overflow). When creating files in new directories, always use 'mkdir -p <dir> && touch <path>'.",
"description": "Execute a shell command and return stdout. Requires allow_shell. Use for: find, grep, cat. Prefer file_read for reading files (has max_chars to prevent context overflow). When creating files in new directories, always use 'mkdir -p <dir> && touch <path>'. Execution is bounded by the agent's configured `timeout` (seconds, default 1); commands that exceed it are killed and return a timeout error — increase `timeout` in the tool agent config for long-running commands.",
"parameters": {
"type": "object",
"properties": {
+9 -1
View File
@@ -351,7 +351,9 @@ class ToolAgent(Agent):
except asyncio.TimeoutError as timeout_err:
raise ExecutionError(
f"Command '{command}' timed out after {self.timeout} seconds"
f"Command '{command}' timed out after {self.timeout} seconds. "
"Increase the tool agent's 'timeout' config value (in seconds) "
"if this command legitimately needs more time."
) from timeout_err
async def _shell_tool(
@@ -603,6 +605,12 @@ class ToolAgent(Agent):
) as response:
content = await response.text()
return f"Status: {response.status}\n\n{content}"
except asyncio.TimeoutError as timeout_err:
raise ExecutionError(
f"HTTP request to '{url}' timed out after {self.timeout} seconds. "
"Increase the tool agent's 'timeout' config value (in seconds) "
"if this endpoint legitimately needs more time."
) from timeout_err
except Exception as e:
raise ExecutionError(f"HTTP request failed: {e}") from e