feat(agents): surface timeout budget in tool schemas and errors #84

Merged

CoreRasurae merged 3 commits from feature/m2-tool-timeout-awareness into master 2026-07-31 08:31:16 +00:00

Conversation 2 Commits 3 Files Changed 9

+149 -31

CoreRasurae commented 2026-07-30 17:42:20 +00:00

Member

Copy Link

Copy Source

Summary

• Surfaces the tool agent's configurable timeout (seconds) in the LLM-facing schemas for shell and http_request — the only two built-in tools whose execution actually enforces it — so the model knows upfront these calls are time-boxed.
• When either tool times out, the raised error now names timeout as the adjustable config field (not just the elapsed seconds), and http_request now distinguishes a timeout from other request failures instead of collapsing everything into a generic "HTTP request failed" message. Non-timeout HTTP failures are unaffected.
• No new ADR: this stays within ADR-2030's already-accepted extensions (D-1 schema registry, D-4/D-6 error-quality improvements).
• Second, separate commit: wires the previously-unused benchmark_regression nox session into CI as a new, non-blocking benchmark job — ASV benchmarks existed (asv.conf.json, benchmarks/) but were never invoked by any workflow. Kept informational (not in status-check's required list) since a local run took ~15 minutes.

Closes #82

Test plan

• nox -s lint — clean
• nox -s format -- --check — clean
• nox -s typecheck — 0 errors (1 pre-existing, unrelated warning: langchain_google_genai import)
• nox -s security_scan — 0 findings (bandit + semgrep)
• nox -s dead_code — clean
• nox -s unit_tests — 141 features / 2782 scenarios / 13172 steps passed
• nox -s coverage_report — 96.8% (>= 96.5% threshold); llm_tools.py 100%, tool.py 98% (no missing coverage on changed lines)
• nox -s integration_tests (Robot Framework) — 27/27 suites passed
• nox -s benchmark_regression (ASV) — "BENCHMARKS NOT SIGNIFICANTLY CHANGED", as expected for a messaging-only change

🤖 Generated with Claude Code

## Summary - Surfaces the tool agent's configurable `timeout` (seconds) in the LLM-facing schemas for `shell` and `http_request` — the only two built-in tools whose execution actually enforces it — so the model knows upfront these calls are time-boxed. - When either tool times out, the raised error now names `timeout` as the adjustable config field (not just the elapsed seconds), and `http_request` now distinguishes a timeout from other request failures instead of collapsing everything into a generic "HTTP request failed" message. Non-timeout HTTP failures are unaffected. - No new ADR: this stays within ADR-2030's already-accepted extensions (D-1 schema registry, D-4/D-6 error-quality improvements). - Second, separate commit: wires the previously-unused `benchmark_regression` nox session into CI as a new, non-blocking `benchmark` job — ASV benchmarks existed (`asv.conf.json`, `benchmarks/`) but were never invoked by any workflow. Kept informational (not in `status-check`'s required list) since a local run took ~15 minutes. Closes #82 ## Test plan - [x] `nox -s lint` — clean - [x] `nox -s format -- --check` — clean - [x] `nox -s typecheck` — 0 errors (1 pre-existing, unrelated warning: `langchain_google_genai` import) - [x] `nox -s security_scan` — 0 findings (bandit + semgrep) - [x] `nox -s dead_code` — clean - [x] `nox -s unit_tests` — 141 features / 2782 scenarios / 13172 steps passed - [x] `nox -s coverage_report` — 96.8% (>= 96.5% threshold); `llm_tools.py` 100%, `tool.py` 98% (no missing coverage on changed lines) - [x] `nox -s integration_tests` (Robot Framework) — 27/27 suites passed - [x] `nox -s benchmark_regression` (ASV) — "BENCHMARKS NOT SIGNIFICANTLY CHANGED", as expected for a messaging-only change 🤖 Generated with [Claude Code](https://claude.com/claude-code)

CoreRasurae added this to the v2.1.0 milestone 2026-07-30 17:42:20 +00:00

CoreRasurae added the

Type

Feature

label 2026-07-30 17:43:29 +00:00

CoreRasurae added a new dependency 2026-07-30 17:43:31 +00:00

#82 feat(agents): surface timeout budget in tool schemas and errors

CoreRasurae force-pushed feature/m2-tool-timeout-awareness from 0ddbdd8312 to 91cb2b0f95 2026-07-30 17:56:59 +00:00 Compare

CoreRasurae force-pushed feature/m2-tool-timeout-awareness from 91cb2b0f95 to 2d3b45f2f5 2026-07-30 18:15:36 +00:00 Compare

CoreRasurae added 2 commits 2026-07-30 18:24:54 +00:00

ci: wire ASV benchmark regression check into CI as an informational job ... d1417417c8

The `benchmark` and `benchmark_regression` nox sessions existed and
were fully functional (asv.conf.json configured, benchmarks/ populated),
but neither was ever invoked by any CI workflow — only Robot Framework
integration_tests was actually wired in and required for merge.

Adds a `benchmark` job running `nox -s benchmark_regression` (asv
continuous against origin/master). Requires fetch-depth: 0 plus an
explicit fetch of origin/master, since the other jobs' shallow/ref-only
checkout wouldn't have that ref available for the diff.

Deliberately NOT added to status-check's required job list: a local run
of benchmark_regression took ~15 minutes end-to-end (69 benchmarks x 2
commits x repeated timing rounds for statistical confidence), which is
too slow to gate every merge. continue-on-error: true and the nox
session's own success_codes=[0, 2] keep it non-blocking; results are
still uploaded as an artifact for manual inspection, matching the
project's stated intent for benchmark checks to be informational.

Refs: #82

ci(fix): get CI jobs up and running again b8177be061

CoreRasurae force-pushed feature/m2-tool-timeout-awareness from 2d3b45f2f5 to c04b7ba7eb 2026-07-30 18:24:54 +00:00 Compare

CoreRasurae added 1 commit 2026-07-30 18:30:12 +00:00

feat(agents): surface timeout budget in tool schemas and errors ... 4b354f5b2d

The `shell` and `http_request` built-in tools are the only two whose
execution actually honors the tool agent's `timeout` config field
(ToolAgent._execute_shell_command, ToolAgent._http_request_tool), but
their LLM-facing schemas (_BUILTIN_TOOL_SCHEMAS in llm_tools.py) never
told the model a timeout applied, and neither tool's timeout error
named the config field that controls it.

- _BUILTIN_TOOL_SCHEMAS["shell"] and ["http_request"] descriptions now
  state that execution is bounded by the agent's configured `timeout`
  (seconds), matching the existing precedent of documenting `file_read`'s
  `max_chars` behavior in its schema.
- _execute_shell_command's timeout ExecutionError now names `timeout`
  as the adjustable config field, in addition to the elapsed seconds it
  already reported.
- _http_request_tool now catches asyncio.TimeoutError distinctly from
  the generic `except Exception`, so a request timeout produces a
  message naming the config field instead of the previous generic
  "HTTP request failed" text (which gave no indication a timeout was
  the cause). Non-timeout HTTP failures are unaffected — same
  exception handling, same message shape as before.

No change to enforcement mechanics: `self.timeout` remains the sole
config field (default 1s, Actor Configuration Standard §4.5). This is
schema/message content only, within the extension precedent already
established by ADR-2030 (D-1 schema registry, D-4/D-6 error-quality
improvements) — no new ADR was needed for this change.

Tests: extended features/tool_agent.feature (updated the two existing
shell/http_request timeout scenarios to also assert the new config-field
hint) and features/llm_tools_coverage.feature (new scenarios asserting
both schema descriptions mention the timeout budget). Full suite:
141 features / 2782 scenarios / 13172 steps passed; coverage 96.8%
(>=96.5% threshold). Robot integration_tests: 27/27 suites passed.
ASV benchmark_regression: no significant change, as expected for a
messaging-only change.

ISSUES CLOSED: #82

CoreRasurae force-pushed feature/m2-tool-timeout-awareness from c04b7ba7eb to 4b354f5b2d 2026-07-30 18:30:12 +00:00 Compare

CoreRasurae referenced this pull request 2026-07-30 21:48:57 +00:00

feat(agents): add offset-based pagination to file_read #85

hurui200320 approved these changes 2026-07-31 06:10:57 +00:00

hurui200320 left a comment

Member

Copy Link

Copy Source

PR Review: !84 (Ticket #82)

Verdict: Approve

The implementation correctly surfaces the configurable timeout budget in the LLM-facing schemas for shell and http_request, and enriches both timeout error paths with the timeout config field name while preserving the existing message shape for non-timeout HTTP failures. Tests cover the new schema wording and both timeout error messages, plus the non-timeout HTTP failure regression guard. No critical or major correctness, security, or performance issues were found.

Critical Issues

None

Major Issues

None

Minor Issues

• Coverage threshold documentation mismatch (noxfile.py:27, CONTRIBUTING.md:317-318, issue #82 acceptance criteria): The PR reports overall coverage of 96.8%, which passes the COVERAGE_THRESHOLD = 96.5 actually enforced by nox -s coverage_report, but falls below the 97% threshold stated in CONTRIBUTING.md and in the issue's acceptance criteria. This is a pre-existing project inconsistency, but it means the issue's stated coverage criterion is not technically met. Recommendation: reconcile the documented threshold (CONTRIBUTING.md, issue template) with the value in noxfile.py, or bring coverage to ≥97%.

Nits

None

Summary

A focused, well-tested change that extends the ADR-2030 precedent for schema hints and actionable error messages to the timeout config field. The timeout-awareness wording is accurate relative to Actor Configuration Standard §4.5, and the new asyncio.TimeoutError branch in _http_request_tool correctly distinguishes timeouts from other request failures without changing non-timeout behavior. Safe to merge after the minor coverage-threshold discrepancy is acknowledged or resolved.

## PR Review: !84 (Ticket #82) ### Verdict: Approve The implementation correctly surfaces the configurable `timeout` budget in the LLM-facing schemas for `shell` and `http_request`, and enriches both timeout error paths with the `timeout` config field name while preserving the existing message shape for non-timeout HTTP failures. Tests cover the new schema wording and both timeout error messages, plus the non-timeout HTTP failure regression guard. No critical or major correctness, security, or performance issues were found. ### Critical Issues None ### Major Issues None ### Minor Issues - **Coverage threshold documentation mismatch** (`noxfile.py:27`, `CONTRIBUTING.md:317-318`, issue #82 acceptance criteria): The PR reports overall coverage of 96.8%, which passes the `COVERAGE_THRESHOLD = 96.5` actually enforced by `nox -s coverage_report`, but falls below the 97% threshold stated in `CONTRIBUTING.md` and in the issue's acceptance criteria. This is a pre-existing project inconsistency, but it means the issue's stated coverage criterion is not technically met. Recommendation: reconcile the documented threshold (CONTRIBUTING.md, issue template) with the value in `noxfile.py`, or bring coverage to ≥97%. ### Nits None ### Summary A focused, well-tested change that extends the ADR-2030 precedent for schema hints and actionable error messages to the `timeout` config field. The timeout-awareness wording is accurate relative to Actor Configuration Standard §4.5, and the new `asyncio.TimeoutError` branch in `_http_request_tool` correctly distinguishes timeouts from other request failures without changing non-timeout behavior. Safe to merge after the minor coverage-threshold discrepancy is acknowledged or resolved.

hurui200320 approved these changes 2026-07-31 06:15:29 +00:00

CoreRasurae merged commit 4b354f5b2d into master 2026-07-31 08:31:16 +00:00

CoreRasurae deleted branch feature/m2-tool-timeout-awareness 2026-07-31 08:31:19 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

hurui200320

Labels

Clear labels

auto/blocked-by-deps

PR blocked by an open issue dependency. Operator must close the dep (or remove the dependency link) before the merge driver can act. Auto-cleared by merge_drive when no open deps remain.

auto/ci-timeout

Most recent merge cycle hit CI timeout. Driver excludes this PR while last merge_cycle row is < 30 min old; label persists thereafter as visible history.

auto/claimed-implementer

Currently being processed by an implementer worker.

auto/claimed-merge

Currently being processed by the merge driver.

auto/claimed-reviewer

Currently being processed by a reviewer worker.

auto/driver-down

Merge driver heartbeat stale; pipeline halted. Closed automatically on next clean tick.

auto/invariant-violation

Detected master commit violating the strict merge invariant. Tracked as an issue (not a PR label); kept here for label completeness.

auto/last-attempt-tier-0

In-cycle escalation: most recent attempt ran at the Tier 0 slot (`tier-0`). Slot's model defined in .opencode/models/tiers.yaml.

auto/last-attempt-tier-1

In-cycle escalation: most recent attempt ran at the Tier 1 slot (`tier-1`). Slot's model defined in .opencode/models/tiers.yaml.

auto/last-attempt-tier-2

In-cycle escalation: most recent attempt ran at the Tier 2 slot (`tier-2`). Slot's model defined in .opencode/models/tiers.yaml. Gated behind IMPLEMENTER_ESCALATION_TIER2_ENABLED.

auto/last-attempt-tier-min

In-cycle escalation: most recent attempt ran at the Tier -1 slot (`tier-min`). Slot's model defined in .opencode/models/tiers.yaml. Suffix is ``-min`` (not ``--1``) so the Forgejo UI reads naturally.

Automation Tracking

Tracking issues used by the AI Automation system for agents to communicate and report.

auto/needs-conflict-resolution

Rebase conflict needs LLM conflict-resolver.

auto/needs-implementer

Failing CI needs implementer attention.

auto/postmortem

Documenting a driver incident or rollback.

auto/ready-to-merge

Reviewer has APPROVED this PR and no later REQUEST_CHANGES is outstanding. The merge driver requires this label to even consider a PR for merging. Set by the reviewer worker on APPROVE; cleared on REQUEST_CHANGES.

auto/restart-throttled

Train repeatedly lost master-tempo races. Driver excludes via merge_cycle until cooldown elapses; label persists as visible history.

auto/revert

Revert PR backing out an invariant violation. Fast-tracked through the merge driver.

auto/sentinel

Sentinel PR duplicated from upstream into a personal fork by tools/duplicate_prs_to_fork.py for pipeline testing. Lives only in the fork; the canonical pipeline never sees it.

auto/stale-inactivity

No implementer activity for N days. Flagged for human review. Auto-cleared on next push to head branch.

auto/unstable

Repeatedly fails on current master (>= 3 ci-fail-on-rebased-sha releases in 12 h). Excluded from driver until human triage.

Blocked

A ticket in a blocked state and unable to complete until some other task is completed first.

Bounty

$100

A bounty of $100 for any open-source contributor who provides a MR that solves this issue

Bounty

$1000

A bounty of $1000 for any open-source contributor who provides a MR that solves this issue

Bounty

$10000

A bounty of $10000 for any open-source contributor who provides a MR that solves this issue

Bounty

$20

A bounty of $20 for any open-source contributor who provides a MR that solves this issue

Bounty

$2000

A bounty of $2000 for any open-source contributor who provides a MR that solves this issue

Bounty

$250

A bounty of $250 for any open-source contributor who provides a MR that solves this issue

Bounty

$50

A bounty of $50 for any open-source contributor who provides a MR that solves this issue

Bounty

$500

A bounty of $500 for any open-source contributor who provides a MR that solves this issue

Bounty

$5000

A bounty of $5000 for any open-source contributor who provides a MR that solves this issue

Bounty

$750

A bounty of $750 for any open-source contributor who provides a MR that solves this issue

MoSCoW

Could have

Could have feature in order to satisfy the epic/legendary.

MoSCoW

Must have

Must have feature in order to satisfy the epic/legendary.

MoSCoW

Should have

Should have feature in order to satisfy the epic/legendary.

Needs Feedback

There are questions in the ticket that can not be completed until the project owner provides clarity.

Points

1

1 man-hours worth of work for an expert with no learning curve.

Points

13

13 man-hours worth of work for an expert with no learning curve.

Points

2

2 man-hours worth of work for an expert with no learning curve.

Points

21

21 man-hours worth of work for an expert with no learning curve.

Points

3

3 man-hours worth of work for an expert with no learning curve.

Points

34

34 man-hours worth of work for an expert with no learning curve.

Points

5

5 man-hours worth of work for an expert with no learning curve.

Points

55

55 man-hours worth of work for an expert with no learning curve.

Points

8

8 man-hours worth of work for an expert with no learning curve.

Points

88

88 man-hours worth of work for an expert with no learning curve.

Priority

Backlog

This ticket has backlogged priority and is not to be worked on yet

Priority

CI Blocker

Critical priority issue that blocks CI/CD pipeline and prevents PR merges

Priority

Critical

The priority is critical

Priority

High

The priority is high

Priority

Low

The priority is low

Priority

Medium

The priority is medium

Signed-off: Owner

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

Signed-off: Scrum Master

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

Signed-off: Tech Lead

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

Spike

A ticket for learning a tool or technology that is needed to be able to do future planning and design.

State

Completed

The ticket has been fully implemented, completed, and merged with the source code. This label should only be applied once a ticket is closed.

State

Duplicate

A ticket that represents the same content as an existing ticket.

State

In Progress

A ticket that is actively being developed.

State

In Review

A ticket that has had some code completed to implement but is waiting to pass peer review and is not yet merged in.

State

Paused

This ticket's work started but wasn't finished. It's on hold (likely in a feature branch) and will be resumed later, either due to a blocker or a delay.

State

Unverified

All new tickets start in this state. A developer may set it to show the ticket is unverified. This means we haven't agreed to work on it. It will either move to a verified state or be closed as wontdo.

State

Verified

The issue has been verified by a developer as legitimate. It will be worked on and verified tickets are now considered part of the backlog.

State

Wont Do

This ticket has been decided it wont be done. This may mean the bug has been determined to not be real (cant verify) or the feature is one we have decided we dont want to adopt.

Type

Automation

Any edits or discussion about the AI automated coding system.

Type

Bug

Something that doesnt work as intended.

Type

Discussion

Anytime a ticket represents a discussion about a subject and doesnt fall into one of the other categories.

Type

Documentation

An error or improvement needed in the documentation.

Type

Epic

Any first tier epic. That is, an epic which contains only issues as children and will not have sub-epics.

Type

Feature

Some new functionality not present.

Type

Legendary

A type of Epic which will contain other Epics.

Type

Refactor

A code change that restructures existing code without changing its external behavior.

Type

Support

Someone needs help using the project.

Type

Task

A generic task that doesnt fit into the other type categories.

Type

Testing

Work exclusively focusing on fixing or expanding testing.

No Label

Type

Feature

Milestone

No items

No Milestone v2.1.0

Projects

Clear projects

No project

Assignees

Clear assignees

aditya (Aditya Chhabra) aleenaumair (Aleena Umair) brent.edwards (Brent Edwards) CoreRasurae (Luis Mendes) drew (Drew Morris) eugen.thaci (Eugen Thaci) freemo (Jeffrey Phillips Freeman) HAL9000 (HAL 9000) HAL9001 (HAL9001) hamza.khyari (Hamza Khyari) hurui200320 (Rui Hu) justin.morris khird (Kyle Hird) org.cleveragents

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Blocks

#82 feat(agents): surface timeout budget in tool schemas and errors

cleveragents/cleveractors-core

Reference: cleveragents/cleveractors-core#84