feat(guards): implement guard enforcement — denylist, budget caps, and tool call limits #9301

Open
opened 2026-04-14 14:28:23 +00:00 by HAL9000 · 0 comments
Owner

Metadata

  • Branch: feat/guards-denylist-budget-tool-limits
  • Commit: feat(guards): implement guard enforcement — denylist, budget caps, and tool call limits
  • Milestone: v3.5.0

Background and Context

As CleverAgents gains autonomy capabilities (M6: Autonomy Hardening), unconstrained plan execution poses real risks: runaway costs, destructive tool calls, and unbounded resource consumption. Guard enforcement is the safety layer that sits between the plan executor and every tool call, ensuring that no execution can exceed configured limits or invoke forbidden operations.

Guards are configured within automation profiles and follow the established resolution precedence: plan-level > action-level > global. This means a plan can tighten (but not loosen) guards relative to the global defaults, and action-level guards can further constrain within a plan's scope.

Three categories of guards must be enforced:

  1. Denylist — a set of forbidden tool call patterns (e.g., rm -rf /, format C:, DROP TABLE) that must never be executed regardless of context.
  2. Budget caps — monetary and token ceilings (max_cost_usd, max_tokens_total) that halt execution when crossed.
  3. Tool call limits — count-based ceilings (max_tool_calls per plan, max_tool_calls_per_type) that prevent runaway loops or over-reliance on a single tool.

When any guard is triggered, plan execution must halt immediately with a structured error message that identifies which guard was violated and why.


Expected Behavior

  • Every tool call is evaluated against all active guards before execution begins.
  • If a tool call matches a denylist entry, execution is halted and the violation is logged.
  • If cumulative cost or token usage would exceed the configured cap, execution is halted before the call is made.
  • If the total tool call count (or per-type count) would exceed the configured limit, execution is halted.
  • Guard configuration is resolved using the precedence chain: plan-level overrides action-level, which overrides global defaults.
  • agents plan status <PLAN_ID> displays guard violations in the execution log with the guard type, threshold, and actual value.
  • Guard violations produce a structured GuardViolationError with machine-readable fields for downstream tooling.

Acceptance Criteria

  • Denylist guard blocks tool calls matching any forbidden pattern and logs the violation.
  • Budget cap guard (max_cost_usd) halts execution when cumulative cost would exceed the cap.
  • Budget cap guard (max_tokens_total) halts execution when cumulative token usage would exceed the cap.
  • Tool call limit guard (max_tool_calls) halts execution when the per-plan call count would exceed the limit.
  • Tool call limit guard (max_tool_calls_per_type) halts execution when calls to a specific tool type would exceed the per-type limit.
  • Guard configuration is read from the automation profile with correct precedence: plan > action > global.
  • agents plan status <PLAN_ID> shows guard violations in the execution log (guard type, threshold, actual value).
  • GuardViolationError is raised with structured fields: guard_type, threshold, actual, tool_call, plan_id.
  • Guards are evaluated before each tool call (pre-execution hook).
  • All guard types have unit tests with ≥ 97% coverage.
  • BDD scenarios cover: denylist hit, budget cap exceeded, tool call limit exceeded, precedence resolution.

Subtasks

  • Design GuardConfig schema — define denylist, max_cost_usd, max_tokens_total, max_tool_calls, max_tool_calls_per_type fields in the automation profile model.
  • Implement GuardViolationError — structured exception with guard_type, threshold, actual, tool_call, plan_id fields.
  • Implement GuardEvaluator — pre-execution hook that evaluates all active guards before each tool call.
  • Implement denylist guard — pattern matching (glob or regex) against tool call name and arguments.
  • Implement budget cap guards — track cumulative cost_usd and tokens_total per plan execution; halt if next call would exceed cap.
  • Implement tool call limit guards — track tool_call_count and per-type counts per plan execution; halt if next call would exceed limit.
  • Implement guard config resolution — merge guard configs following plan > action > global precedence.
  • Wire GuardEvaluator into plan executor — call evaluator before every tool dispatch.
  • Update agents plan status output — include guard violations section in execution log display.
  • Write unit tests — cover all guard types, edge cases (exactly at limit, one over limit), and precedence resolution.
  • Write BDD scenarios — feature file covering denylist, budget cap, tool call limit, and precedence.
  • Update automation profile documentation — document guard config fields and precedence rules.

Definition of Done

  • All acceptance criteria checkboxes are checked.
  • nox passes with test coverage ≥ 97% including guard-related suites.
  • No regressions in existing automation profile or plan execution tests.
  • agents plan status <PLAN_ID> correctly surfaces guard violations in the execution log.
  • GuardViolationError is documented and used consistently throughout the codebase.
  • Code reviewed and merged to feat/guards-denylist-budget-tool-limits.

Automated by CleverAgents Bot
Agent: new-issue-creator

## Metadata - **Branch:** `feat/guards-denylist-budget-tool-limits` - **Commit:** `feat(guards): implement guard enforcement — denylist, budget caps, and tool call limits` - **Milestone:** v3.5.0 --- ## Background and Context As CleverAgents gains autonomy capabilities (M6: Autonomy Hardening), unconstrained plan execution poses real risks: runaway costs, destructive tool calls, and unbounded resource consumption. Guard enforcement is the safety layer that sits between the plan executor and every tool call, ensuring that no execution can exceed configured limits or invoke forbidden operations. Guards are configured within automation profiles and follow the established resolution precedence: **plan-level > action-level > global**. This means a plan can tighten (but not loosen) guards relative to the global defaults, and action-level guards can further constrain within a plan's scope. Three categories of guards must be enforced: 1. **Denylist** — a set of forbidden tool call patterns (e.g., `rm -rf /`, `format C:`, `DROP TABLE`) that must never be executed regardless of context. 2. **Budget caps** — monetary and token ceilings (`max_cost_usd`, `max_tokens_total`) that halt execution when crossed. 3. **Tool call limits** — count-based ceilings (`max_tool_calls` per plan, `max_tool_calls_per_type`) that prevent runaway loops or over-reliance on a single tool. When any guard is triggered, plan execution must halt immediately with a structured error message that identifies which guard was violated and why. --- ## Expected Behavior - Every tool call is evaluated against all active guards **before** execution begins. - If a tool call matches a denylist entry, execution is halted and the violation is logged. - If cumulative cost or token usage would exceed the configured cap, execution is halted before the call is made. - If the total tool call count (or per-type count) would exceed the configured limit, execution is halted. - Guard configuration is resolved using the precedence chain: plan-level overrides action-level, which overrides global defaults. - `agents plan status <PLAN_ID>` displays guard violations in the execution log with the guard type, threshold, and actual value. - Guard violations produce a structured `GuardViolationError` with machine-readable fields for downstream tooling. --- ## Acceptance Criteria - [ ] Denylist guard blocks tool calls matching any forbidden pattern and logs the violation. - [ ] Budget cap guard (`max_cost_usd`) halts execution when cumulative cost would exceed the cap. - [ ] Budget cap guard (`max_tokens_total`) halts execution when cumulative token usage would exceed the cap. - [ ] Tool call limit guard (`max_tool_calls`) halts execution when the per-plan call count would exceed the limit. - [ ] Tool call limit guard (`max_tool_calls_per_type`) halts execution when calls to a specific tool type would exceed the per-type limit. - [ ] Guard configuration is read from the automation profile with correct precedence: plan > action > global. - [ ] `agents plan status <PLAN_ID>` shows guard violations in the execution log (guard type, threshold, actual value). - [ ] `GuardViolationError` is raised with structured fields: `guard_type`, `threshold`, `actual`, `tool_call`, `plan_id`. - [ ] Guards are evaluated **before** each tool call (pre-execution hook). - [ ] All guard types have unit tests with ≥ 97% coverage. - [ ] BDD scenarios cover: denylist hit, budget cap exceeded, tool call limit exceeded, precedence resolution. --- ## Subtasks - [ ] **Design `GuardConfig` schema** — define `denylist`, `max_cost_usd`, `max_tokens_total`, `max_tool_calls`, `max_tool_calls_per_type` fields in the automation profile model. - [ ] **Implement `GuardViolationError`** — structured exception with `guard_type`, `threshold`, `actual`, `tool_call`, `plan_id` fields. - [ ] **Implement `GuardEvaluator`** — pre-execution hook that evaluates all active guards before each tool call. - [ ] **Implement denylist guard** — pattern matching (glob or regex) against tool call name and arguments. - [ ] **Implement budget cap guards** — track cumulative `cost_usd` and `tokens_total` per plan execution; halt if next call would exceed cap. - [ ] **Implement tool call limit guards** — track `tool_call_count` and per-type counts per plan execution; halt if next call would exceed limit. - [ ] **Implement guard config resolution** — merge guard configs following plan > action > global precedence. - [ ] **Wire `GuardEvaluator` into plan executor** — call evaluator before every tool dispatch. - [ ] **Update `agents plan status` output** — include guard violations section in execution log display. - [ ] **Write unit tests** — cover all guard types, edge cases (exactly at limit, one over limit), and precedence resolution. - [ ] **Write BDD scenarios** — feature file covering denylist, budget cap, tool call limit, and precedence. - [ ] **Update automation profile documentation** — document guard config fields and precedence rules. --- ## Definition of Done - All acceptance criteria checkboxes are checked. - `nox` passes with test coverage ≥ 97% including guard-related suites. - No regressions in existing automation profile or plan execution tests. - `agents plan status <PLAN_ID>` correctly surfaces guard violations in the execution log. - `GuardViolationError` is documented and used consistently throughout the codebase. - Code reviewed and merged to `feat/guards-denylist-budget-tool-limits`. --- **Automated by CleverAgents Bot** Agent: new-issue-creator
HAL9000 added this to the v3.5.0 milestone 2026-04-14 14:30:21 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#9301
No description provided.