feat(execution-limits): add structured ExecutionError kind/reason fields; enforce all 5 execution limits in PureLangGraph #15

New issue

Open

opened 2026-06-03 06:00:01 +00:00 by hurui200320 · 0 comments

hurui200320 commented

2026-06-03 06:00:01 +00:00

Member

Background

PureLangGraph has a hardcoded depth heuristic (max(2000, len(self.nodes) * 50)) and no concept of model-call budgets, tool-call budgets, request timeout, or cost limits. ExecutionError is a bare subclass with no structured fields, making it impossible for the router to programmatically determine why an execution failed.

The CleverThis platform needs to enforce per-plan resource quotas and return the correct HTTP status code (429 for budget exhaustion, 500 for platform configuration errors like a missing pricing entry).

Spec references: ADR-2029 (Execution Limits, Error Mapping)

Depends on: #14 — token data from ActorResult is needed for cost accumulation.

What Is Currently Missing

ExecutionError has no kind or reason fields.
PureLangGraph enforces only a hardcoded depth heuristic; no other limits.
No model-call counting, tool-call counting, timeout wrapping, or cost accumulation.
No pricing table parameter on PureLangGraph or Executor.

Acceptance Criteria

ExecutionError update (cleveractors/core/exceptions.py):

class ExecutionError(CleverAgentsException):
    def __init__(self, message: str, kind: str = "", reason: str = ""):
        super().__init__(message)
        self.kind = kind    # 'depth'|'model_calls'|'tool_calls'|'timeout'|'cost'|''
        self.reason = reason  # 'budget_exhausted'|'missing_pricing_entry'|''

All existing raise ExecutionError(msg) call sites continue to work (both fields default to "").

PureLangGraph limit enforcement (using limits and pricing stored on Executor from C4):

max_depth: Replace heuristic with limits["max_depth"]. Breach → ExecutionError(..., kind="depth").
max_model_calls: Counter per LLM node invocation. Breach → ExecutionError(..., kind="model_calls").
max_tool_calls: Counter per tool node invocation. Breach → ExecutionError(..., kind="tool_calls").
timeout_ms: asyncio.wait_for(coro, timeout=limits["timeout_ms"]/1000). asyncio.TimeoutError → ExecutionError(..., kind="timeout").
max_cost_usd: After each LLM node, compute cost from pricing[provider][model] × token counts. Cumulative breach → ExecutionError(..., kind="cost", reason="budget_exhausted"). Missing pricing entry → ExecutionError(..., kind="cost", reason="missing_pricing_entry") — never proceed with assumed zero cost.

Subtasks

Add kind and reason fields to ExecutionError with default "" values
Thread limits and pricing from Executor down into PureLangGraph
Replace hardcoded depth heuristic with limits["max_depth"]
Add max_model_calls counter and enforcement
Add max_tool_calls counter and enforcement
Wrap execution in asyncio.wait_for for timeout_ms
Add cost accumulation after each LLM node using token data from C5
Enforce max_cost_usd with missing_pricing_entry detection
Export updated ExecutionError from cleveractors/__init__.py and __all__
Write tests for each of the 5 limit types being exceeded
Write test for missing_pricing_entry scenario (never zero-cost fallback)
Verify all existing tests still pass

Definition of Done

All subtasks checked off.
Each of the 5 limit types raises ExecutionError with the correct kind (and reason where applicable).
from cleveractors import ExecutionError exposes the class with kind and reason attributes.
All tests pass. Coverage at or above project threshold.

## Background `PureLangGraph` has a hardcoded depth heuristic (`max(2000, len(self.nodes) * 50)`) and no concept of model-call budgets, tool-call budgets, request timeout, or cost limits. `ExecutionError` is a bare subclass with no structured fields, making it impossible for the router to programmatically determine why an execution failed. The CleverThis platform needs to enforce per-plan resource quotas and return the correct HTTP status code (429 for budget exhaustion, 500 for platform configuration errors like a missing pricing entry). **Spec references:** ADR-2029 (Execution Limits, Error Mapping) **Depends on:** #14 — token data from `ActorResult` is needed for cost accumulation. ## What Is Currently Missing - `ExecutionError` has no `kind` or `reason` fields. - `PureLangGraph` enforces only a hardcoded depth heuristic; no other limits. - No model-call counting, tool-call counting, timeout wrapping, or cost accumulation. - No pricing table parameter on `PureLangGraph` or `Executor`. ## Acceptance Criteria **`ExecutionError` update (`cleveractors/core/exceptions.py`):** ```python class ExecutionError(CleverAgentsException): def __init__(self, message: str, kind: str = "", reason: str = ""): super().__init__(message) self.kind = kind # 'depth'|'model_calls'|'tool_calls'|'timeout'|'cost'|'' self.reason = reason # 'budget_exhausted'|'missing_pricing_entry'|'' ``` All existing `raise ExecutionError(msg)` call sites continue to work (both fields default to `""`). **`PureLangGraph` limit enforcement (using `limits` and `pricing` stored on `Executor` from C4):** 1. **`max_depth`**: Replace heuristic with `limits["max_depth"]`. Breach → `ExecutionError(..., kind="depth")`. 2. **`max_model_calls`**: Counter per LLM node invocation. Breach → `ExecutionError(..., kind="model_calls")`. 3. **`max_tool_calls`**: Counter per tool node invocation. Breach → `ExecutionError(..., kind="tool_calls")`. 4. **`timeout_ms`**: `asyncio.wait_for(coro, timeout=limits["timeout_ms"]/1000)`. `asyncio.TimeoutError` → `ExecutionError(..., kind="timeout")`. 5. **`max_cost_usd`**: After each LLM node, compute cost from `pricing[provider][model]` × token counts. Cumulative breach → `ExecutionError(..., kind="cost", reason="budget_exhausted")`. Missing pricing entry → `ExecutionError(..., kind="cost", reason="missing_pricing_entry")` — **never proceed with assumed zero cost**. ## Subtasks - [ ] Add `kind` and `reason` fields to `ExecutionError` with default `""` values - [ ] Thread `limits` and `pricing` from `Executor` down into `PureLangGraph` - [ ] Replace hardcoded depth heuristic with `limits["max_depth"]` - [ ] Add `max_model_calls` counter and enforcement - [ ] Add `max_tool_calls` counter and enforcement - [ ] Wrap execution in `asyncio.wait_for` for `timeout_ms` - [ ] Add cost accumulation after each LLM node using token data from C5 - [ ] Enforce `max_cost_usd` with `missing_pricing_entry` detection - [ ] Export updated `ExecutionError` from `cleveractors/__init__.py` and `__all__` - [ ] Write tests for each of the 5 limit types being exceeded - [ ] Write test for `missing_pricing_entry` scenario (never zero-cost fallback) - [ ] Verify all existing tests still pass ## Definition of Done - All subtasks checked off. - Each of the 5 limit types raises `ExecutionError` with the correct `kind` (and `reason` where applicable). - `from cleveractors import ExecutionError` exposes the class with `kind` and `reason` attributes. - All tests pass. Coverage at or above project threshold.

Rows
Columns