fix(graph): enforce USD budget at each agent invocation with pre-flight gate
CI / lint (pull_request) Successful in 50s
CI / typecheck (pull_request) Successful in 55s
CI / security (pull_request) Successful in 53s
CI / quality (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 3m15s
CI / integration_tests (pull_request) Successful in 1m14s
CI / build (pull_request) Successful in 39s
CI / coverage (pull_request) Successful in 3m16s
CI / status-check (pull_request) Successful in 3s
CI / lint (push) Successful in 37s
CI / typecheck (push) Successful in 56s
CI / security (push) Successful in 54s
CI / quality (push) Successful in 37s
CI / unit_tests (push) Successful in 3m15s
CI / integration_tests (push) Successful in 1m14s
CI / build (push) Successful in 39s
CI / coverage (push) Successful in 3m16s
CI / status-check (push) Successful in 3s

Enforces the USD budget (max_cost_usd) as a hard pre-flight gate before
every node execution and at each main-agent ainvoke round inside the
multi-turn tool loop, with pruning passes silently skipped when the
budget is exhausted.

Pre-flight gate (pure_graph.py):
  _check_budget_pre_flight() raises ExecutionError(kind='cost',
  reason='budget_exhausted') immediately before node execution in both
  _execute_from_node and all three branches of _stream_from_node.

In-node budget gating (llm.py):
  _check_budget_before_invoke() gates every _retry_ainvoke() call
  inside _execute_tool_loop, process_message, and stream_message.
  _should_skip_pruning() skips pruning passes when budget exhausted.
  _accumulate_cost_after_invoke() computes USD cost from token counts
  using the pricing table and advances current_accumulated_cost so
  subsequent budget checks see the updated cost.

ContextVar plumbing (retry.py, pure_graph.py):
  current_accumulated_cost, current_max_cost_usd, and current_pricing
  ContextVars carry per-node budget state from PureLangGraph into
  LLMAgent. Set before node.execute() via _set_budget_context_for_node,
  reset in finally blocks via _reset_budget_context.

Reviewer fixes (rui.hu #80):
  - Move cost accumulation inside try block in terminal streaming branch
    to prevent ContextVar leak (was set after finally reset).
  - Standardize log precision to $%.6f across all budget messages.
  - Remove orphaned 'already exhausted' step (criterion unreachable).
  - Add dedicated streaming budget enforcement scenario.
  - Remove redundant = None default on _reset_budget_context pricing_token.

ISSUES CLOSED: #76
This commit was merged in pull request #80.
This commit is contained in:
CoreRasurae
2026-07-09 14:25:31 +00:00
committed by Developer
parent 2072bc3ad1
commit a8f68b8262
12 changed files with 836 additions and 556 deletions
+1
View File
@@ -35,6 +35,7 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm
**Module:** `src/cleveractors/agents/factory.py` (`AgentFactory.__init__`, `_create_agent_instance`), `src/cleveractors/agents/llm.py` (`LLMAgent.__init__`, `_execute_tool_loop`), `src/cleveractors/runtime.py` (`Executor.__init__`, `create_executor`), `src/cleveractors/runtime_dispatch.py` (`_execute_tool`, `_execute_graph`, `_execute_graph_stream`, `_execute_llm`, `_execute_llm_stream`, `_execute_multi_actor`), `src/cleveractors/core/application.py` (`load_configuration`, `_create_agents`). BDD: scenarios in `features/tool_agent_class_injection.feature`.
- **Pre-Flight USD Budget Enforcement at Each Agent Invocation (issue #76)** (`pure_graph.py`, `llm.py`, `retry.py`, `exceptions.py`): Enforces the USD budget (`max_cost_usd`) as a hard pre-flight gate before every node execution and at each main-agent `ainvoke` round inside the multi-turn tool loop, with pruning passes silently skipped when the budget is exhausted. Node-boundary pre-flight uses `>=` (blocks next node when at limit) while post-node checks use `>` (current node completes at limit). The `current_accumulated_cost`, `current_max_cost_usd`, and `current_pricing` ContextVars carry per-node budget state from `PureLangGraph` into `LLMAgent`, enabling in-node cost tracking after each `ainvoke` round so subsequent budget checks see the updated cost. ContextVars are reset in `finally` blocks across all four execution branches (`_execute_from_node`, terminal-stream, intermediate-stream, non-AGENT stream) to prevent leaks. Cost formula deduplication: `_compute_cost` in `exceptions.py` centralises the pricing-lookup + cost-computation logic formerly duplicated across 5 call sites (3 in `pure_graph.py`, 1 in `runtime_dispatch.py`, 1 in `llm.py`), eliminating 267 lines of redundancy. Single-LLM streaming path (`_execute_llm_stream`) now raises `missing_pricing_entry` when `max_cost_usd` is specified but the pricing table is empty, mirroring the graph-path behaviour. BDD: scenarios in `features/execution_limits.feature`, `features/llm_agent_tool_loop.feature`, and `features/execute_stream.feature`.
- **LLM Agent Token-Budget Awareness and Tool Output Pruning (issue #61, #65)** (`llm.py`): Two complementary mechanisms to prevent context-window exhaustion in the multi-turn tool-call loop, with accurate token tracking for billing.
**Token-budget awareness** (`token_budget_percent` config, default off): tracks actual token consumption from LLM response metadata before each invocation. Emits a warning at 75% budget consumption. When the budget ceiling is exceeded, injects a synthesis prompt, permits one final tool-call round, then forces a text-only response. Token counts from all rounds (including budget-exhaustion synthesis, stuck-model synthesis, and pruning passes) are accumulated for accurate billing.
+20
View File
@@ -227,6 +227,26 @@ def after_scenario(context, scenario):
except Exception:
pass
# Reset budget ContextVars set by test steps (issue #76 test hygiene).
# Without this, budget ContextVars set in llm_agent_tool_loop scenarios
# leak across scenarios and can cause order-dependent flakiness
# (Graa review PR #80).
if hasattr(context, "_budget_cost_token"):
from cleveractors.agents.retry import (
current_accumulated_cost,
current_max_cost_usd,
current_pricing,
)
try:
current_accumulated_cost.reset(context._budget_cost_token)
if getattr(context, "_budget_max_token", None) is not None:
current_max_cost_usd.reset(context._budget_max_token)
if getattr(context, "_budget_pricing_token", None) is not None:
current_pricing.reset(context._budget_pricing_token)
except Exception:
pass
# Restore monkey-patched validation constants
if hasattr(context, "_original_max_dfs"):
import cleveractors.validation._limits as _limits
+12
View File
@@ -866,6 +866,18 @@ Feature: Executor.execute_stream() - token-by-token streaming delivery
# ── Major #1 fix: max_cost_usd enforced for intermediate AGENT nodes ──────
Scenario: Streaming graph exhausts budget via post-node check on first node
Given an Executor with a two-agent sequential graph and max_cost_usd 0.40 with pricing (stream)
And a mock astream yielding single token "BudgetNodeCost" with usage prompt=3000000 completion=0 (stream)
When I attempt execute_stream expecting an error with message "test" (stream)
Then an ExecutionError with kind "cost" and reason "budget_exhausted" should be raised (stream kind+reason)
Scenario: Second streaming node blocked by pre-flight when first node exactly hits limit
Given an Executor with a two-agent sequential graph and max_cost_usd 0.30 at 0.15/M prompt pricing (stream)
And a mock astream yielding single token "ExactPreFlight" with usage prompt=2000000 completion=0 (stream)
When I attempt execute_stream expecting an error with message "test" (stream)
Then an ExecutionError with kind "cost" and reason "budget_exhausted" should be raised (stream kind+reason)
Scenario: execute_stream raises ExecutionError when intermediate AGENT node exceeds max_cost_usd
Given an Executor with a two-agent sequential graph and max_cost_usd 0.0 with pricing (stream)
And a mock astream yielding single token "IntCostToken" with usage prompt=1000 completion=1000 (stream)
+23 -2
View File
@@ -108,10 +108,11 @@ Feature: Execution Limits Enforcement in PureLangGraph (ADR-2029)
Then an ExecutionError should be raised with kind "cost" (elim)
And the reason should be "missing_pricing_entry" (elim)
Scenario: Empty pricing table skips cost calculation entirely
Scenario: Empty pricing table raises missing_pricing_entry when limit is specified
Given a graph with an LLM node using 2000000 prompt tokens and max_cost_usd 0.001 (elim)
When the graph is executed with an empty pricing table (elim)
Then no ExecutionError should be raised (elim)
Then an ExecutionError should be raised with kind "cost" (elim)
And the reason should be "missing_pricing_entry" (elim)
Scenario: LLM node with incomplete model pricing entry missing prompt rate raises missing_pricing_entry
Given a graph with an LLM node for provider "openai" model "gpt-4.1-mini" (elim)
@@ -205,6 +206,26 @@ Feature: Execution Limits Enforcement in PureLangGraph (ADR-2029)
Then no ExecutionError should be raised (elim)
And a warning should have been logged for non-dict token usage (elim)
# ── Budget pre-flight enforcement (issue #76) ──────────────────────────────
Scenario: Zero max_cost_usd triggers pre-flight failure before first node
Given a graph with an LLM node using 100 prompt tokens and max_cost_usd 0.00 (elim)
When the graph is executed with pricing for "openai"/"gpt-4.1-mini" at 0.15 prompt rate (elim)
Then an ExecutionError should be raised with kind "cost" (elim)
And the reason should be "budget_exhausted" (elim)
Scenario: Two LLM nodes exhaust budget via post-node check on first node
Given a graph with 2 LLM nodes each using 3000000 prompt tokens and max_cost_usd 0.40 (elim)
When the graph is executed with pricing for "openai"/"gpt-4.1-mini" at 0.15 prompt rate (elim)
Then an ExecutionError should be raised with kind "cost" (elim)
And the reason should be "budget_exhausted" (elim)
Scenario: Second LLM node blocked by pre-flight when first node exactly hits limit
Given a graph with 2 LLM nodes each using 2000000 prompt tokens and max_cost_usd 0.30 (elim)
When the graph is executed with pricing for "openai"/"gpt-4.1-mini" at 0.15 prompt rate (elim)
Then an ExecutionError should be raised with kind "cost" (elim)
And the reason should be "budget_exhausted" (elim)
# ── Happy-path result correctness ─────────────────────────────────────────
Scenario: Graph within max_depth returns non-None result
+19
View File
@@ -95,3 +95,22 @@ Feature: LLMAgent._execute_tool_loop() — shared tool-call orchestration helper
Then a _ToolLoopError should be raised
And the _ToolLoopError any_invocation_made should be false
And the _ToolLoopError cause should be a ConfigurationError
# ── USD cost budget exhaustion (in-node gating, issue #76) ────────────────
Scenario: _execute_tool_loop raises budget_exhausted when pricing context is set and cost exceeds limit
Given I have an LLMAgent with tool loop test config for tool "echo" and tool_max_rounds 5
And I set a mock that makes one tool call then returns a final answer "cost triggered"
And the USD budget ContextVars are set for in-node gating
When I run _execute_tool_loop with an initial messages list
Then an ExecutionError with kind "cost" and reason "budget_exhausted" should be raised
Scenario: _execute_tool_loop skips pruning pass when budget tripped mid-loop (issue #76)
Given I have an LLMAgent with tool loop test config for tool "file_read" and pruning enabled
And I set a mock that makes one file_read tool call then returns "Final text"
And I set a mock pruning model that adds extra tokens
And the USD budget ContextVars are set so first round exhausts limit
When I run _execute_tool_loop with an initial messages list
Then an ExecutionError with kind "cost" and reason "budget_exhausted" should be raised
And the accumulated prompt should NOT include pruning tokens
And the accumulated completion should NOT include pruning tokens
+51
View File
@@ -4762,6 +4762,57 @@ def _two_agent_sequential_config() -> dict[str, Any]:
}
@given(
"an Executor with a two-agent sequential graph and"
" max_cost_usd 0.40 with pricing (stream)"
)
def step_es_two_agent_max_cost_040(context: Any) -> None:
"""Graph: START → agent_a (AGENT, intermediate) → agent_b (AGENT, terminal) → END.
max_cost_usd=0.40 with prompt pricing at 0.50/M so that a first node
producing 3M prompt tokens costs $1.50 and exhausts the budget. The
post-node cost check catches the exhaustion; the pre-flight gate is also
exercised as a no-op before the first node. This exercises the streaming
path cost enforcement for a non-zero budget (issue #76, AC4).
"""
context.es_executor = create_executor(
config_dict=_two_agent_sequential_config(),
credentials={"openai": {"api_key": "test-key"}},
limits={"max_cost_usd": 0.40},
pricing={
"openai": {
"gpt-3.5-turbo": {"prompt": 0.50, "completion": 1.50},
}
},
)
@given(
"an Executor with a two-agent sequential graph and"
" max_cost_usd 0.30 at 0.15/M prompt pricing (stream)"
)
def step_es_two_agent_max_cost_030_exact(context: Any) -> None:
"""Graph: START → agent_a (AGENT, intermediate) → agent_b (AGENT, terminal) → END.
max_cost_usd=0.30 with prompt pricing at 0.15/M so that a first node
producing 2M prompt tokens costs exactly $0.30. The post-node check uses
``>`` so the first node completes (0.30 is NOT > 0.30). The second node's
pre-flight gate uses ``>=`` so 0.30 >= 0.30 blocks it, exercising the
intentional asymmetry between post-node and pre-flight checks
(Graa review PR #80).
"""
context.es_executor = create_executor(
config_dict=_two_agent_sequential_config(),
credentials={"openai": {"api_key": "test-key"}},
limits={"max_cost_usd": 0.30},
pricing={
"openai": {
"gpt-3.5-turbo": {"prompt": 0.15, "completion": 0.60},
}
},
)
@given(
"an Executor with a two-agent sequential graph and max_cost_usd 1.0"
" with missing provider pricing (stream)"
+87 -1
View File
@@ -11,7 +11,12 @@ from behave import given, then, when
from langchain_core.messages import AIMessage, ToolMessage
from cleveractors.agents.llm import LLMAgent, _ToolLoopError
from cleveractors.core.exceptions import ConfigurationError
from cleveractors.agents.retry import (
current_accumulated_cost,
current_max_cost_usd,
current_pricing,
)
from cleveractors.core.exceptions import ConfigurationError, ExecutionError
from cleveractors.templates.renderer import TemplateRenderer
# ---------------------------------------------------------------------------
@@ -383,6 +388,39 @@ def step_mock_one_round_then_error(context):
# ---------------------------------------------------------------------------
@given("the USD budget ContextVars are set for in-node gating")
def step_set_usd_budget(context):
"""Set budget ContextVars so the in-node cost gating fires.
Uses a pricing table where a single 15/5 token invoke at
$1000/M prompt/$1000/M completion costs $0.02, exceeding $0.01 limit.
"""
# Cost per round: 15/1M * 1000 + 5/1M * 1000 = 0.015 + 0.005 = $0.02
# With limit $0.01, the second main invoke check should fire.
# Save previous tokens so after_scenario can reset them (issue #76 test hygiene).
context._budget_cost_token = current_accumulated_cost.set(0.0)
context._budget_max_token = current_max_cost_usd.set(0.01)
context._budget_pricing_token = current_pricing.set(
{"openai": {"gpt-3.5-turbo": {"prompt": 1000.0, "completion": 1000.0}}}
)
@given("the USD budget ContextVars are set so first round exhausts limit")
def step_set_usd_budget_exhausted_mid_round(context):
"""Set budget ContextVars so the first round ainvoke exhausts the limit.
The pre-flight check passes (0.0 < limit), but after the first ainvoke
round the accumulated cost exceeds the limit. This triggers pruning skip
and then ExecutionError on the next round pre-flight check.
"""
# Save previous tokens so after_scenario can reset them (issue #76 test hygiene).
context._budget_cost_token = current_accumulated_cost.set(0.0)
context._budget_max_token = current_max_cost_usd.set(0.015)
context._budget_pricing_token = current_pricing.set(
{"openai": {"gpt-3.5-turbo": {"prompt": 1000.0, "completion": 1000.0}}}
)
@when("I run _execute_tool_loop with an initial messages list")
def step_run_execute_tool_loop(context):
agent = _make_agent(context.tool_loop_config)
@@ -549,6 +587,38 @@ def step_loop_result_includes_pruning_tokens(context):
)
@then("the accumulated prompt should NOT include pruning tokens")
def step_accumulated_prompt_no_pruning_tokens(context):
if hasattr(context, "tool_loop_extra_pruning_prompt"):
expected_min = context.tool_loop_extra_pruning_prompt
if context.tool_loop_result is not None:
actual = context.tool_loop_result.accumulated_prompt
elif isinstance(getattr(context, "tool_loop_error", None), _ToolLoopError):
actual = context.tool_loop_error.accumulated_prompt
else:
actual = 0
assert actual < expected_min, (
f"Expected accumulated_prompt to NOT include pruning tokens, "
f"but got {actual} >= pruning threshold {expected_min}"
)
@then("the accumulated completion should NOT include pruning tokens")
def step_accumulated_completion_no_pruning_tokens(context):
if hasattr(context, "tool_loop_extra_pruning_completion"):
expected_min = context.tool_loop_extra_pruning_completion
if context.tool_loop_result is not None:
actual = context.tool_loop_result.accumulated_completion
elif isinstance(getattr(context, "tool_loop_error", None), _ToolLoopError):
actual = context.tool_loop_error.accumulated_completion
else:
actual = 0
assert actual < expected_min, (
f"Expected accumulated_completion to NOT include pruning tokens, "
f"but got {actual} >= pruning threshold {expected_min}"
)
@then("the messages list should contain a ToolMessage with not available content")
def step_messages_contain_not_available(context):
msgs = context.tool_loop_messages
@@ -599,3 +669,19 @@ def step_tool_loop_error_cause_config_error(context):
assert isinstance(context.tool_loop_error.cause, ConfigurationError), (
f"Expected cause=ConfigurationError, got {type(context.tool_loop_error.cause).__name__}"
)
@then('an ExecutionError with kind "{kind}" and reason "{reason}" should be raised')
def step_execution_error_kind_reason(context, kind, reason):
assert context.tool_loop_error is not None, "Expected an error but got none"
# The error may be a _ToolLoopError wrapping an ExecutionError
cause = context.tool_loop_error
if isinstance(cause, _ToolLoopError):
cause = cause.cause
assert isinstance(cause, ExecutionError), (
f"Expected ExecutionError, got {type(cause).__name__}: {cause}"
)
assert cause.kind == kind, f"Expected kind={kind!r}, got {cause.kind!r}: {cause}"
assert cause.reason == reason, (
f"Expected reason={reason!r}, got {cause.reason!r}: {cause}"
)
+154 -1
View File
@@ -39,13 +39,20 @@ from cleveractors.agents.base import AgentWithMemory
from cleveractors.agents.llm_client import build_chat_model
from cleveractors.agents.llm_imports import populate_langchain_globals
from cleveractors.agents.llm_tools import normalize_tool_entry as _normalize_tool_entry
from cleveractors.agents.retry import _get_provider_url, call_with_retry
from cleveractors.agents.retry import (
_get_provider_url,
call_with_retry,
current_accumulated_cost,
current_max_cost_usd,
current_pricing,
)
from cleveractors.agents.tool import ToolAgent
from cleveractors.core.exceptions import (
AgentCreationError,
ConfigurationError,
ExecutionError,
MissingUsageMetadataError,
_compute_cost,
)
from cleveractors.result import MAX_REASONABLE_TOKENS
from cleveractors.templates.renderer import TemplateRenderer
@@ -517,6 +524,114 @@ class LLMAgent(AgentWithMemory):
provider_url=provider_url,
)
def _check_budget_before_invoke(self, where: str = "main") -> None:
"""Raise ``budget_exhausted`` if accumulated cost already at or exceeds limit.
Called *before* each main-agent ``ainvoke`` round inside the tool
loop so that zero/negative remaining budget fails fast (issue #76).
Does **not** apply to pruning passes those are silently skipped
via :meth:`_should_skip_pruning` instead.
"""
acc = current_accumulated_cost.get()
limit = current_max_cost_usd.get()
if limit is not None and acc >= limit:
logger.warning(
"Agent %s: budget exhausted before %s invoke ($%.6f >= $%.6f limit)",
self.name,
where,
acc,
limit,
)
raise ExecutionError(
f"Cost limit exceeded pre-flight: "
f"{acc:.6f} USD >= {limit} USD at {where} invoke",
kind="cost",
reason="budget_exhausted",
)
def _should_skip_pruning(self, tool_name: str) -> bool:
"""Return True when the budget is already exhausted and pruning should be skipped.
When the accumulated cost has already crossed the limit, pruning
passes (extra LLM calls) are suppressed and the raw tool output is
used instead (issue #76).
"""
acc = current_accumulated_cost.get()
limit = current_max_cost_usd.get()
if limit is not None and acc >= limit:
logger.info(
"Agent %s: budget exhausted, skipping pruning for %r "
"($%.6f >= $%.6f limit)",
self.name,
tool_name,
acc,
limit,
)
return True
return False
def _accumulate_cost_after_invoke(
self,
prompt_tokens: int,
completion_tokens: int,
where: str = "main",
model: str | None = None,
) -> None:
"""Compute USD cost from token counts and advance ``current_accumulated_cost``.
Uses the pricing table from ``current_pricing`` ContextVar (set by
``PureLangGraph`` before each node execution). The cost is computed
using the same formula as ``PureLangGraph._execute_from_node``:
cost = prompt_tokens / 1M * prompt_rate
+ completion_tokens / 1M * completion_rate
When *model* is provided (e.g. from a pruning pass that uses a
different model), that model is used for the pricing lookup instead
of ``self.model``. This ensures the in-node budget gate reflects
the actual spend even when ``pruning_model != self.model``, and works
correctly when the ``pruning_model != self.model`` guard is removed
in the future (Graa review PR #80).
Fails fast with ``missing_pricing_entry`` when a budget limit
(``current_max_cost_usd``) is set but pricing is unavailable for
the agent's provider/model — silently skipping cost tracking would
allow uncontrolled spending against the defined limit.
"""
pricing = current_pricing.get()
limit = current_max_cost_usd.get()
provider = self.provider
model = model or self.model
if not pricing:
if limit is not None:
raise ExecutionError(
f"Cost limit (${limit}) specified but pricing table is empty "
f"or missing for agent '{self.name}' ({provider}/{model}). "
f"Provide pricing data or remove max_cost_usd.",
kind="cost",
reason="missing_pricing_entry",
)
return
try:
_cost = _compute_cost(
pricing, provider, model, prompt_tokens, completion_tokens
)
except ExecutionError:
if limit is not None:
raise
return
_current = current_accumulated_cost.get()
current_accumulated_cost.set(_current + _cost)
logger.debug(
"Agent %s: accumulated $%.6f after %s invoke "
"(%d prompt + %d completion tokens)",
self.name,
_current + _cost,
where,
prompt_tokens,
completion_tokens,
)
def _get_model_context_window(self) -> int:
"""Return the model's advertised context window size in tokens.
@@ -891,11 +1006,13 @@ class LLMAgent(AgentWithMemory):
"call to complete your answer, do it now."
)
messages.append(HumanMessage(content=_synthesis_text))
self._check_budget_before_invoke("synthesis")
response = await self._retry_ainvoke(messages, **invoke_kwargs)
_any_invocation_made = True
_bp, _bc, _ = self._extract_token_counts(response)
_accumulated_prompt += _bp
_accumulated_completion += _bc
self._accumulate_cost_after_invoke(_bp, _bc, "synthesis")
_synth_tool_calls: list[dict[str, Any]] = (
getattr(response, "tool_calls", None) or []
)
@@ -972,6 +1089,7 @@ class LLMAgent(AgentWithMemory):
and _bq_output_prune is not False
and tool_name in self._pruning_tool_filter
and len(raw_out) > self._pruning_threshold
and not self._should_skip_pruning(tool_name)
):
try:
(
@@ -999,6 +1117,12 @@ class LLMAgent(AgentWithMemory):
raise
_accumulated_prompt += _bp_tok
_accumulated_completion += _bc_tok
self._accumulate_cost_after_invoke(
_bp_tok,
_bc_tok,
"pruning",
model=(self._pruning_model or self.model),
)
logger.info(
"Agent %s: pruning pass for %r: "
"%d -> %d chars",
@@ -1031,20 +1155,28 @@ class LLMAgent(AgentWithMemory):
tool_call_id=call_id,
)
)
self._check_budget_before_invoke("synthesis_followup")
response = await self._retry_ainvoke(messages)
_any_invocation_made = True
_bp3, _bc3, _ = self._extract_token_counts(response)
_accumulated_prompt += _bp3
_accumulated_completion += _bc3
self._accumulate_cost_after_invoke(
_bp3, _bc3, "synthesis_followup"
)
_budget_exhausted = True
break
# ── Pre-invoke budget check (issue #76) ─────────────────────
self._check_budget_before_invoke("main")
# ── Regular ainvoke ────────────────────────────────────────
response = await self._retry_ainvoke(messages, **invoke_kwargs)
_any_invocation_made = True
_mp, _mc, _ = self._extract_token_counts(response)
_accumulated_prompt += _mp
_accumulated_completion += _mc
self._accumulate_cost_after_invoke(_mp, _mc, "main")
response_tool_calls: list[dict[str, Any]] = (
getattr(response, "tool_calls", None) or []
@@ -1121,6 +1253,7 @@ class LLMAgent(AgentWithMemory):
and _output_prune is not False
and tool_name in self._pruning_tool_filter
and len(raw_tool_output) > self._pruning_threshold
and not self._should_skip_pruning(tool_name)
):
_task_context = list(messages)
try:
@@ -1149,6 +1282,12 @@ class LLMAgent(AgentWithMemory):
raise
_accumulated_prompt += _mlp_tok
_accumulated_completion += _mlc_tok
self._accumulate_cost_after_invoke(
_mlp_tok,
_mlc_tok,
"pruning",
model=(self._pruning_model or self.model),
)
logger.info(
"Agent %s: pruning pass for %r: %d -> %d chars",
self.name,
@@ -1200,11 +1339,13 @@ class LLMAgent(AgentWithMemory):
)
)
)
self._check_budget_before_invoke("stuck_model")
response = await self._retry_ainvoke(messages, tools=self._lc_tools)
_any_invocation_made = True
_sp, _sc, _ = self._extract_token_counts(response)
_accumulated_prompt += _sp
_accumulated_completion += _sc
self._accumulate_cost_after_invoke(_sp, _sc, "stuck_model")
# Allow *one* final tool-call round so the model can
# write files or perform other last-minute operations.
@@ -1288,11 +1429,15 @@ class LLMAgent(AgentWithMemory):
tool_call_id=call_id,
)
)
self._check_budget_before_invoke("stuck_model_followup")
response = await self._retry_ainvoke(messages)
_any_invocation_made = True
_sfp, _sfc, _ = self._extract_token_counts(response)
_accumulated_prompt += _sfp
_accumulated_completion += _sfc
self._accumulate_cost_after_invoke(
_sfp, _sfc, "stuck_model_followup"
)
# Log accumulated token usage across all rounds.
if _accumulated_prompt > 0 or _accumulated_completion > 0:
@@ -1531,10 +1676,12 @@ class LLMAgent(AgentWithMemory):
_completion_tokens = _loop_result.accumulated_completion
else:
# No tools: single plain ainvoke() — no tool dispatch or synthesis.
self._check_budget_before_invoke("main")
_no_tools_resp = await self._retry_ainvoke(messages)
_pt, _ct, _ = self._extract_token_counts(_no_tools_resp)
_prompt_tokens = _pt
_completion_tokens = _ct
self._accumulate_cost_after_invoke(_pt, _ct, "main")
response_text = str(_no_tools_resp.content)
# Capture token counts immediately after ainvoke() succeeds.
# Setting the sentinel variables here marks the "ainvoke succeeded"
@@ -1851,6 +1998,9 @@ class LLMAgent(AgentWithMemory):
return # generator done; finally block still runs
# ── No-tools path: real token-by-token astream() ─────────────
# Budget gate before streaming starts (issue #76).
self._check_budget_before_invoke("main")
# Wrap the initial stream connection (first chunk) in retry so
# transient LLM provider failures are retried at chunk granularity
# (ADR-2032 D-5 / issue #70 review). Once the first chunk
@@ -1952,6 +2102,9 @@ class LLMAgent(AgentWithMemory):
_captured_completion = _completion_tokens
self._last_token_usage = (_captured_prompt, _captured_completion)
last_token_usage_var.set((_captured_prompt, _captured_completion))
self._accumulate_cost_after_invoke(
_captured_prompt, _captured_completion, "stream"
)
# Update memory if enabled (spec §4.4.4).
# Mirrors process_message() memory block.
+15
View File
@@ -27,6 +27,21 @@ current_graph_name: contextvars.ContextVar[str] = contextvars.ContextVar(
"current_graph_name", default=""
)
# Per-task budget state carried from PureLangGraph into LLMAgent so
# in-node LLM invocations (main-agent rounds and pruning passes) can
# check whether the USD budget has already been exhausted (issue #76).
# Set by PureLangGraph before each node execution; read by LLMAgent's
# _execute_tool_loop and _run_pruning_pass.
current_accumulated_cost: contextvars.ContextVar[float] = contextvars.ContextVar(
"current_accumulated_cost", default=0.0
)
current_max_cost_usd: contextvars.ContextVar[float | None] = contextvars.ContextVar(
"current_max_cost_usd", default=None
)
current_pricing: contextvars.ContextVar[dict[str, Any] | None] = contextvars.ContextVar(
"current_pricing", default=None
)
_INITIAL_RETRY_DELAY: float = 0.5
+61
View File
@@ -61,6 +61,67 @@ class ExecutionError(CleverAgentsException):
self.reason = reason
def _compute_cost(
pricing: dict,
provider: str,
model: str,
prompt_tokens: int,
completion_tokens: int,
) -> float:
"""Compute USD cost from a pricing table.
Looks up *provider model {prompt, completion}* rates from the
pricing dict, validates them, and returns the cost in USD.
cost = prompt_tokens / 1M * prompt_rate + completion_tokens / 1M * completion_rate
Raises ``ExecutionError(kind="cost", reason="missing_pricing_entry")``
when the provider, model, or rate keys are missing or invalid.
This exists in ``exceptions.py`` because it is the one module imported
by all three cost-enforcement consumers (``pure_graph.py``,
``runtime_dispatch.py``, ``llm.py``) without circular-dependency risk.
"""
_provider_pricing = pricing.get(provider)
if not isinstance(_provider_pricing, dict):
raise ExecutionError(
f"Missing pricing entry for provider '{provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_model_pricing = _provider_pricing.get(model)
if not isinstance(_model_pricing, dict):
raise ExecutionError(
f"Missing pricing entry for model '{model}' under provider '{provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_prompt_rate_raw = _model_pricing.get("prompt")
_completion_rate_raw = _model_pricing.get("completion")
if _prompt_rate_raw is None or _completion_rate_raw is None:
raise ExecutionError(
f"Incomplete pricing entry for model '{model}' "
f"under provider '{provider}': missing 'prompt' "
f"or 'completion' rate key",
kind="cost",
reason="missing_pricing_entry",
)
try:
_prompt_rate = float(_prompt_rate_raw)
_completion_rate = float(_completion_rate_raw)
except (TypeError, ValueError) as err:
raise ExecutionError(
f"Invalid pricing rate for model '{model}' "
f"under provider '{provider}': {err}",
kind="cost",
reason="missing_pricing_entry",
) from err
return (
prompt_tokens / 1_000_000.0 * _prompt_rate
+ completion_tokens / 1_000_000.0 * _completion_rate
)
class ApplicationError(CleverAgentsException):
"""Exception raised for application-level errors."""
+374 -510
View File
@@ -8,17 +8,24 @@ allowing it to work properly in run mode without timeouts.
from __future__ import annotations
import asyncio
import contextvars
import copy
import logging
from collections import defaultdict, deque
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Deque, Final, List, Optional, Set
from cleveractors.agents.base import Agent
from cleveractors.agents.retry import (
current_accumulated_cost,
current_max_cost_usd,
current_pricing,
)
from cleveractors.context_manager import ContextManager
from cleveractors.core.exceptions import ExecutionError
from cleveractors.core.exceptions import ExecutionError, _compute_cost
from cleveractors.langgraph.dynamic_router import DynamicRouterNode
from cleveractors.langgraph.nodes import Edge, Node, NodeConfig, NodeType
from cleveractors.langgraph.state import GraphState, StateManager
@@ -535,6 +542,152 @@ class PureLangGraph:
global_context.clear()
global_context.update(final_metadata)
def _get_max_cost_usd_limit(self) -> float | None:
"""Parse and return the ``max_cost_usd`` limit, or ``None`` if absent.
Raises ``ExecutionError(kind='cost', reason='missing_pricing_entry')``
when ``max_cost_usd`` is specified in limits but pricing is empty
a limit without pricing is a configuration error.
Returns ``None`` (no-op) only when the limit key is absent,
the value is a bool (nonsensical), or the value cannot be coerced
to float.
"""
_raw = self._limits.get("max_cost_usd")
if _raw is None:
return None
if not self._pricing:
raise ExecutionError(
"max_cost_usd limit specified but pricing table is empty. "
"Provide pricing data or remove max_cost_usd.",
kind="cost",
reason="missing_pricing_entry",
)
if isinstance(_raw, bool):
return None
try:
return float(_raw)
except (TypeError, ValueError):
return None
def _set_budget_context_for_node(
self,
) -> tuple[
contextvars.Token[float] | None,
contextvars.Token[float | None] | None,
contextvars.Token[dict[str, Any] | None] | None,
]:
"""Set budget ContextVars for in-node budget checks (issue #76).
Must be called before each node execution. Returns a tuple of
``(cost_token, max_token, pricing_token)`` that the caller must
reset in a ``finally`` block after the node completes.
When ``_pricing`` is empty or ``max_cost_usd`` is absent, the
ContextVars are set to zero / ``None`` respectively so that
in-node checks are no-ops.
"""
_cost = self._accumulated_cost
_max = self._get_max_cost_usd_limit()
_cost_token = current_accumulated_cost.set(_cost)
_max_token: contextvars.Token[float | None] | None = None
if _max is not None:
_max_token = current_max_cost_usd.set(_max)
_pricing_token: contextvars.Token[dict[str, Any] | None] | None = None
if self._pricing:
_pricing_token = current_pricing.set(self._pricing)
return (_cost_token, _max_token, _pricing_token)
@staticmethod
def _reset_budget_context(
cost_token: contextvars.Token[float] | None,
max_token: contextvars.Token[float | None] | None,
pricing_token: contextvars.Token[dict[str, Any] | None] | None,
) -> None:
"""Reset budget ContextVars after node execution."""
if cost_token is not None:
current_accumulated_cost.reset(cost_token)
if max_token is not None:
current_max_cost_usd.reset(max_token)
if pricing_token is not None:
current_pricing.reset(pricing_token)
def _check_budget_pre_flight(self, node_name: str) -> None:
"""Raise ``budget_exhausted`` if accumulated cost already at or exceeds limit.
Called *before* a node executes so that a zero/negative remaining
budget fails immediately without entering the node or its retry
mechanism (issue #76). No-op when pricing is empty or no limit is set.
Uses ``>=`` (not ``>``) intentionally: exactly-at-limit blocks the
*next* node from entering but does not abort the current node's
post-node check (which uses ``>``). This guarantees that a node
that exactly hits the limit completes normally; only the next node
sees the pre-flight block (Graa review PR #80).
"""
_max = self._get_max_cost_usd_limit()
if _max is not None and self._accumulated_cost >= _max:
self.logger.warning(
"Budget pre-flight: accumulated $%.6f >= limit $%.6f before node %s",
self._accumulated_cost,
_max,
node_name,
)
raise ExecutionError(
f"Cost limit exceeded pre-flight: "
f"{self._accumulated_cost:.6f} USD >= {_max} USD"
f" before node '{node_name}'",
kind="cost",
reason="budget_exhausted",
)
@asynccontextmanager
async def _node_budget_context(self, node_name: str) -> AsyncGenerator[None, None]:
"""Context manager wrapping the budget pre-flight, ContextVar setup,
and finally-reset that was copy-pasted across all four execution
branches. Using ``async with self._node_budget_context(node_name):``
makes the reset un-forgettable.
"""
self._check_budget_pre_flight(node_name)
_cost_tok, _max_tok, _pricing_tok = self._set_budget_context_for_node()
try:
yield
finally:
self._reset_budget_context(_cost_tok, _max_tok, _pricing_tok)
def _check_post_node_cost_limit(self) -> None:
"""Check accumulated cost against ``max_cost_usd`` after a node completes.
Raises ``ExecutionError(kind='cost', reason='budget_exhausted')`` when
the limit is exceeded. Uses ``>`` (not ``>=``) intentionally: a node
that exactly hits the limit completes normally; only the *next* node's
pre-flight check (``>=``) blocks it.
This centralises the bool-guard + float() coercion that was previously
inlined in all three post-node cost blocks.
"""
_raw = self._limits.get("max_cost_usd")
if _raw is None:
return
if isinstance(_raw, bool):
raise ExecutionError(
f"Invalid max_cost_usd value {_raw!r}: bool is not a valid cost limit",
kind="cost",
)
try:
_limit = float(_raw)
except (TypeError, ValueError) as _err:
raise ExecutionError(
f"Invalid max_cost_usd value {_raw!r}: {_err}",
kind="cost",
) from _err
if self._accumulated_cost > _limit:
raise ExecutionError(
f"Cost limit exceeded: {self._accumulated_cost:.6f} USD > {_raw} USD",
kind="cost",
reason="budget_exhausted",
)
async def _execute_from_node(
self, node_name: str, message: Any, depth: int = 0
) -> Any:
@@ -790,204 +943,88 @@ class PureLangGraph:
# Pass graph name for retry error reporting (ADR-2032 D-7).
state.metadata["graph_name"] = self.name
# Execute node with current state
try:
self.logger.debug(f"About to execute node.execute() for {node_name}")
result = await node.execute(state)
result_str = (
str(result)[:100] if isinstance(result, str) else str(result)[:100]
)
self.logger.debug(f"Node {node_name} returned: {result_str}")
# ── Budget pre-flight + ContextVar setup (issue #76) ──────────────
async with self._node_budget_context(node_name):
try:
self.logger.debug(f"About to execute node.execute() for {node_name}")
result = await node.execute(state)
result_str = (
str(result)[:100] if isinstance(result, str) else str(result)[:100]
)
self.logger.debug(f"Node {node_name} returned: {result_str}")
# Update state with node results
if isinstance(result, dict):
# Extract token usage BEFORE passing to update_state so the
# internal side-channel key is never written into GraphState
# (m5 of review for issue #14). Popping here means a custom
# state_class that accidentally defines _node_token_usage as a
# real field will not receive the internal dict.
_tok_info = result.pop("_node_token_usage", None)
# Update state with node results
if isinstance(result, dict):
# Extract token usage BEFORE passing to update_state
_tok_info = result.pop("_node_token_usage", None)
self.state_manager.update_state(result, node_id=node_name)
# Extract the actual message content from the result
# Agents return messages in a 'messages' array with 'content' field
if "messages" in result and result["messages"]:
# Get the last message's content
last_message = result["messages"][-1]
if isinstance(last_message, dict) and "content" in last_message:
output_message = last_message["content"]
self.state_manager.update_state(result, node_id=node_name)
if "messages" in result and result["messages"]:
last_message = result["messages"][-1]
if isinstance(last_message, dict) and "content" in last_message:
output_message = last_message["content"]
else:
output_message = str(last_message)
else:
output_message = str(last_message)
output_message = result.get(
"content", result.get("output", message)
)
if _tok_info is not None and not isinstance(_tok_info, dict):
self.logger.warning(
"Node %s returned non-dict _node_token_usage (%r); "
"token accounting skipped",
node_name,
type(_tok_info).__name__,
)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
if self._pricing:
_node_cost = _compute_cost(
self._pricing,
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
self._accumulated_cost += _node_cost
self._check_post_node_cost_limit()
else:
# Fallback to direct content/output fields
output_message = result.get(
"content", result.get("output", message)
)
output_message = result
# Collect per-node token usage if this agent node reported it
# (AC4, issue #14). Node._execute_agent() sets
# _node_token_usage in the result dict for LLM agent nodes.
# m-1 fix: log a warning when _node_token_usage is present but
# not a dict so that malformed values surface in production logs
# rather than silently disabling token accounting and cost
# enforcement.
if _tok_info is not None and not isinstance(_tok_info, dict):
self.logger.warning(
"Node %s returned non-dict _node_token_usage (%r); "
"token accounting skipped",
node_name,
type(_tok_info).__name__,
)
if isinstance(_tok_info, dict):
# m-2 fix: compute _pt and _ct once at the top of the
# isinstance block and reuse them in both _node_usages and
# the cost block, eliminating the double _safe_token_int()
# call that doubled log noise for malformed inputs.
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
self.state_manager.state.metadata["last_output"] = output_message
# Cost accumulation and enforcement (ADR-2029, issue #15).
# Only when the router supplied a non-empty pricing table; an
# empty dict means cost tracking was not requested and is skipped
# entirely. When the table is non-empty, a missing provider or
# model entry is a hard error — proceeding with an assumed zero
# price is forbidden because it would enable unbounded spend.
if self._pricing:
_provider = str(_tok_info.get("provider", "unknown"))
_model = str(_tok_info.get("model", "unknown"))
# _pt and _ct already computed above (m-2 fix)
_provider_pricing = self._pricing.get(_provider)
if _provider_pricing is None or not isinstance(
_provider_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for provider '{_provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_model_pricing = _provider_pricing.get(_model)
if _model_pricing is None or not isinstance(
_model_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for model '{_model}' "
f"under provider '{_provider}'",
kind="cost",
reason="missing_pricing_entry",
output_str = str(output_message) if output_message else ""
for prefix in ["GOTO_", "ROUTE_"]:
if prefix in output_str:
cmd_part = output_str.split(":", 1)[0]
target = cmd_part.replace(prefix, "").strip().lower()
if target:
self.state_manager.state.metadata["next_node"] = target
self.logger.debug(
f"Parsed routing command from {node_name}: "
f"next_node={target}"
)
break
# Validate that both rate keys exist and are numeric
# (ADR-2029: "proceeding with a missing or zero price is
# forbidden"). A missing key or a non-numeric value is
# treated the same as a missing pricing entry — the
# library must not silently fall back to $0/token.
_prompt_rate_raw = _model_pricing.get("prompt")
_completion_rate_raw = _model_pricing.get("completion")
if _prompt_rate_raw is None or _completion_rate_raw is None:
raise ExecutionError(
f"Incomplete pricing entry for model '{_model}' "
f"under provider '{_provider}': missing 'prompt' "
f"or 'completion' rate key",
kind="cost",
reason="missing_pricing_entry",
)
try:
# Rates are USD per million tokens (ADR-2029 example:
# gpt-4.1-mini prompt=$0.15/1M, completion=$0.60/1M).
_prompt_rate = float(_prompt_rate_raw)
_completion_rate = float(_completion_rate_raw)
except (TypeError, ValueError) as _rate_err:
raise ExecutionError(
f"Invalid pricing rate for model '{_model}' "
f"under provider '{_provider}': {_rate_err}",
kind="cost",
reason="missing_pricing_entry",
) from _rate_err
_node_cost = (
_pt / 1_000_000.0 * _prompt_rate
+ _ct / 1_000_000.0 * _completion_rate
)
self._accumulated_cost += _node_cost
# C-1 fix (ADR-2029, issue #15): coerce max_cost_usd
# with a narrow try/except BEFORE the broad
# "except Exception" handler so that a malformed value
# (e.g. a non-numeric string) raises ExecutionError
# immediately rather than being silently swallowed by
# the broad handler and causing the graph to return the
# input string as if execution succeeded.
_max_cost = self._limits.get("max_cost_usd")
if _max_cost is not None:
# Bool guard: float(True)==1.0 / float(False)==0.0
# would silently set a nonsensical budget. Reject
# bools before the float() conversion, matching the
# pattern used for max_depth/max_model_calls/max_tool_calls.
if isinstance(_max_cost, bool):
raise ExecutionError(
f"Invalid max_cost_usd value {_max_cost!r}: "
"bool is not a valid cost limit",
kind="cost",
)
try:
_max_cost_float = float(_max_cost)
except (TypeError, ValueError) as _cost_err:
# A malformed limit value (e.g. a non-numeric string)
# is a router configuration error, not a missing pricing
# entry. Use reason="" (the documented default for
# non-cost-specific errors per ADR-2029) so router logs
# and alerts correctly identify the root cause.
raise ExecutionError(
f"Invalid max_cost_usd value "
f"{_max_cost!r}: {_cost_err}",
kind="cost",
) from _cost_err
if self._accumulated_cost > _max_cost_float:
raise ExecutionError(
f"Cost limit exceeded: "
f"{self._accumulated_cost:.6f} USD "
f"> {_max_cost} USD",
kind="cost",
reason="budget_exhausted",
)
else:
output_message = result
# Store the output message in state for edge conditions to use
self.state_manager.state.metadata["last_output"] = output_message
# Parse CleverAgents v2.0 routing commands from agent outputs
# so edge conditions can route to the correct next node.
output_str = str(output_message) if output_message else ""
for prefix in ["GOTO_", "ROUTE_"]:
if prefix in output_str:
# Extract target node from "GOTO_NODE:rest_of_message"
cmd_part = output_str.split(":", 1)[0]
target = cmd_part.replace(prefix, "").strip().lower()
if target:
self.state_manager.state.metadata["next_node"] = target
self.logger.debug(
f"Parsed routing command from {node_name}: next_node={target}"
)
break
except ExecutionError:
# Limit-enforcement errors (depth, model_calls, tool_calls, timeout,
# cost) must propagate without suppression so the router can map
# them to the correct HTTP status code (ADR-2029, issue #15).
raise
except Exception as e:
self.logger.error(f"Error executing node {node_name}: {e}", exc_info=True)
output_message = message
except ExecutionError:
raise
except Exception as e:
self.logger.error(
f"Error executing node {node_name}: {e}", exc_info=True
)
output_message = message
# Determine next nodes
self.logger.debug(
@@ -1577,50 +1614,62 @@ class PureLangGraph:
# they are also yielded immediately (fast path) or held back
# until the stream completes (slow path).
_collected_tokens: List[str] = []
try:
async for token in node._stream_agent(state):
_collected_tokens.append(token) # always accumulate
if _all_edges_unconditional:
yield token # yield immediately for fast path
full_response = "".join(_collected_tokens) # always correct
# ── Budget pre-flight + ContextVar setup (issue #76) ───────────
async with self._node_budget_context(node_name):
try:
async for token in node._stream_agent(state):
_collected_tokens.append(token)
if _all_edges_unconditional:
yield token
full_response = "".join(_collected_tokens)
# Update state with the full response only on success.
# Mirrors _execute_from_node which does NOT update state.messages
# on failure — it only sets state.metadata["last_output"] for
# routing purposes. Updating state on failure would persist the
# user's input as an assistant message, corrupting routing
# decisions and memory for subsequent turns (Major #2 fix).
agent_result: dict[str, Any] = {
"messages": [
{
"role": "assistant",
"content": full_response,
"node": node_name,
"agent": node.config.agent,
}
],
"current_node": node_name,
"metadata": {"last_agent_node": node_name},
}
self.state_manager.update_state(agent_result, node_id=node_name)
except ExecutionError:
raise
except Exception as e: # pylint: disable=broad-exception-caught
# M3 fix: re-raise non-ExecutionError exceptions so callers
# can distinguish "stream completed normally" from "stream
# failed mid-way" and map them to HTTP 5xx/4xx. Wrapping
# in ExecutionError mirrors the non-streaming path's
# exception handling and preserves the original cause.
# The exception type is logged at ERROR level; the message
# is deliberately sanitised (no {e} interpolation) to avoid
# leaking sensitive provider details (API key fragments,
# internal URLs) — use `from e` to preserve the cause chain.
self.logger.error(
"Agent node '%s' streaming failed: %s",
node_name,
type(e).__name__,
)
raise ExecutionError("Agent node streaming failed") from e
agent_result: dict[str, Any] = {
"messages": [
{
"role": "assistant",
"content": full_response,
"node": node_name,
"agent": node.config.agent,
}
],
"current_node": node_name,
"metadata": {"last_agent_node": node_name},
}
self.state_manager.update_state(agent_result, node_id=node_name)
_tok_info = getattr(node, "_last_stream_usage", None)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
if self._pricing:
_stream_node_cost = _compute_cost(
self._pricing,
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
self._accumulated_cost += _stream_node_cost
self._check_post_node_cost_limit()
except ExecutionError:
raise
except Exception as e:
self.logger.error(
"Agent node '%s' streaming failed: %s",
node_name,
type(e).__name__,
)
raise ExecutionError("Agent node streaming failed") from e
# Update last_output for routing (mirrors non-streaming path).
# Note: this is only reached on the success path; the except blocks
@@ -1651,105 +1700,6 @@ class PureLangGraph:
)
break
# Collect per-node token usage (stored in node._last_stream_usage)
_tok_info = getattr(node, "_last_stream_usage", None)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
# M3 fix: Cost enforcement in streaming path.
# Mirrors _execute_from_node cost block so that max_cost_usd
# is honoured for streaming requests (the most expensive use case).
if self._pricing:
_stream_provider = str(_tok_info.get("provider", "unknown"))
_stream_model = str(_tok_info.get("model", "unknown"))
_stream_provider_pricing = self._pricing.get(_stream_provider)
if _stream_provider_pricing is None or not isinstance(
_stream_provider_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for provider '{_stream_provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_stream_model_pricing = _stream_provider_pricing.get(
_stream_model
)
if _stream_model_pricing is None or not isinstance(
_stream_model_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for model '{_stream_model}' "
f"under provider '{_stream_provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_stream_prompt_rate_raw = _stream_model_pricing.get("prompt")
_stream_completion_rate_raw = _stream_model_pricing.get(
"completion"
)
if (
_stream_prompt_rate_raw is None
or _stream_completion_rate_raw is None
):
raise ExecutionError(
f"Incomplete pricing entry for model '{_stream_model}' "
f"under provider '{_stream_provider}': missing 'prompt' "
f"or 'completion' rate key",
kind="cost",
reason="missing_pricing_entry",
)
try:
_stream_prompt_rate = float(_stream_prompt_rate_raw)
_stream_completion_rate = float(_stream_completion_rate_raw)
except (TypeError, ValueError) as _stream_rate_err:
raise ExecutionError(
f"Invalid pricing rate for model '{_stream_model}' "
f"under provider '{_stream_provider}': "
f"{_stream_rate_err}",
kind="cost",
reason="missing_pricing_entry",
) from _stream_rate_err
_stream_node_cost = (
_pt / 1_000_000.0 * _stream_prompt_rate
+ _ct / 1_000_000.0 * _stream_completion_rate
)
self._accumulated_cost += _stream_node_cost
_stream_max_cost = self._limits.get("max_cost_usd")
if _stream_max_cost is not None:
if isinstance(_stream_max_cost, bool):
raise ExecutionError(
f"Invalid max_cost_usd value {_stream_max_cost!r}: "
"bool is not a valid cost limit",
kind="cost",
)
try:
_stream_max_cost_float = float(_stream_max_cost)
except (TypeError, ValueError) as _stream_cost_err:
raise ExecutionError(
f"Invalid max_cost_usd value "
f"{_stream_max_cost!r}: {_stream_cost_err}",
kind="cost",
) from _stream_cost_err
if self._accumulated_cost > _stream_max_cost_float:
raise ExecutionError(
f"Cost limit exceeded: "
f"{self._accumulated_cost:.6f} USD "
f"> {_stream_max_cost} USD",
kind="cost",
reason="budget_exhausted",
)
# Confirmed terminal: _statically_terminal=True guarantees that
# all static successors are END/end sentinels, so
# _get_next_nodes() can only return those same targets (or a
@@ -1768,149 +1718,61 @@ class PureLangGraph:
else:
# Intermediate AGENT node: use ainvoke() (node.execute) to avoid
# buffering tokens only to discard them (AC2).
try:
result = await node.execute(state)
if isinstance(result, dict):
_tok_info = result.pop("_node_token_usage", None)
self.state_manager.update_state(result, node_id=node_name)
if "messages" in result and result["messages"]:
last_msg = result["messages"][-1]
if isinstance(last_msg, dict) and "content" in last_msg:
full_response = last_msg["content"]
# ── Budget pre-flight + ContextVar setup (issue #76) ───────────
async with self._node_budget_context(node_name):
try:
result = await node.execute(state)
if isinstance(result, dict):
_tok_info = result.pop("_node_token_usage", None)
self.state_manager.update_state(result, node_id=node_name)
if "messages" in result and result["messages"]:
last_msg = result["messages"][-1]
if isinstance(last_msg, dict) and "content" in last_msg:
full_response = last_msg["content"]
else:
full_response = str(last_msg)
else:
full_response = str(last_msg)
full_response = result.get(
"content", result.get("output", message)
)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(
_tok_info.get("completion_tokens", 0)
)
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
if self._pricing:
_int_node_cost = _compute_cost(
self._pricing,
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
self._accumulated_cost += _int_node_cost
self._check_post_node_cost_limit()
else:
full_response = result.get(
"content", result.get("output", message)
full_response = (
str(result) if result is not None else message
)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
# M3 fix (intermediate branch): Cost enforcement in
# streaming path for intermediate AGENT nodes.
# Mirrors the terminal AGENT branch cost block so that
# max_cost_usd is honoured for every AGENT node in a
# multi-AGENT graph, not just the terminal one.
if self._pricing:
_int_provider = str(
_tok_info.get("provider", "unknown")
)
_int_model = str(_tok_info.get("model", "unknown"))
_int_provider_pricing = self._pricing.get(_int_provider)
if _int_provider_pricing is None or not isinstance(
_int_provider_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for provider "
f"'{_int_provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_int_model_pricing = _int_provider_pricing.get(
_int_model
)
if _int_model_pricing is None or not isinstance(
_int_model_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for model "
f"'{_int_model}' under provider "
f"'{_int_provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_int_prompt_rate_raw = _int_model_pricing.get("prompt")
_int_completion_rate_raw = _int_model_pricing.get(
"completion"
)
if (
_int_prompt_rate_raw is None
or _int_completion_rate_raw is None
):
raise ExecutionError(
f"Incomplete pricing entry for model "
f"'{_int_model}' under provider "
f"'{_int_provider}': missing 'prompt' "
f"or 'completion' rate key",
kind="cost",
reason="missing_pricing_entry",
)
try:
_int_prompt_rate = float(_int_prompt_rate_raw)
_int_completion_rate = float(
_int_completion_rate_raw
)
except (TypeError, ValueError) as _int_rate_err:
raise ExecutionError(
f"Invalid pricing rate for model "
f"'{_int_model}' under provider "
f"'{_int_provider}': {_int_rate_err}",
kind="cost",
reason="missing_pricing_entry",
) from _int_rate_err
_int_node_cost = (
_pt / 1_000_000.0 * _int_prompt_rate
+ _ct / 1_000_000.0 * _int_completion_rate
)
self._accumulated_cost += _int_node_cost
_int_max_cost = self._limits.get("max_cost_usd")
if _int_max_cost is not None:
if isinstance(_int_max_cost, bool):
raise ExecutionError(
f"Invalid max_cost_usd value "
f"{_int_max_cost!r}: bool is not a "
"valid cost limit",
kind="cost",
)
try:
_int_max_cost_float = float(_int_max_cost)
except (
TypeError,
ValueError,
) as _int_cost_err:
raise ExecutionError(
f"Invalid max_cost_usd value "
f"{_int_max_cost!r}: {_int_cost_err}",
kind="cost",
) from _int_cost_err
if self._accumulated_cost > _int_max_cost_float:
raise ExecutionError(
f"Cost limit exceeded: "
f"{self._accumulated_cost:.6f} USD "
f"> {_int_max_cost} USD",
kind="cost",
reason="budget_exhausted",
)
else:
full_response = str(result) if result is not None else message
except ExecutionError:
raise
except Exception as e: # pylint: disable=broad-exception-caught
# M3 fix: re-raise non-ExecutionError exceptions so callers
# can distinguish "node completed normally" from "node
# failed" and map them to HTTP 5xx/4xx. Wrapping in
# ExecutionError mirrors the non-streaming path's behavior.
# The exception type is logged at ERROR level; the message
# is deliberately sanitised (no {e} interpolation) to avoid
# leaking sensitive provider details — use `from e` to
# preserve the cause chain.
self.logger.error(
"Intermediate agent node '%s' ainvoke failed: %s",
node_name,
type(e).__name__,
)
raise ExecutionError("Intermediate agent node failed") from e
except ExecutionError:
raise
except Exception as e:
self.logger.error(
"Intermediate agent node '%s' ainvoke failed: %s",
node_name,
type(e).__name__,
)
raise ExecutionError("Intermediate agent node failed") from e
self.state_manager.state.metadata["last_output"] = full_response
@@ -2023,45 +1885,47 @@ class PureLangGraph:
else:
# Non-AGENT node: run with node.execute() (ainvoke path)
output_message: Any = message
try:
result = await node.execute(state)
if isinstance(result, dict):
_tok_info = result.pop("_node_token_usage", None)
self.state_manager.update_state(result, node_id=node_name)
if "messages" in result and result["messages"]:
last_msg = result["messages"][-1]
if isinstance(last_msg, dict) and "content" in last_msg:
output_message = last_msg["content"]
# ── Budget pre-flight + ContextVar setup (issue #76) ───────────
async with self._node_budget_context(node_name):
output_message: Any = message
try:
result = await node.execute(state)
if isinstance(result, dict):
_tok_info = result.pop("_node_token_usage", None)
self.state_manager.update_state(result, node_id=node_name)
if "messages" in result and result["messages"]:
last_msg = result["messages"][-1]
if isinstance(last_msg, dict) and "content" in last_msg:
output_message = last_msg["content"]
else:
output_message = str(last_msg)
else:
output_message = str(last_msg)
else:
output_message = result.get(
"content", result.get("output", message)
)
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
output_message = result.get(
"content", result.get("output", message)
)
)
else:
output_message = result
except ExecutionError:
raise
except Exception as e: # pylint: disable=broad-exception-caught
self.logger.error(
"Non-agent node '%s' execution failed: %s",
node_name,
type(e).__name__,
)
output_message = message
if isinstance(_tok_info, dict):
_pt = _safe_token_int(_tok_info.get("prompt_tokens", 0))
_ct = _safe_token_int(_tok_info.get("completion_tokens", 0))
self._node_usages.append(
(
str(_tok_info.get("node_id", node_name)),
str(_tok_info.get("provider", "unknown")),
str(_tok_info.get("model", "unknown")),
_pt,
_ct,
)
)
else:
output_message = result
except ExecutionError:
raise
except Exception as e:
self.logger.error(
"Non-agent node '%s' execution failed: %s",
node_name,
type(e).__name__,
)
output_message = message
self.state_manager.state.metadata["last_output"] = output_message
+19 -42
View File
@@ -37,6 +37,7 @@ from cleveractors.core.exceptions import (
AgentCreationError,
ConfigurationError,
ExecutionError,
_compute_cost,
)
from cleveractors.langgraph.nodes import Edge, NodeConfig, NodeType
from cleveractors.langgraph.pure_graph import PureGraphConfig, PureLangGraph
@@ -948,49 +949,25 @@ async def _execute_llm_stream(
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
)
if executor.pricing:
_llm_provider_pricing = executor.pricing.get(provider)
if _llm_provider_pricing is None or not isinstance(
_llm_provider_pricing, dict
):
raise ExecutionError(
f"Missing pricing entry for provider '{provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_llm_model_pricing = _llm_provider_pricing.get(model)
if _llm_model_pricing is None or not isinstance(_llm_model_pricing, dict):
raise ExecutionError(
f"Missing pricing entry for model '{model}' "
f"under provider '{provider}'",
kind="cost",
reason="missing_pricing_entry",
)
_llm_prompt_rate_raw = _llm_model_pricing.get("prompt")
_llm_completion_rate_raw = _llm_model_pricing.get("completion")
if _llm_prompt_rate_raw is None or _llm_completion_rate_raw is None:
raise ExecutionError(
f"Incomplete pricing entry for model '{model}' "
f"under provider '{provider}': missing 'prompt' "
f"or 'completion' rate key",
kind="cost",
reason="missing_pricing_entry",
)
try:
_llm_prompt_rate = float(_llm_prompt_rate_raw)
_llm_completion_rate = float(_llm_completion_rate_raw)
except (TypeError, ValueError) as _llm_rate_err:
raise ExecutionError(
f"Invalid pricing rate for model '{model}' "
f"under provider '{provider}': {_llm_rate_err}",
kind="cost",
reason="missing_pricing_entry",
) from _llm_rate_err
_llm_node_cost = (
prompt_tokens / 1_000_000.0 * _llm_prompt_rate
+ completion_tokens / 1_000_000.0 * _llm_completion_rate
# If max_cost_usd is set but pricing is empty, hard-fail at once
# (consistent with PureLangGraph._get_max_cost_usd_limit).
_llm_max_cost = executor.limits.get("max_cost_usd")
if _llm_max_cost is not None and not executor.pricing:
raise ExecutionError(
"max_cost_usd limit specified but pricing table is empty. "
"Provide pricing data or remove max_cost_usd.",
kind="cost",
reason="missing_pricing_entry",
)
if executor.pricing:
_llm_node_cost = _compute_cost(
executor.pricing,
provider,
model,
prompt_tokens,
completion_tokens,
)
_llm_max_cost = executor.limits.get("max_cost_usd")
if _llm_max_cost is not None:
if isinstance(_llm_max_cost, bool):
raise ExecutionError(