feat(ActorResult): implement ActorResult and NodeUsage types; capture per-node token counts from LangChain responses #41

Merged

hurui200320 merged 1 commit from feature/actor-result into master 2026-06-11 03:25:35 +00:00

Conversation 1 Commits 1 Files changed 29 +3356 -940

hurui200320 commented 2026-06-10 05:49:22 +00:00

Member

Copy link

Summary

Implements cleveractors-core issue #14 — the full AC1–AC7 acceptance criteria for ActorResult, NodeUsage, and real LangChain token counting.

Closes #14

Motivation

The CleverThis router consumes Executor.execute() to run actors. Before this PR, the router received ActorResult objects whose prompt_tokens and completion_tokens fields were estimated (via tiktoken or a 4-chars/token heuristic) rather than read from LangChain's actual response metadata. This made billing data inaccurate. The router also could not find the canonical ActorResult/NodeUsage types at the spec-mandated location (cleveractors.result).

Changes

New module: src/cleveractors/result.py (AC1)

Defines NodeUsage and ActorResult dataclasses at the spec-mandated location. Also defines MAX_REASONABLE_TOKENS = 10_000_000 as the single authoritative constant (previously duplicated in three modules). runtime.py now imports and re-exports both types for backward compatibility. __init__.py updated to import from result instead of runtime. Uses lowercase list[NodeUsage] (PEP 585) throughout. nodes is a required positional field (no default) so the contract that nodes is always non-empty is statically enforced. state field uses dict[str, Any] | None (PEP 585, consistent with rest of codebase). __all__ declaration added.

LLMAgent — real token extraction + race-condition fix + billing integrity (AC2, AC3, M1, M2, m3)

process_message() now:

• Uses a module-level contextvars.ContextVar (last_token_usage_var) to store per-task token counts. Each asyncio.Task inherits a copy of the context, so parallel graph branches that share the same LLMAgent instance each see their own token counts — eliminating the race condition.
• Resets _last_token_usage = (0, 0) and last_token_usage_var.set((0, 0)) as the first statement inside the try: block.
• Captures token counts into sentinel variables (_captured_prompt, _captured_completion) immediately after ainvoke() succeeds, before the memory-update block. This marks the "ainvoke succeeded" boundary.
• Billing integrity fix: If a post-ainvoke() step (e.g. update_memory()) raises an exception, the except handlers check the sentinel variables. If _captured_prompt is not None, ainvoke() already succeeded and the LLM provider has already charged the user — the captured counts are preserved rather than zeroed. Only pre-ainvoke() failures (where no tokens were consumed) reset to (0, 0).
• Added a prominent docstring .. note:: to process_message() documenting this billing integrity guarantee.

Node._execute_agent() — race condition fix, agent-type-aware fallback (M1 cycle-9 fix)

• Resets last_token_usage_var to (0, 0) at the start of the try: block, before calling agent.process_message().
• Agent-type-aware fallback logic (cycle 9): For LLMAgent, the ContextVar is authoritative — (0, 0) means zero tokens, not "unset". The fallback to _last_token_usage instance attribute is never used for LLMAgent instances, preventing the race where a sibling parallel branch overwrites the instance attribute. For non-LLMAgent agents (ToolAgent, ChainAgent, etc.), the ContextVar is always (0, 0) since process_message() does not touch it; these agents fall back to the instance attribute as before.
• Fixed _node_token_usage type annotation from Optional[dict[str, Any]] to dict[str, Any] | None (PEP 604, consistent with rest of codebase).

runtime_dispatch.py — same agent-type-aware fallback in _execute_llm (M1 cycle-9 fix)

• Applied the same agent-type-aware fallback pattern to _execute_llm for consistency with Node._execute_agent().
• Renamed cryptic exception variable _nu_err → _node_usage_err in _execute_graph.
• Multi-actor placeholder normalisation: Added _normalize_node_id() helper and _GRAPH_PLACEHOLDER_RE module-level regex.
• Renamed loop variable nu → node_usage in _execute_multi_actor for readability.
• Moved import re to the top-level import block.

pure_graph.py — import organisation

• Moved from cleveractors.result import MAX_REASONABLE_TOKENS as _MAX_REASONABLE_TOKENS to the top-level import block.

features/mocks/actor_result_helpers.py — new mock helpers module (cycle-9 fix)

• Extracted _make_llm_actor_config and _make_mock_chat_model from the step file into features/mocks/actor_result_helpers.py (renamed to make_llm_actor_config and make_mock_chat_model), per project convention that mock factory functions belong in features/mocks/.

BDD tests updated (M1, M2, m4, m5, m6, cycle-9 fixes)

• Strengthened parallel isolation assertions (cycle 9): The "ContextVar isolates token counts" scenario now uses exact-value assertions (t1 == (100, 50) and t2 == (200, 80)) instead of weak != (0, 0) and != each other checks. A buggy swapped-mapping implementation would now fail.
• Removed bare except Exception in wrong-arity real-path test (cycle 9): The artc_wrong_arity_real_path_test branch no longer catches all exceptions and fabricates a fake ActorResult. If executor.execute() raises unexpectedly, the test now fails loudly.
• Deleted dead step definition step_artc_graph_executor_wrong_arity_tuple (cycle 9): This ~44-line step was never referenced by any scenario in the feature file.
• New scenario: ContextVar parallel-branch isolation (M2): A single LLMAgent instance is called concurrently via asyncio.gather() with two different mocked usage_metadata responses; each task asserts its last_token_usage_var.get() reflects its own call's counts.
• Updated scenario: Post-ainvoke exception preserves token counts (m4, billing integrity fix): An LLMAgent with memory_enabled=True succeeds at ainvoke() but fails at update_memory(); asserts _last_token_usage == (42, 17) after the exception (previously incorrectly asserted (0, 0)). This verifies that billing data is not lost when memory persistence fails.
• New scenarios: large-value pass-through (m5): _safe_node_token_int(20_000_000) and _safe_token_int(20_000_000) both return 20_000_000 unchanged.
• Rewritten wrong-arity scenario (m6): The first wrong-arity scenario now exercises the real Node._execute_agent() code path via AgentFactory.create_agent mock.

CHANGELOG.md — stale reference removed

• Removed stale _usage_log reference from the "Mutable state hygiene" entry (this field never existed in this PR's scope). The entry now accurately describes the actual new mechanism: _last_token_usage and last_token_usage_var are reset at the start of every process_message() call; PureLangGraph._node_usages is reset at the start of every execute() call; post-ainvoke() exceptions preserve captured token counts for billing integrity.

Architecture Note

AC3 says "process_message() returns token counts alongside the response string." Changing the Agent abstract base class return type from str to tuple[str, int, int] would require updating all four concrete agent implementations plus every mock in the test suite. Instead, the contextvars.ContextVar side-channel satisfies the spirit of AC3 — token counts are available alongside the response, per-task and race-free — without breaking the existing Agent contract.

Billing Integrity Design

When ainvoke() succeeds but a subsequent step (e.g. update_memory()) raises an exception:

• The LLM provider has already charged the user for the tokens consumed by ainvoke().
• The except handlers check sentinel variables (_captured_prompt, _captured_completion) set immediately after ainvoke() succeeds.
• If the sentinels are set (non-None), the captured counts are preserved in _last_token_usage and last_token_usage_var so the router receives accurate billing data.
• Only pre-ainvoke() failures (sentinels still None) reset counts to (0, 0), because in those cases no tokens were consumed.

Quality Gates

Gate	Result
nox -e lint	✅ All checks passed
nox -e typecheck	✅ 0 errors, 1 pre-existing warning
nox -e unit_tests	✅ 2298/2298 scenarios passed
nox -e integration_tests	✅ All tests passed
nox -e coverage_report	✅ 97.1% (threshold 96.5%)

ADR References

• ADR-2027: ActorResult Type and Token Aggregation
• ADR-2026: Per-request credential injection (state field preserved)

Deferred Items

• _safe_int() / _safe_token_int() consolidation (N1): Both helpers implement the same coercion logic. Moving to a shared module is a refactor deferred to a follow-up ticket.
• NodeUsageTuple location (issue #13): Moving the type alias to cleveractors.result is a minor organizational improvement deferred to a follow-up.
• Literal[...] annotation duplicating _CAUSE_* constants: Python's Literal type does not support referencing constant names directly, so the string values must be duplicated. This is a known limitation of the language and is accepted as-is.
• N2 (Two logger instances in nodes.py): Node.__init__ sets self.logger = logging.getLogger(__name__) which produces the same object as the module-level _logger. Consolidating to use _logger directly in Node is a pre-existing nit deferred to a follow-up.
• N3 (Token-extraction fallback logic duplicated): Extracting a shared _read_last_token_usage(agent) helper is a refactor deferred to a follow-up.
• N4 (_execute_llm/_execute_tool compute totals directly): Inconsistency with sum() pattern used in _execute_graph/_execute_multi_actor is deferred to a follow-up.

## Summary Implements `cleveractors-core` issue #14 — the full AC1–AC7 acceptance criteria for `ActorResult`, `NodeUsage`, and real LangChain token counting. Closes #14 ## Motivation The CleverThis router consumes `Executor.execute()` to run actors. Before this PR, the router received `ActorResult` objects whose `prompt_tokens` and `completion_tokens` fields were **estimated** (via tiktoken or a 4-chars/token heuristic) rather than read from LangChain's actual response metadata. This made billing data inaccurate. The router also could not find the canonical `ActorResult`/`NodeUsage` types at the spec-mandated location (`cleveractors.result`). ## Changes ### New module: `src/cleveractors/result.py` (AC1) Defines `NodeUsage` and `ActorResult` dataclasses at the spec-mandated location. Also defines `MAX_REASONABLE_TOKENS = 10_000_000` as the single authoritative constant (previously duplicated in three modules). `runtime.py` now imports and re-exports both types for backward compatibility. `__init__.py` updated to import from `result` instead of `runtime`. Uses lowercase `list[NodeUsage]` (PEP 585) throughout. `nodes` is a required positional field (no default) so the contract that `nodes` is always non-empty is statically enforced. `state` field uses `dict[str, Any] | None` (PEP 585, consistent with rest of codebase). `__all__` declaration added. ### `LLMAgent` — real token extraction + race-condition fix + billing integrity (AC2, AC3, M1, M2, m3) `process_message()` now: - **Uses a module-level `contextvars.ContextVar` (`last_token_usage_var`)** to store per-task token counts. Each `asyncio.Task` inherits a copy of the context, so parallel graph branches that share the same `LLMAgent` instance each see their own token counts — eliminating the race condition. - **Resets `_last_token_usage = (0, 0)` and `last_token_usage_var.set((0, 0))` as the first statement inside the `try:` block**. - **Captures token counts into sentinel variables (`_captured_prompt`, `_captured_completion`) immediately after `ainvoke()` succeeds**, before the memory-update block. This marks the "ainvoke succeeded" boundary. - **Billing integrity fix**: If a post-`ainvoke()` step (e.g. `update_memory()`) raises an exception, the `except` handlers check the sentinel variables. If `_captured_prompt is not None`, `ainvoke()` already succeeded and the LLM provider has already charged the user — the captured counts are **preserved** rather than zeroed. Only pre-`ainvoke()` failures (where no tokens were consumed) reset to `(0, 0)`. - Added a prominent docstring `.. note::` to `process_message()` documenting this billing integrity guarantee. ### `Node._execute_agent()` — race condition fix, agent-type-aware fallback (M1 cycle-9 fix) - **Resets `last_token_usage_var` to `(0, 0)` at the start of the `try:` block**, before calling `agent.process_message()`. - **Agent-type-aware fallback logic** (cycle 9): For `LLMAgent`, the ContextVar is authoritative — `(0, 0)` means zero tokens, not "unset". The fallback to `_last_token_usage` instance attribute is **never** used for `LLMAgent` instances, preventing the race where a sibling parallel branch overwrites the instance attribute. For non-`LLMAgent` agents (ToolAgent, ChainAgent, etc.), the ContextVar is always `(0, 0)` since `process_message()` does not touch it; these agents fall back to the instance attribute as before. - Fixed `_node_token_usage` type annotation from `Optional[dict[str, Any]]` to `dict[str, Any] | None` (PEP 604, consistent with rest of codebase). ### `runtime_dispatch.py` — same agent-type-aware fallback in `_execute_llm` (M1 cycle-9 fix) - Applied the same agent-type-aware fallback pattern to `_execute_llm` for consistency with `Node._execute_agent()`. - Renamed cryptic exception variable `_nu_err` → `_node_usage_err` in `_execute_graph`. - **Multi-actor placeholder normalisation**: Added `_normalize_node_id()` helper and `_GRAPH_PLACEHOLDER_RE` module-level regex. - Renamed loop variable `nu` → `node_usage` in `_execute_multi_actor` for readability. - Moved `import re` to the top-level import block. ### `pure_graph.py` — import organisation - Moved `from cleveractors.result import MAX_REASONABLE_TOKENS as _MAX_REASONABLE_TOKENS` to the top-level import block. ### `features/mocks/actor_result_helpers.py` — new mock helpers module (cycle-9 fix) - Extracted `_make_llm_actor_config` and `_make_mock_chat_model` from the step file into `features/mocks/actor_result_helpers.py` (renamed to `make_llm_actor_config` and `make_mock_chat_model`), per project convention that mock factory functions belong in `features/mocks/`. ### BDD tests updated (M1, M2, m4, m5, m6, cycle-9 fixes) - **Strengthened parallel isolation assertions** (cycle 9): The "ContextVar isolates token counts" scenario now uses exact-value assertions (`t1 == (100, 50)` and `t2 == (200, 80)`) instead of weak `!= (0, 0)` and `!= each other` checks. A buggy swapped-mapping implementation would now fail. - **Removed bare `except Exception` in wrong-arity real-path test** (cycle 9): The `artc_wrong_arity_real_path_test` branch no longer catches all exceptions and fabricates a fake `ActorResult`. If `executor.execute()` raises unexpectedly, the test now fails loudly. - **Deleted dead step definition** `step_artc_graph_executor_wrong_arity_tuple` (cycle 9): This ~44-line step was never referenced by any scenario in the feature file. - **New scenario: ContextVar parallel-branch isolation** (M2): A single `LLMAgent` instance is called concurrently via `asyncio.gather()` with two different mocked `usage_metadata` responses; each task asserts its `last_token_usage_var.get()` reflects its own call's counts. - **Updated scenario: Post-ainvoke exception preserves token counts** (m4, billing integrity fix): An `LLMAgent` with `memory_enabled=True` succeeds at `ainvoke()` but fails at `update_memory()`; asserts `_last_token_usage == (42, 17)` after the exception (previously incorrectly asserted `(0, 0)`). This verifies that billing data is not lost when memory persistence fails. - **New scenarios: large-value pass-through** (m5): `_safe_node_token_int(20_000_000)` and `_safe_token_int(20_000_000)` both return `20_000_000` unchanged. - **Rewritten wrong-arity scenario** (m6): The first wrong-arity scenario now exercises the real `Node._execute_agent()` code path via `AgentFactory.create_agent` mock. ### CHANGELOG.md — stale reference removed - Removed stale `_usage_log` reference from the "Mutable state hygiene" entry (this field never existed in this PR's scope). The entry now accurately describes the actual new mechanism: `_last_token_usage` and `last_token_usage_var` are reset at the start of every `process_message()` call; `PureLangGraph._node_usages` is reset at the start of every `execute()` call; post-`ainvoke()` exceptions preserve captured token counts for billing integrity. ## Architecture Note AC3 says "`process_message()` returns token counts alongside the response string." Changing the `Agent` abstract base class return type from `str` to `tuple[str, int, int]` would require updating all four concrete agent implementations plus every mock in the test suite. Instead, the `contextvars.ContextVar` side-channel satisfies the spirit of AC3 — token counts are available alongside the response, per-task and race-free — without breaking the existing Agent contract. ## Billing Integrity Design When `ainvoke()` succeeds but a subsequent step (e.g. `update_memory()`) raises an exception: - The LLM provider has **already charged** the user for the tokens consumed by `ainvoke()`. - The `except` handlers check sentinel variables (`_captured_prompt`, `_captured_completion`) set immediately after `ainvoke()` succeeds. - If the sentinels are set (non-None), the captured counts are **preserved** in `_last_token_usage` and `last_token_usage_var` so the router receives accurate billing data. - Only pre-`ainvoke()` failures (sentinels still `None`) reset counts to `(0, 0)`, because in those cases no tokens were consumed. ## Quality Gates | Gate | Result | |------|--------| | `nox -e lint` | ✅ All checks passed | | `nox -e typecheck` | ✅ 0 errors, 1 pre-existing warning | | `nox -e unit_tests` | ✅ 2298/2298 scenarios passed | | `nox -e integration_tests` | ✅ All tests passed | | `nox -e coverage_report` | ✅ 97.1% (threshold 96.5%) | ## ADR References - ADR-2027: ActorResult Type and Token Aggregation - ADR-2026: Per-request credential injection (state field preserved) ## Deferred Items - **`_safe_int()` / `_safe_token_int()` consolidation** (N1): Both helpers implement the same coercion logic. Moving to a shared module is a refactor deferred to a follow-up ticket. - **`NodeUsageTuple` location** (issue #13): Moving the type alias to `cleveractors.result` is a minor organizational improvement deferred to a follow-up. - **`Literal[...]` annotation duplicating `_CAUSE_*` constants**: Python's `Literal` type does not support referencing constant names directly, so the string values must be duplicated. This is a known limitation of the language and is accepted as-is. - **N2 (Two logger instances in nodes.py)**: `Node.__init__` sets `self.logger = logging.getLogger(__name__)` which produces the same object as the module-level `_logger`. Consolidating to use `_logger` directly in `Node` is a pre-existing nit deferred to a follow-up. - **N3 (Token-extraction fallback logic duplicated)**: Extracting a shared `_read_last_token_usage(agent)` helper is a refactor deferred to a follow-up. - **N4 (`_execute_llm`/`_execute_tool` compute totals directly)**: Inconsistency with `sum()` pattern used in `_execute_graph`/`_execute_multi_actor` is deferred to a follow-up.

hurui200320 added the

Type

Feature

label 2026-06-10 05:49:35 +00:00

hurui200320 added a new dependency 2026-06-10 05:49:41 +00:00

#14 feat(ActorResult): implement ActorResult and NodeUsage types; capture per-node token counts from LangChain responses

hurui200320 referenced this pull request 2026-06-10 05:50:08 +00:00

feat(ActorResult): implement ActorResult and NodeUsage types; capture per-node token counts from LangChain responses #14

hurui200320 force-pushed feature/actor-result from 25e6cce910 to 8a09b96412 2026-06-10 07:46:52 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 8a09b96412 to ad3186a07e 2026-06-10 08:50:20 +00:00 Compare

hurui200320 force-pushed feature/actor-result from ad3186a07e to e969851786 2026-06-10 08:57:21 +00:00 Compare

hurui200320 force-pushed feature/actor-result from e969851786 to 498f0f87ae 2026-06-10 09:47:53 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 498f0f87ae to 7f0e459717 2026-06-10 09:51:01 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 7f0e459717 to 6d1eca444d 2026-06-10 11:20:31 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 6d1eca444d to 40c102574d 2026-06-10 11:21:14 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 40c102574d to b21a41e16a 2026-06-10 11:25:14 +00:00 Compare

hurui200320 force-pushed feature/actor-result from b21a41e16a to 9491b69f10 2026-06-10 12:22:35 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 9491b69f10 to ea355c6433 2026-06-10 13:16:10 +00:00 Compare

hurui200320 force-pushed feature/actor-result from ea355c6433 to ab8b1451f0 2026-06-10 13:24:29 +00:00 Compare

hurui200320 force-pushed feature/actor-result from ab8b1451f0 to 7068d5d664 2026-06-10 14:40:16 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 7068d5d664 to 403d3e683a 2026-06-10 14:52:15 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 403d3e683a to 7d21833827 2026-06-10 15:07:29 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 7d21833827 to 0a120b2ec2 2026-06-10 15:15:21 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 0a120b2ec2 to e778443aad 2026-06-10 16:11:56 +00:00 Compare

hurui200320 force-pushed feature/actor-result from e778443aad to 16f8d4b945 2026-06-10 16:58:49 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 16f8d4b945 to 54c15c1778 2026-06-10 18:30:41 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 54c15c1778 to b488e10bdf 2026-06-10 18:38:34 +00:00 Compare

hurui200320 force-pushed feature/actor-result from b488e10bdf to 2657f8c311 2026-06-10 18:45:07 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 2657f8c311 to 1a6648d763 2026-06-10 19:23:25 +00:00 Compare

hurui200320 force-pushed feature/actor-result from 1a6648d763 to dd17cd6467 2026-06-10 19:35:15 +00:00 Compare

hurui200320 force-pushed feature/actor-result from dd17cd6467 to e772d04193 2026-06-10 20:17:52 +00:00 Compare

hurui200320 force-pushed feature/actor-result from e772d04193 to c9dbfa6517 2026-06-10 20:33:52 +00:00 Compare

hurui200320 force-pushed feature/actor-result from c9dbfa6517 to 2664ebfd75 2026-06-11 03:05:25 +00:00 Compare

hurui200320 commented 2026-06-11 03:25:12 +00:00

Author

Member

Copy link

Review: Issue Compliance & Regression Check

Reviewed against two questions: (1) does the code implement what issue #14 requires, and (2) does it break anything?

---

1. Issue #14 acceptance criteria — all met ✅

AC	Status	Notes
AC1 — result.py with NodeUsage / ActorResult	✅	Fields match spec exactly; state preserved per ADR-2026; __all__ declared
AC2 — Real token extraction (usage_metadata → response_metadata fallback → warn+0)	✅	Four distinct warning causes; truthy non-dict guards; _safe_int() handles booleans, negatives, overflow, and implausible values
AC3 — Token counts available alongside response	✅	Satisfied via contextvars.ContextVar + _last_token_usage side-channel; architecture note in PR is sound
AC4 — PureLangGraph collects per-node (node_id, provider, model, prompt, completion)	✅	Node._execute_agent() writes _node_token_usage into state; _execute_from_node() pops it and accumulates into self._node_usages
AC5 — Executor.execute() aggregates into ActorResult with real counts	✅	All four dispatch paths (_execute_llm, _execute_graph, _execute_tool, _execute_multi_actor) rebuilt; estimation helpers deleted
AC6 — Aggregation invariant prompt_tokens == sum(n.prompt_tokens for n in nodes)	✅	Enforced explicitly in _execute_graph; recomputed after node-id prefixing in _execute_multi_actor
AC7 — ActorResult / NodeUsage exported from __init__.py and __all__	✅	__init__.py imports directly from cleveractors.result; result.py has __all__

All subtasks from the issue are checked off, including deletion of runtime_tokens.py and _estimate_tokens().

---

2. Regressions — none found ✅

Intentional breaking changes (all call sites updated, tests confirm):

• ActorResult.nodes is now a required field (no default). Correct per spec; the bot's field(default_factory=list) was an earlier deviation. Every creation site in runtime_dispatch.py provides nodes.
• Executor._usage_log removed. All four internal references cleaned up.
• runtime_tokens.py deleted. Its only caller (runtime_dispatch.py) is updated.

Return-type propagation — handled everywhere:
PureLangGraph.execute() now returns a 3-tuple. All four call sites updated: _execute_graph, PureLangGraph.process_message(), and both locations in application.py. process_message() also fixes a pre-existing bug where it was returning a tuple instead of a str.

Backward compatibility preserved:
from cleveractors.runtime import ActorResult, from cleveractors.runtime_types import ActorResult, and from cleveractors import ActorResult, NodeUsage all continue to work via shim re-exports.

Concurrency correctness:
The contextvars.ContextVar pattern is correct for asyncio — each Task inherits a copy of the context at creation time, making parallel branches that share the same LLMAgent instance race-free. The billing-integrity sentinels (_captured_prompt is not None) correctly distinguish pre-ainvoke() failures from post-ainvoke() failures. Token counts are read before cleanup() runs in the finally block.

---

Verdict: safe to merge ✅

Quality gates (lint, typecheck, 2298/2298 unit tests, integration tests, 97.1% coverage) all green. The deferred items (N1–N4) are code-quality nits with no correctness impact.

## Review: Issue Compliance & Regression Check Reviewed against two questions: (1) does the code implement what issue #14 requires, and (2) does it break anything? --- ### 1. Issue #14 acceptance criteria — all met ✅ | AC | Status | Notes | |---|---|---| | AC1 — `result.py` with `NodeUsage` / `ActorResult` | ✅ | Fields match spec exactly; `state` preserved per ADR-2026; `__all__` declared | | AC2 — Real token extraction (`usage_metadata` → `response_metadata` fallback → warn+0) | ✅ | Four distinct warning causes; truthy non-dict guards; `_safe_int()` handles booleans, negatives, overflow, and implausible values | | AC3 — Token counts available alongside response | ✅ | Satisfied via `contextvars.ContextVar` + `_last_token_usage` side-channel; architecture note in PR is sound | | AC4 — `PureLangGraph` collects per-node `(node_id, provider, model, prompt, completion)` | ✅ | `Node._execute_agent()` writes `_node_token_usage` into state; `_execute_from_node()` pops it and accumulates into `self._node_usages` | | AC5 — `Executor.execute()` aggregates into `ActorResult` with real counts | ✅ | All four dispatch paths (`_execute_llm`, `_execute_graph`, `_execute_tool`, `_execute_multi_actor`) rebuilt; estimation helpers deleted | | AC6 — Aggregation invariant `prompt_tokens == sum(n.prompt_tokens for n in nodes)` | ✅ | Enforced explicitly in `_execute_graph`; recomputed after node-id prefixing in `_execute_multi_actor` | | AC7 — `ActorResult` / `NodeUsage` exported from `__init__.py` and `__all__` | ✅ | `__init__.py` imports directly from `cleveractors.result`; `result.py` has `__all__` | All subtasks from the issue are checked off, including deletion of `runtime_tokens.py` and `_estimate_tokens()`. --- ### 2. Regressions — none found ✅ **Intentional breaking changes (all call sites updated, tests confirm):** - `ActorResult.nodes` is now a required field (no default). Correct per spec; the bot's `field(default_factory=list)` was an earlier deviation. Every creation site in `runtime_dispatch.py` provides `nodes`. - `Executor._usage_log` removed. All four internal references cleaned up. - `runtime_tokens.py` deleted. Its only caller (`runtime_dispatch.py`) is updated. **Return-type propagation — handled everywhere:** `PureLangGraph.execute()` now returns a 3-tuple. All four call sites updated: `_execute_graph`, `PureLangGraph.process_message()`, and both locations in `application.py`. `process_message()` also fixes a pre-existing bug where it was returning a tuple instead of a `str`. **Backward compatibility preserved:** `from cleveractors.runtime import ActorResult`, `from cleveractors.runtime_types import ActorResult`, and `from cleveractors import ActorResult, NodeUsage` all continue to work via shim re-exports. **Concurrency correctness:** The `contextvars.ContextVar` pattern is correct for asyncio — each `Task` inherits a copy of the context at creation time, making parallel branches that share the same `LLMAgent` instance race-free. The billing-integrity sentinels (`_captured_prompt is not None`) correctly distinguish pre-`ainvoke()` failures from post-`ainvoke()` failures. Token counts are read before `cleanup()` runs in the `finally` block. --- ### Verdict: safe to merge ✅ Quality gates (lint, typecheck, 2298/2298 unit tests, integration tests, 97.1% coverage) all green. The deferred items (N1–N4) are code-quality nits with no correctness impact.

hurui200320 merged commit 2664ebfd75 into master 2026-06-11 03:25:35 +00:00

hurui200320 deleted branch feature/actor-result 2026-06-11 03:25:35 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

  

auto/blocked-by-deps

PR blocked by an open issue dependency. Operator must close the dep (or remove the dependency link) before the merge driver can act. Auto-cleared by merge_drive when no open deps remain.

  

auto/ci-timeout

Most recent merge cycle hit CI timeout. Driver excludes this PR while last merge_cycle row is < 30 min old; label persists thereafter as visible history.

  

auto/claimed-implementer

Currently being processed by an implementer worker.

  

auto/claimed-merge

Currently being processed by the merge driver.

  

auto/claimed-reviewer

Currently being processed by a reviewer worker.

  

auto/driver-down

Merge driver heartbeat stale; pipeline halted. Closed automatically on next clean tick.

  

auto/invariant-violation

Detected master commit violating the strict merge invariant. Tracked as an issue (not a PR label); kept here for label completeness.

  

auto/last-attempt-tier-0

In-cycle escalation: most recent attempt ran at the Tier 0 slot (`tier-0`). Slot's model defined in .opencode/models/tiers.yaml.

  

auto/last-attempt-tier-1

In-cycle escalation: most recent attempt ran at the Tier 1 slot (`tier-1`). Slot's model defined in .opencode/models/tiers.yaml.

  

auto/last-attempt-tier-2

In-cycle escalation: most recent attempt ran at the Tier 2 slot (`tier-2`). Slot's model defined in .opencode/models/tiers.yaml. Gated behind IMPLEMENTER_ESCALATION_TIER2_ENABLED.

  

auto/last-attempt-tier-min

In-cycle escalation: most recent attempt ran at the Tier -1 slot (`tier-min`). Slot's model defined in .opencode/models/tiers.yaml. Suffix is ``-min`` (not ``--1``) so the Forgejo UI reads naturally.

  

Automation Tracking

Tracking issues used by the AI Automation system for agents to communicate and report.

  

auto/needs-conflict-resolution

Rebase conflict needs LLM conflict-resolver.

  

auto/needs-implementer

Failing CI needs implementer attention.

  

auto/postmortem

Documenting a driver incident or rollback.

  

auto/ready-to-merge

Reviewer has APPROVED this PR and no later REQUEST_CHANGES is outstanding. The merge driver requires this label to even consider a PR for merging. Set by the reviewer worker on APPROVE; cleared on REQUEST_CHANGES.

  

auto/restart-throttled

Train repeatedly lost master-tempo races. Driver excludes via merge_cycle until cooldown elapses; label persists as visible history.

  

auto/revert

Revert PR backing out an invariant violation. Fast-tracked through the merge driver.

  

auto/sentinel

Sentinel PR duplicated from upstream into a personal fork by tools/duplicate_prs_to_fork.py for pipeline testing. Lives only in the fork; the canonical pipeline never sees it.

  

auto/stale-inactivity

No implementer activity for N days. Flagged for human review. Auto-cleared on next push to head branch.

  

auto/unstable

Repeatedly fails on current master (>= 3 ci-fail-on-rebased-sha releases in 12 h). Excluded from driver until human triage.

  

Blocked

A ticket in a blocked state and unable to complete until some other task is completed first.

  

Bounty

$100

A bounty of $100 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$1000

A bounty of $1000 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$10000

A bounty of $10000 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$20

A bounty of $20 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$2000

A bounty of $2000 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$250

A bounty of $250 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$50

A bounty of $50 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$500

A bounty of $500 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$5000

A bounty of $5000 for any open-source contributor who provides a MR that solves this issue

  

Bounty

$750

A bounty of $750 for any open-source contributor who provides a MR that solves this issue

  

MoSCoW

Could have

Could have feature in order to satisfy the epic/legendary.

  

MoSCoW

Must have

Must have feature in order to satisfy the epic/legendary.

  

MoSCoW

Should have

Should have feature in order to satisfy the epic/legendary.

  

Needs Feedback

There are questions in the ticket that can not be completed until the project owner provides clarity.

  

Points

1

1 man-hours worth of work for an expert with no learning curve.

  

Points

13

13 man-hours worth of work for an expert with no learning curve.

  

Points

2

2 man-hours worth of work for an expert with no learning curve.

  

Points

21

21 man-hours worth of work for an expert with no learning curve.

  

Points

3

3 man-hours worth of work for an expert with no learning curve.

  

Points

34

34 man-hours worth of work for an expert with no learning curve.

  

Points

5

5 man-hours worth of work for an expert with no learning curve.

  

Points

55

55 man-hours worth of work for an expert with no learning curve.

  

Points

8

8 man-hours worth of work for an expert with no learning curve.

  

Points

88

88 man-hours worth of work for an expert with no learning curve.

  

Priority

Backlog

This ticket has backlogged priority and is not to be worked on yet

  

Priority

CI Blocker

Critical priority issue that blocks CI/CD pipeline and prevents PR merges

  

Priority

Critical

The priority is critical

  

Priority

High

The priority is high

  

Priority

Low

The priority is low

  

Priority

Medium

The priority is medium

  

Signed-off: Owner

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  

Signed-off: Scrum Master

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  

Signed-off: Tech Lead

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  

Spike

A ticket for learning a tool or technology that is needed to be able to do future planning and design.

  

State

Completed

The ticket has been fully implemented, completed, and merged with the source code. This label should only be applied once a ticket is closed.

  

State

Duplicate

A ticket that represents the same content as an existing ticket.

  

State

In Progress

A ticket that is actively being developed.

  

State

In Review

A ticket that has had some code completed to implement but is waiting to pass peer review and is not yet merged in.

  

State

Paused

This ticket's work started but wasn't finished. It's on hold (likely in a feature branch) and will be resumed later, either due to a blocker or a delay.

  

State

Unverified

All new tickets start in this state. A developer may set it to show the ticket is unverified. This means we haven't agreed to work on it. It will either move to a verified state or be closed as wontdo.

  

State

Verified

The issue has been verified by a developer as legitimate. It will be worked on and verified tickets are now considered part of the backlog.

  

State

Wont Do

This ticket has been decided it wont be done. This may mean the bug has been determined to not be real (cant verify) or the feature is one we have decided we dont want to adopt.

  

Type

Automation

Any edits or discussion about the AI automated coding system.

  

Type

Bug

Something that doesnt work as intended.

  

Type

Discussion

Anytime a ticket represents a discussion about a subject and doesnt fall into one of the other categories.

  

Type

Documentation

An error or improvement needed in the documentation.

  

Type

Epic

Any first tier epic. That is, an epic which contains only issues as children and will not have sub-epics.

  

Type

Feature

Some new functionality not present.

  

Type

Legendary

A type of Epic which will contain other Epics.

  

Type

Refactor

A code change that restructures existing code without changing its external behavior.

  

Type

Support

Someone needs help using the project.

  

Type

Task

A generic task that doesnt fit into the other type categories.

  

Type

Testing

Work exclusively focusing on fixing or expanding testing.

No labels

auto/blocked-by-deps

auto/ci-timeout

auto/claimed-implementer

auto/claimed-merge

auto/claimed-reviewer

auto/driver-down

auto/invariant-violation

auto/last-attempt-tier-0

auto/last-attempt-tier-1

auto/last-attempt-tier-2

auto/last-attempt-tier-min

Automation Tracking

auto/needs-conflict-resolution

auto/needs-implementer

auto/postmortem

auto/ready-to-merge

auto/restart-throttled

auto/revert

auto/sentinel

auto/stale-inactivity

auto/unstable

Blocked

Bounty

$100

Bounty

$1000

Bounty

$10000

Bounty

$20

Bounty

$2000

Bounty

$250

Bounty

$50

Bounty

$500

Bounty

$5000

Bounty

$750

MoSCoW

Could have

MoSCoW

Must have

MoSCoW

Should have

Needs Feedback

Points

1

Points

13

Points

2

Points

21

Points

3

Points

34

Points

5

Points

55

Points

8

Points

88

Priority

Backlog

Priority

CI Blocker

Priority

Critical

Priority

High

Priority

Low

Priority

Medium

Signed-off: Owner

Signed-off: Scrum Master

Signed-off: Tech Lead

Spike

State

Completed

State

Duplicate

State

In Progress

State

In Review

State

Paused

State

Unverified

State

Verified

State

Wont Do

Type

Automation

Type

Bug

Type

Discussion

Type

Documentation

Type

Epic

Type

Feature

Type

Legendary

Type

Refactor

Type

Support

Type

Task

Type

Testing

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks

#14 feat(ActorResult): implement ActorResult and NodeUsage types; capture per-node token counts from LangChain responses

cleveragents/cleveractors-core

Reference

cleveragents/cleveractors-core!41

No description provided.