cleveragents/cleveragents-core

Fork 3

BUG-HUNT: [business-rule] CostMetadata.record_usage allows total_tokens to desync from input_tokens + output_tokens #7770

New issue

Open

opened 2026-04-12 03:30:30 +00:00 by HAL9000 · 3 comments

HAL9000 commented

2026-04-12 03:30:30 +00:00

Owner

Bug Report: Business Rule — CostMetadata.record_usage total_tokens Can Desync

Severity Assessment

Impact: total_tokens is accumulated separately from input_tokens + output_tokens, creating a potential for desynced totals if calls fail mid-update or if only part of the update succeeds. Additionally, total_tokens starts at 0 and is updated by adding input_tokens + output_tokens per call, but the field is also independently mutable via validate_assignment=True — meaning external code can set total_tokens to a value inconsistent with input_tokens + output_tokens.
Likelihood: Medium — occurs if external code directly assigns to total_tokens or if record_usage is called in a concurrent context without locking.
Priority: Medium

Location

File: src/cleveragents/domain/models/core/cost_metadata.py
Class: CostMetadata
Method: record_usage
Lines: ~60-100

Description

CostMetadata has three token fields: total_tokens, input_tokens, and output_tokens. The record_usage method updates all three:

self.input_tokens += input_tokens
self.output_tokens += output_tokens
self.total_tokens += input_tokens + output_tokens

However, since the model has validate_assignment=True but no invariant validator enforcing total_tokens == input_tokens + output_tokens, external code can set any of these fields independently and create an inconsistent state:

md = CostMetadata()
md.record_usage(input_tokens=10, output_tokens=5, cost=0.01, provider="openai")
# total_tokens=15, input=10, output=5 -- consistent
md.total_tokens = 0  # reset total but not input/output -- INCONSISTENT!
# Now total_tokens=0 but input_tokens=10, output_tokens=5

Also, total_tokens serves no purpose beyond input_tokens + output_tokens — it is a derived value that should always equal their sum, but is stored independently without enforcement.

Evidence

# From cost_metadata.py
class CostMetadata(BaseModel):
    total_tokens: int = Field(default=0, ge=0, ...)
    input_tokens: int = Field(default=0, ge=0, ...)
    output_tokens: int = Field(default=0, ge=0, ...)

    def record_usage(self, *, input_tokens, output_tokens, cost, provider):
        self.input_tokens += input_tokens
        self.output_tokens += output_tokens
        self.total_tokens += input_tokens + output_tokens  # Tracked separately
        # No invariant: total_tokens should == input_tokens + output_tokens

    model_config = ConfigDict(validate_assignment=True)  # External mutations allowed

Expected Behavior

Either:

total_tokens should be a @computed_field / @property that returns input_tokens + output_tokens, eliminating the desynced state possibility, OR
A model validator should enforce total_tokens == input_tokens + output_tokens after every mutation.

Actual Behavior

total_tokens is an independent mutable field with no invariant enforcement, allowing desynced state.

Suggested Fix

Convert total_tokens to a computed property:

from pydantic import computed_field

class CostMetadata(BaseModel):
    input_tokens: int = Field(default=0, ge=0)
    output_tokens: int = Field(default=0, ge=0)
    total_cost: float = Field(default=0.0, ge=0.0)
    # ... other fields ...

    @computed_field
    @property
    def total_tokens(self) -> int:
        return self.input_tokens + self.output_tokens

TDD Note

After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD.

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: bug-hunter

## Bug Report: Business Rule — CostMetadata.record_usage total_tokens Can Desync ### Severity Assessment - **Impact**: `total_tokens` is accumulated separately from `input_tokens + output_tokens`, creating a potential for desynced totals if calls fail mid-update or if only part of the update succeeds. Additionally, `total_tokens` starts at 0 and is updated by adding `input_tokens + output_tokens` per call, but the field is also independently mutable via `validate_assignment=True` — meaning external code can set `total_tokens` to a value inconsistent with `input_tokens + output_tokens`. - **Likelihood**: Medium — occurs if external code directly assigns to `total_tokens` or if `record_usage` is called in a concurrent context without locking. - **Priority**: Medium ### Location - **File**: `src/cleveragents/domain/models/core/cost_metadata.py` - **Class**: `CostMetadata` - **Method**: `record_usage` - **Lines**: ~60-100 ### Description `CostMetadata` has three token fields: `total_tokens`, `input_tokens`, and `output_tokens`. The `record_usage` method updates all three: ```python self.input_tokens += input_tokens self.output_tokens += output_tokens self.total_tokens += input_tokens + output_tokens ``` However, since the model has `validate_assignment=True` but no invariant validator enforcing `total_tokens == input_tokens + output_tokens`, external code can set any of these fields independently and create an inconsistent state: ```python md = CostMetadata() md.record_usage(input_tokens=10, output_tokens=5, cost=0.01, provider="openai") # total_tokens=15, input=10, output=5 -- consistent md.total_tokens = 0 # reset total but not input/output -- INCONSISTENT! # Now total_tokens=0 but input_tokens=10, output_tokens=5 ``` Also, `total_tokens` serves no purpose beyond `input_tokens + output_tokens` — it is a derived value that should always equal their sum, but is stored independently without enforcement. ### Evidence ```python # From cost_metadata.py class CostMetadata(BaseModel): total_tokens: int = Field(default=0, ge=0, ...) input_tokens: int = Field(default=0, ge=0, ...) output_tokens: int = Field(default=0, ge=0, ...) def record_usage(self, *, input_tokens, output_tokens, cost, provider): self.input_tokens += input_tokens self.output_tokens += output_tokens self.total_tokens += input_tokens + output_tokens # Tracked separately # No invariant: total_tokens should == input_tokens + output_tokens model_config = ConfigDict(validate_assignment=True) # External mutations allowed ``` ### Expected Behavior Either: 1. `total_tokens` should be a `@computed_field` / `@property` that returns `input_tokens + output_tokens`, eliminating the desynced state possibility, OR 2. A model validator should enforce `total_tokens == input_tokens + output_tokens` after every mutation. ### Actual Behavior `total_tokens` is an independent mutable field with no invariant enforcement, allowing desynced state. ### Suggested Fix Convert `total_tokens` to a computed property: ```python from pydantic import computed_field class CostMetadata(BaseModel): input_tokens: int = Field(default=0, ge=0) output_tokens: int = Field(default=0, ge=0) total_cost: float = Field(default=0.0, ge=0.0) # ... other fields ... @computed_field @property def total_tokens(self) -> int: return self.input_tokens + self.output_tokens ``` ### Category business-rule ### TDD Note After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: bug-hunter

HAL9000 added this to the v3.2.0 milestone

2026-04-12 03:43:58 +00:00

HAL9000 referenced this issue

2026-04-12 03:47:56 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 1) #7724

HAL9000 added the

labels

2026-04-12 05:47:53 +00:00

freemo added the

Type

Bug

label

2026-04-12 18:33:37 +00:00

HAL9000 added

and removed

labels

2026-04-14 05:58:18 +00:00

HAL9000 commented

2026-04-14 05:58:18 +00:00

Author

Owner

✅ Verified — Business rule bug: CostMetadata allows total_tokens to desync from input+output. MoSCoW: Should-have. Priority: Medium.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — Business rule bug: CostMetadata allows total_tokens to desync from input+output. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 commented

2026-04-14 06:11:35 +00:00

Author

Owner

✅ Verified — Business rule bug: CostMetadata allows total_tokens to desync from input+output. MoSCoW: Should-have. Priority: Medium.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor