cleveragents/cleveragents-core

Fork 3

UAT: `LLMTrace` model missing spec-required fields — `actor_name`, `total_tokens`, `temperature`, and `context_refs` absent #1983

New issue

Open

opened 2026-04-03 00:30:37 +00:00 by freemo · 2 comments

freemo commented

2026-04-03 00:30:37 +00:00

Owner

Metadata

Branch: fix/llm-trace-missing-spec-fields
Commit Message: fix(observability): align LLMTrace model fields with specification — add missing fields and rename actor to actor_name
Milestone: v3.6.0
Parent Epic: #945

Description

The LLMTrace model in src/cleveragents/domain/models/observability/llm_trace.py is missing four fields that are explicitly defined in the specification's LLMTrace class definition. Additionally, the field actor in the code should be named actor_name per the spec.

Specification-defined LLMTrace fields (spec §LLM Call Tracing):

class LLMTrace(BaseModel):
    trace_id: str                     # ULID
    plan_id: str
    decision_id: str | None
    actor_name: str                   # <-- spec uses actor_name
    provider: str
    model: str
    prompt_tokens: int
    completion_tokens: int
    total_tokens: int                 # <-- MISSING from code
    cost_usd: float
    latency_ms: int
    temperature: float                # <-- MISSING from code
    tool_calls: list[str]             # spec: list[str], code: list[dict]
    context_hash: str
    context_refs: list[str]           # <-- MISSING from code
    streaming: bool
    retry_count: int
    error: str | None
    timestamp: datetime

Actual implementation (src/cleveragents/domain/models/observability/llm_trace.py):

actor (should be actor_name per spec)
total_tokens — MISSING
temperature — MISSING
context_refs — MISSING
tool_calls is list[dict[str, Any]] (spec says list[str] — tool names only)

Steps to Reproduce:

from cleveragents.domain.models.observability.llm_trace import LLMTrace
fields = LLMTrace.model_fields
print('actor_name' in fields)   # False (field is named 'actor')
print('total_tokens' in fields) # False (missing)
print('temperature' in fields)  # False (missing)
print('context_refs' in fields) # False (missing)

Expected Behavior (per spec):
The LLMTrace model should include all fields defined in the specification, with correct names and types.

Code Location:
src/cleveragents/domain/models/observability/llm_trace.py

Subtasks

Rename actor field to actor_name (with backward-compatible migration)
Add total_tokens: int field (computed as prompt_tokens + completion_tokens or stored separately)
Add temperature: float field
Add context_refs: list[str] field (resource IDs referenced in context)
Change tool_calls from list[dict] to list[str] (tool names only, per spec)
Update LLMTraceRepository to persist/restore new fields
Update tests to cover new fields

Definition of Done

LLMTrace model has all spec-required fields with correct names and types
actor_name field present (not actor)
total_tokens, temperature, context_refs fields present
tool_calls is list[str] (tool names)
Repository persists and restores all fields
Tests verify all fields
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/llm-trace-missing-spec-fields` - **Commit Message**: `fix(observability): align LLMTrace model fields with specification — add missing fields and rename actor to actor_name` - **Milestone**: v3.6.0 - **Parent Epic**: #945 ## Description The `LLMTrace` model in `src/cleveragents/domain/models/observability/llm_trace.py` is missing four fields that are explicitly defined in the specification's `LLMTrace` class definition. Additionally, the field `actor` in the code should be named `actor_name` per the spec. **Specification-defined LLMTrace fields (spec §LLM Call Tracing):** ```python class LLMTrace(BaseModel): trace_id: str # ULID plan_id: str decision_id: str | None actor_name: str # <-- spec uses actor_name provider: str model: str prompt_tokens: int completion_tokens: int total_tokens: int # <-- MISSING from code cost_usd: float latency_ms: int temperature: float # <-- MISSING from code tool_calls: list[str] # spec: list[str], code: list[dict] context_hash: str context_refs: list[str] # <-- MISSING from code streaming: bool retry_count: int error: str | None timestamp: datetime ``` **Actual implementation (`src/cleveragents/domain/models/observability/llm_trace.py`):** - `actor` (should be `actor_name` per spec) - `total_tokens` — MISSING - `temperature` — MISSING - `context_refs` — MISSING - `tool_calls` is `list[dict[str, Any]]` (spec says `list[str]` — tool names only) **Steps to Reproduce:** ```python from cleveragents.domain.models.observability.llm_trace import LLMTrace fields = LLMTrace.model_fields print('actor_name' in fields) # False (field is named 'actor') print('total_tokens' in fields) # False (missing) print('temperature' in fields) # False (missing) print('context_refs' in fields) # False (missing) ``` **Expected Behavior (per spec):** The `LLMTrace` model should include all fields defined in the specification, with correct names and types. **Code Location:** `src/cleveragents/domain/models/observability/llm_trace.py` ## Subtasks - [ ] Rename `actor` field to `actor_name` (with backward-compatible migration) - [ ] Add `total_tokens: int` field (computed as `prompt_tokens + completion_tokens` or stored separately) - [ ] Add `temperature: float` field - [ ] Add `context_refs: list[str]` field (resource IDs referenced in context) - [ ] Change `tool_calls` from `list[dict]` to `list[str]` (tool names only, per spec) - [ ] Update `LLMTraceRepository` to persist/restore new fields - [ ] Update tests to cover new fields ## Definition of Done - [ ] `LLMTrace` model has all spec-required fields with correct names and types - [ ] `actor_name` field present (not `actor`) - [ ] `total_tokens`, `temperature`, `context_refs` fields present - [ ] `tool_calls` is `list[str]` (tool names) - [ ] Repository persists and restores all fields - [ ] Tests verify all fields - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.6.0 milestone

2026-04-03 00:30:53 +00:00

freemo added a new dependency

2026-04-03 00:33:11 +00:00

#945 Legendary: Specification Alignment — Close All Gaps Between Implementation and Specification

freemo referenced this issue

2026-04-03 00:39:04 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 501 — v3.7.0 #2003

freemo referenced this issue

2026-04-03 00:49:11 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 511 — v3.7.0 #2004

freemo referenced this issue

2026-04-03 00:59:17 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 521 — v3.7.0 #2005

freemo referenced this issue

2026-04-03 01:09:23 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 531 — v3.7.0 #2008

freemo referenced this issue

2026-04-03 01:19:29 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 541 — v3.7.0 #2009

freemo referenced this issue

2026-04-03 01:29:36 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 551 — v3.7.0 #2010

freemo referenced this issue

2026-04-03 01:39:45 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 561 — v3.7.0 #2011

freemo referenced this issue

2026-04-03 01:49:53 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 571 — v3.7.0 #2012

freemo referenced this issue

2026-04-03 02:00:02 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 581 — v3.7.0 #2013

freemo referenced this issue

2026-04-03 02:10:08 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 591 — v3.7.0 #2015

freemo referenced this issue

2026-04-03 02:20:13 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 601 — v3.7.0 #2016

freemo referenced this issue

2026-04-03 02:30:19 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 611 — v3.7.0 #2017

freemo referenced this issue

2026-04-03 02:40:25 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 621 — v3.7.0 #2019

freemo commented

2026-04-03 02:43:31 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium (confirmed)
Milestone: v3.5.0 (confirmed — LLMTrace is part of the autonomy hardening milestone)
MoSCoW: Should Have — The spec requires actor_name, total_tokens, temperature, and context_refs fields on LLMTrace. These are spec-mandated fields that should be present for proper observability of LLM interactions. Not blocking core execution but important for spec compliance.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) - **Milestone**: v3.5.0 (confirmed — LLMTrace is part of the autonomy hardening milestone) - **MoSCoW**: Should Have — The spec requires `actor_name`, `total_tokens`, `temperature`, and `context_refs` fields on `LLMTrace`. These are spec-mandated fields that should be present for proper observability of LLM interactions. Not blocking core execution but important for spec compliance. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-03 02:50:31 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 631 — v3.7.0 #2032

freemo referenced this issue

2026-04-03 03:00:38 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 641 — v3.7.0 #2033

freemo commented

2026-04-05 06:53:02 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium
Milestone: v3.6.0
MoSCoW: Should Have — already correctly labeled; LLMTrace model completeness is important for observability and debugging
Parent Epic: #397 (Server & Autonomy Infrastructure)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium - **Milestone**: v3.6.0 - **MoSCoW**: Should Have — already correctly labeled; `LLMTrace` model completeness is important for observability and debugging - **Parent Epic**: #397 (Server & Autonomy Infrastructure) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-06 09:18:32 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775