UAT: `_forward_trace_to_langsmith` missing `start_time` and `end_time` — LangSmith SDK requires timing for accurate run tracking #4043

New issue

Open

opened 2026-04-06 09:16:30 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 09:16:30 +00:00

Owner

Metadata

Branch: fix/trace-service-langsmith-missing-timing
Commit Message: fix(observability): add start_time and end_time to LangSmith create_run call
Milestone: (none — backlog)
Parent Epic: #945

Backlog note: This issue was discovered during autonomous operation
on milestone v3.6.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Bug Report

What Was Tested

Code-level analysis of src/cleveragents/application/services/trace_service.py — specifically the _forward_trace_to_langsmith() function and its call to client.create_run().

Expected Behavior

The LangSmith SDK's create_run() method accepts start_time and end_time parameters for accurate latency tracking and timeline visualization in the LangSmith UI. Without these, LangSmith cannot accurately display when the LLM call occurred or how long it took.

The LLMTrace model already stores the information needed to derive these values:

timestamp — when the trace was recorded (can serve as end_time)
latency_ms — duration of the call (can be used to calculate start_time = end_time - latency_ms)

Actual Behavior

The _forward_trace_to_langsmith() function calls client.create_run(**run_data) without start_time or end_time:

# src/cleveragents/application/services/trace_service.py lines 293-313
run_data: dict[str, Any] = {
    "name": f"llm-trace-{trace.trace_id}",
    "run_type": "llm",
    "inputs": {
        "actor": trace.actor,
        "provider": trace.provider,
        "model": trace.model,
        "prompt_tokens": trace.prompt_tokens,
        "streaming": trace.streaming,
    },
    "outputs": {
        "completion_tokens": trace.completion_tokens,
        "cost_usd": trace.cost_usd,
        "latency_ms": trace.latency_ms,
        "tool_calls": trace.tool_calls,
    },
}
# Missing: start_time, end_time
client.create_run(**run_data)

Impact

LangSmith run timeline shows incorrect or missing timing information
Latency analysis in LangSmith UI is inaccurate
The latency_ms field in outputs is a workaround but not the standard LangSmith way to track timing

Code Location

File: src/cleveragents/application/services/trace_service.py
Function: _forward_trace_to_langsmith() (lines 277-314)

Fix Required

Add start_time and end_time to the create_run() call:

from datetime import timedelta

end_time = trace.timestamp
start_time = end_time - timedelta(milliseconds=trace.latency_ms)

run_data["start_time"] = start_time
run_data["end_time"] = end_time
client.create_run(**run_data)

Subtasks

Derive end_time from trace.timestamp and start_time from trace.timestamp - timedelta(milliseconds=trace.latency_ms)
Add start_time and end_time to the run_data dict passed to client.create_run()
Ensure datetime import and timedelta usage are statically typed and pass Pyright
Write BDD scenario in features/ covering the timing fields in the LangSmith forwarding path
Update any existing BDD scenarios for _forward_trace_to_langsmith to assert timing fields are present

Definition of Done

All subtasks above are checked off
_forward_trace_to_langsmith() passes start_time and end_time to client.create_run()
start_time is correctly derived as trace.timestamp - timedelta(milliseconds=trace.latency_ms)
end_time is correctly set to trace.timestamp
BDD scenario covers the timing fields and passes
Commit created with exact first line: fix(observability): add start_time and end_time to LangSmith create_run call
Commit pushed to branch fix/trace-service-langsmith-missing-timing
PR submitted, reviewed, and merged
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/trace-service-langsmith-missing-timing` - **Commit Message**: `fix(observability): add start_time and end_time to LangSmith create_run call` - **Milestone**: *(none — backlog)* - **Parent Epic**: #945 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.6.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Bug Report ### What Was Tested Code-level analysis of `src/cleveragents/application/services/trace_service.py` — specifically the `_forward_trace_to_langsmith()` function and its call to `client.create_run()`. ### Expected Behavior The LangSmith SDK's `create_run()` method accepts `start_time` and `end_time` parameters for accurate latency tracking and timeline visualization in the LangSmith UI. Without these, LangSmith cannot accurately display when the LLM call occurred or how long it took. The `LLMTrace` model already stores the information needed to derive these values: - `timestamp` — when the trace was recorded (can serve as `end_time`) - `latency_ms` — duration of the call (can be used to calculate `start_time = end_time - latency_ms`) ### Actual Behavior The `_forward_trace_to_langsmith()` function calls `client.create_run(**run_data)` without `start_time` or `end_time`: ```python # src/cleveragents/application/services/trace_service.py lines 293-313 run_data: dict[str, Any] = { "name": f"llm-trace-{trace.trace_id}", "run_type": "llm", "inputs": { "actor": trace.actor, "provider": trace.provider, "model": trace.model, "prompt_tokens": trace.prompt_tokens, "streaming": trace.streaming, }, "outputs": { "completion_tokens": trace.completion_tokens, "cost_usd": trace.cost_usd, "latency_ms": trace.latency_ms, "tool_calls": trace.tool_calls, }, } # Missing: start_time, end_time client.create_run(**run_data) ``` ### Impact - LangSmith run timeline shows incorrect or missing timing information - Latency analysis in LangSmith UI is inaccurate - The `latency_ms` field in `outputs` is a workaround but not the standard LangSmith way to track timing ### Code Location - **File**: `src/cleveragents/application/services/trace_service.py` - **Function**: `_forward_trace_to_langsmith()` (lines 277-314) ### Fix Required Add `start_time` and `end_time` to the `create_run()` call: ```python from datetime import timedelta end_time = trace.timestamp start_time = end_time - timedelta(milliseconds=trace.latency_ms) run_data["start_time"] = start_time run_data["end_time"] = end_time client.create_run(**run_data) ``` ## Subtasks - [ ] Derive `end_time` from `trace.timestamp` and `start_time` from `trace.timestamp - timedelta(milliseconds=trace.latency_ms)` - [ ] Add `start_time` and `end_time` to the `run_data` dict passed to `client.create_run()` - [ ] Ensure `datetime` import and `timedelta` usage are statically typed and pass Pyright - [ ] Write BDD scenario in `features/` covering the timing fields in the LangSmith forwarding path - [ ] Update any existing BDD scenarios for `_forward_trace_to_langsmith` to assert timing fields are present ## Definition of Done - [ ] All subtasks above are checked off - [ ] `_forward_trace_to_langsmith()` passes `start_time` and `end_time` to `client.create_run()` - [ ] `start_time` is correctly derived as `trace.timestamp - timedelta(milliseconds=trace.latency_ms)` - [ ] `end_time` is correctly set to `trace.timestamp` - [ ] BDD scenario covers the timing fields and passes - [ ] Commit created with exact first line: `fix(observability): add start_time and end_time to LangSmith create_run call` - [ ] Commit pushed to branch `fix/trace-service-langsmith-missing-timing` - [ ] PR submitted, reviewed, and merged - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 09:17:31 +00:00

#945 Legendary: Specification Alignment — Close All Gaps Between Implementation and Specification

freemo referenced this issue

2026-04-06 09:18:27 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 09:18:32 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo added the

Points

label