UAT: TraceService._langsmith_enabled() bypasses Settings.langsmith_enabled, reads env var directly #3764

New issue

Open

opened 2026-04-05 22:33:12 +00:00 by freemo · 0 comments

freemo commented

2026-04-05 22:33:12 +00:00

Owner

Bug Report

What Was Tested

Code-level analysis of src/cleveragents/application/services/trace_service.py — specifically the LangSmith integration and its relationship to the Settings dependency injection pattern.

Expected Behavior (from spec)

The spec (line ~43888 and ~46101) states:

LangSmith | (optional) | LLM observability | Optional integration for tracing LLM calls, token usage, latency, and cost. Configured via CLEVERAGENTS_LANGSMITH_* environment variables.

LangSmith integration: When CLEVERAGENTS_LANGSMITH_ENABLED is true, all LLM traces are additionally forwarded to LangSmith.

The Settings class already provides a normalized langsmith_enabled field (in config/settings.py line 406) that accepts multiple env var aliases:

CLEVERAGENTS_LANGSMITH_ENABLED
CLEVERAGENTS_LANGSMITH_TRACING_V2
LANGSMITH_TRACING_V2
LANGCHAIN_TRACING_V2

Since TraceService already accepts Settings as a constructor argument (self._settings), it should use self._settings.langsmith_enabled to check whether LangSmith forwarding is enabled.

Actual Behavior

TraceService._langsmith_enabled() is a static method that ignores the injected Settings and reads LANGCHAIN_TRACING_V2 directly from the environment:

# trace_service.py lines 253-256 — WRONG
@staticmethod
def _langsmith_enabled() -> bool:
    """Check whether LangSmith forwarding is enabled."""
    return os.environ.get("LANGCHAIN_TRACING_V2", "").lower() == "true"

This causes two problems:

Only LANGCHAIN_TRACING_V2 is checked — setting CLEVERAGENTS_LANGSMITH_ENABLED=true or LANGSMITH_TRACING_V2=true has no effect on TraceService, even though Settings normalizes all these aliases.
Bypasses DI — the Settings object is injected precisely to centralize configuration. Bypassing it with a direct os.environ.get() call breaks the DI contract and makes the service harder to test (requires env var manipulation instead of settings injection).

Code Location

File: src/cleveragents/application/services/trace_service.py
Lines 253–256 (_langsmith_enabled static method)
Line 76 (call site: if self._langsmith_enabled():)

Steps to Reproduce

Set CLEVERAGENTS_LANGSMITH_ENABLED=true in the environment (without setting LANGCHAIN_TRACING_V2)
Call TraceService.record_trace(trace) — LangSmith forwarding will not be triggered, even though the user configured it via the documented env var

Fix Required

Convert _langsmith_enabled from a static method to an instance method that uses self._settings:

def _langsmith_enabled(self) -> bool:
    """Check whether LangSmith forwarding is enabled."""
    return self._settings.langsmith_enabled

This leverages the existing Settings.langsmith_enabled field which already normalizes all supported env var aliases.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Bug Report ### What Was Tested Code-level analysis of `src/cleveragents/application/services/trace_service.py` — specifically the LangSmith integration and its relationship to the `Settings` dependency injection pattern. ### Expected Behavior (from spec) The spec (line ~43888 and ~46101) states: > **LangSmith** | (optional) | LLM observability | Optional integration for tracing LLM calls, token usage, latency, and cost. Configured via `CLEVERAGENTS_LANGSMITH_*` environment variables. > **LangSmith integration**: When `CLEVERAGENTS_LANGSMITH_ENABLED` is `true`, all LLM traces are additionally forwarded to LangSmith. The `Settings` class already provides a normalized `langsmith_enabled` field (in `config/settings.py` line 406) that accepts multiple env var aliases: - `CLEVERAGENTS_LANGSMITH_ENABLED` - `CLEVERAGENTS_LANGSMITH_TRACING_V2` - `LANGSMITH_TRACING_V2` - `LANGCHAIN_TRACING_V2` Since `TraceService` already accepts `Settings` as a constructor argument (`self._settings`), it should use `self._settings.langsmith_enabled` to check whether LangSmith forwarding is enabled. ### Actual Behavior `TraceService._langsmith_enabled()` is a **static method** that ignores the injected `Settings` and reads `LANGCHAIN_TRACING_V2` directly from the environment: ```python # trace_service.py lines 253-256 — WRONG @staticmethod def _langsmith_enabled() -> bool: """Check whether LangSmith forwarding is enabled.""" return os.environ.get("LANGCHAIN_TRACING_V2", "").lower() == "true" ``` This causes two problems: 1. **Only `LANGCHAIN_TRACING_V2` is checked** — setting `CLEVERAGENTS_LANGSMITH_ENABLED=true` or `LANGSMITH_TRACING_V2=true` has no effect on `TraceService`, even though `Settings` normalizes all these aliases. 2. **Bypasses DI** — the `Settings` object is injected precisely to centralize configuration. Bypassing it with a direct `os.environ.get()` call breaks the DI contract and makes the service harder to test (requires env var manipulation instead of settings injection). ### Code Location - **File**: `src/cleveragents/application/services/trace_service.py` - **Lines 253–256** (`_langsmith_enabled` static method) - **Line 76** (call site: `if self._langsmith_enabled():`) ### Steps to Reproduce 1. Set `CLEVERAGENTS_LANGSMITH_ENABLED=true` in the environment (without setting `LANGCHAIN_TRACING_V2`) 2. Call `TraceService.record_trace(trace)` — LangSmith forwarding will **not** be triggered, even though the user configured it via the documented env var ### Fix Required Convert `_langsmith_enabled` from a static method to an instance method that uses `self._settings`: ```python def _langsmith_enabled(self) -> bool: """Check whether LangSmith forwarding is enabled.""" return self._settings.langsmith_enabled ``` This leverages the existing `Settings.langsmith_enabled` field which already normalizes all supported env var aliases. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo referenced this issue

2026-04-05 22:33:37 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 (Resumed) #3654

freemo referenced this issue

2026-04-05 22:34:53 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 (Resumed) #3654

freemo referenced this issue

2026-04-05 22:34:59 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 (Resumed) #3654

freemo referenced this issue

2026-04-06 09:18:32 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775