63 lines
3.4 KiB
Gherkin
63 lines
3.4 KiB
Gherkin
Feature: LangSmith configuration detection
|
|
As a developer configuring observability
|
|
I want to verify CleverAgents detects LangSmith settings
|
|
So that tracing can be enabled or disabled safely
|
|
|
|
Background:
|
|
Given LangSmith environment is clean
|
|
|
|
Scenario: LangSmith tracing disabled by default
|
|
When I load CleverAgents settings for LangSmith
|
|
Then LangSmith tracing should be disabled
|
|
|
|
Scenario: LangSmith tracing enabled via LangChain env vars
|
|
Given I set environment variable "LANGCHAIN_TRACING_V2" to "true"
|
|
And I set environment variable "LANGCHAIN_API_KEY" to "demo-key"
|
|
And I set environment variable "LANGCHAIN_PROJECT" to "langchain-env-project"
|
|
When I load CleverAgents settings for LangSmith
|
|
Then LangSmith tracing should be enabled
|
|
|
|
Scenario: Building LangSmith config metadata
|
|
Given I set environment variable "CLEVERAGENTS_LANGSMITH_ENABLED" to "true"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_PROJECT" to "cleveragents-core"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_API_KEY" to "demo-key"
|
|
When I load CleverAgents settings for LangSmith
|
|
And I build a LangSmith config with run name "demo-run"
|
|
Then the LangSmith config should include tag "context-analysis"
|
|
And the LangSmith config should include metadata key "langsmith_project"
|
|
|
|
Scenario: LangSmith validation fails without API key
|
|
Given I set environment variable "CLEVERAGENTS_LANGSMITH_ENABLED" to "true"
|
|
When I load CleverAgents settings for LangSmith
|
|
Then LangSmith tracing should be disabled
|
|
And the LangSmith validation errors should mention "API key"
|
|
|
|
Scenario: LangSmith detects LangChain variables
|
|
Given I set environment variable "LANGCHAIN_TRACING_V2" to "true"
|
|
And I set environment variable "LANGCHAIN_API_KEY" to "demo-key"
|
|
And I set environment variable "LANGCHAIN_PROJECT" to "observability-demo"
|
|
When I load CleverAgents settings for LangSmith
|
|
And I build a LangSmith config with run name "auto-detected"
|
|
Then LangSmith tracing should be enabled
|
|
And the LangSmith config should include metadata key "langsmith_project"
|
|
And the LangSmith config should include metadata value "observability-demo" for key "langsmith_project"
|
|
|
|
Scenario: LangSmith synchronization sets LangChain env vars
|
|
Given I set environment variable "CLEVERAGENTS_LANGSMITH_ENABLED" to "true"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_API_KEY" to "sync-key"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_PROJECT" to "sync-project"
|
|
When I load CleverAgents settings for LangSmith
|
|
Then the environment variable "LANGCHAIN_TRACING_V2" should equal "true"
|
|
And the environment variable "LANGCHAIN_PROJECT" should equal "sync-project"
|
|
And the environment variable "LANGCHAIN_API_KEY" should equal "sync-key"
|
|
|
|
Scenario: LangSmith config includes user metadata
|
|
Given I set environment variable "CLEVERAGENTS_LANGSMITH_ENABLED" to "true"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_API_KEY" to "user-key"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_PROJECT" to "user-project"
|
|
And I set environment variable "CLEVERAGENTS_LANGSMITH_USER_ID" to "user-123"
|
|
When I load CleverAgents settings for LangSmith
|
|
And I build a LangSmith config with run name "user-run"
|
|
Then the LangSmith config should include metadata key "user_id"
|
|
And the LangSmith config should include metadata value "user-123" for key "user_id"
|