forked from cleveragents/cleveragents-core
170 lines
8.0 KiB
Gherkin
170 lines
8.0 KiB
Gherkin
Feature: Settings runtime helpers
|
|
Verify environment alias, storage paths, database URLs, production flags,
|
|
provider configuration lookups, and LangSmith config guards behave predictably.
|
|
|
|
Scenario: Environment alias setter syncs env field
|
|
Given no environment variables are set
|
|
When I load the settings with defaults
|
|
And I set the environment alias to "staging"
|
|
Then the environment alias should be "staging"
|
|
And the environment should be "staging"
|
|
|
|
Scenario: Storage base path trims whitespace
|
|
Given the storage base path is "/tmp/storage"
|
|
When I compute the storage path for " logs "
|
|
Then the storage path should be "/tmp/storage/logs"
|
|
|
|
Scenario: Storage base path returns default when type missing
|
|
Given the storage base path is "/tmp/storage"
|
|
When I compute the default storage path
|
|
Then the storage path should be "/tmp/storage"
|
|
|
|
Scenario: SQLite URLs derive implicit test database
|
|
Given no environment variables are set
|
|
When I derive the test database URL from "sqlite:///primary.db"
|
|
Then the derived test database URL should be "sqlite:///primary_test.db"
|
|
|
|
Scenario: Non-SQLite URLs fall back to original string
|
|
When I derive the test database URL from "postgresql://localhost/app"
|
|
Then the derived test database URL should be "postgresql://localhost/app"
|
|
|
|
Scenario: Production mode helpers honour flags
|
|
When I evaluate production mode with env "production", debug "false", and reload "false"
|
|
Then is_production should be True
|
|
And is_production_mode should be True
|
|
|
|
Scenario: Unknown provider alias short-circuits lookups
|
|
Given no API keys are set
|
|
When I check if "unknown" provider is configured
|
|
Then has_provider_configured should be False for "unknown"
|
|
|
|
Scenario: Explicit provider values skip env overrides
|
|
Given the environment variable "OPENAI_API_KEY" is set to "env-key"
|
|
When I instantiate settings with explicit openai API key "constructor-key"
|
|
Then the openai API key should be "constructor-key"
|
|
|
|
Scenario: Provider env vars satisfy configuration checks
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
When I check if "openai" provider is configured
|
|
Then has_provider_configured should be True for "openai"
|
|
|
|
Scenario: Provider fallback reads env vars when attribute unset
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
When I clear the explicit "openai" key and recheck configuration
|
|
Then has_provider_configured should be True for "openai"
|
|
|
|
Scenario: Provider defaults honour environment overrides
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
And the environment variable "CLEVERAGENTS_DEFAULT_PROVIDER" is set to "openai"
|
|
And the environment variable "CLEVERAGENTS_DEFAULT_MODEL" is set to "gpt-4.1-mini"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "openai" from source "env"
|
|
And the provider defaults should report model "gpt-4.1-mini" from source "env"
|
|
|
|
Scenario: Provider defaults fall back to configured providers
|
|
Given no environment variables are set
|
|
And the anthropic API key is set to "anthropic-key"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "anthropic" from source "auto"
|
|
And the provider defaults should report model "claude-sonnet-4-20250514" from source "provider-default"
|
|
|
|
Scenario: Provider validation surfaces Azure endpoint requirements
|
|
Given no environment variables are set
|
|
And the azure API key is set to "azure-key"
|
|
When I load the settings
|
|
And I validate provider configuration for "azure"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "Azure provider requires AZURE_OPENAI_ENDPOINT"
|
|
|
|
Scenario: Provider validation passes when Azure configuration complete
|
|
Given no environment variables are set
|
|
And the azure API key is set to "azure-key"
|
|
And the environment variable "AZURE_OPENAI_ENDPOINT" is set to "https://azure.example.com"
|
|
And the environment variable "AZURE_OPENAI_DEPLOYMENT" is set to "gpt4o"
|
|
When I load the settings
|
|
And I validate provider configuration for "azure"
|
|
Then provider validation should pass with no errors
|
|
|
|
Scenario: LangSmith config builder returns None when disabled
|
|
When I build the LangSmith config
|
|
Then the LangSmith config should be absent
|
|
|
|
Scenario: Relative vector store path resolves against cwd
|
|
Given the vector store path override is "relative/vector"
|
|
When I resolve the vector store path
|
|
Then the resolved vector store path should equal the override under the current working directory
|
|
|
|
Scenario: LangSmith validation surfaces missing credentials
|
|
Given LangSmith is enabled without credentials
|
|
When I validate the LangSmith configuration
|
|
Then LangSmith validation should fail with 2 errors
|
|
|
|
Scenario: LangSmith endpoint synchronization updates LangChain environment
|
|
Given LangSmith is configured with API key "abc123", project "coverage-runner", and endpoint "https://trace.local"
|
|
When I check if LangSmith is enabled
|
|
Then the "LANGCHAIN_ENDPOINT" environment variable should equal "https://trace.local"
|
|
|
|
Scenario: Provider lookup trims api key suffix before resolving
|
|
Given the environment variable "OPENAI_API_KEY" is set to "env-key"
|
|
When I check if "openai_api_key" provider is configured
|
|
Then has_provider_configured should be True for "openai_api_key"
|
|
|
|
Scenario: Provider lookup falls back to direct map entries
|
|
Given the environment variable "HF_TOKEN" is set to "hf-secret"
|
|
When I check if "hf_token" provider is configured
|
|
Then has_provider_configured should be True for "hf_token"
|
|
|
|
Scenario: Custom provider candidate resolves env mapping
|
|
Given a temporary provider "custom" uses env "CUSTOM_API_KEY"
|
|
And the environment variable "CUSTOM_API_KEY" is set to "dynamic-key"
|
|
When I check if "custom_api_key" provider is configured using a constructed instance
|
|
Then has_provider_configured should be True for "custom_api_key"
|
|
|
|
Scenario: Settings default provider populates provider defaults
|
|
Given no environment variables are set
|
|
And no API keys are set
|
|
When I construct settings with default provider "groq"
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "groq" from source "settings"
|
|
|
|
Scenario: Settings default model populates provider defaults
|
|
Given no environment variables are set
|
|
And no API keys are set
|
|
When I construct settings with default model "custom-model"
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report model "custom-model" from source "settings"
|
|
|
|
Scenario: Provider defaults fall back to first configured provider outside priority list
|
|
Given no environment variables are set
|
|
And the environment variable "HF_TOKEN" is set to "hf-secret"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "hf" from source "auto"
|
|
|
|
Scenario: Provider validation surfaces missing providers
|
|
Given no environment variables are set
|
|
When I load the settings
|
|
And I validate provider configuration for "none"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "No AI providers configured."
|
|
|
|
Scenario: Provider validation requires specific environment variables
|
|
Given no environment variables are set
|
|
When I load the settings
|
|
And I validate provider configuration for "openai"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "openai provider requires OPENAI_API_KEY"
|
|
|
|
Scenario: Provider normalization rejects whitespace-only input
|
|
When I normalize the provider name " "
|
|
Then the normalized provider name should be absent
|
|
|
|
Scenario: Provider normalization returns canonical names for unknown providers
|
|
When I normalize the provider name "Custom-LLM "
|
|
Then the normalized provider name should be "customllm"
|