freemo
c8b66c8294
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / security (push) Waiting to run
CI / quality (push) Waiting to run
CI / unit_tests (push) Waiting to run
CI / integration_tests (push) Waiting to run
CI / e2e_tests (push) Waiting to run
CI / coverage (push) Blocked by required conditions
CI / benchmark-regression (push) Blocked by required conditions
CI / benchmark-publish (push) Waiting to run
CI / build (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / helm (push) Waiting to run
CI / status-check (push) Blocked by required conditions
Reviewed and APPROVED. Closes #2871.
211 lines
10 KiB
Gherkin
211 lines
10 KiB
Gherkin
Feature: Settings runtime helpers
|
|
Verify environment alias, storage paths, database URLs, production flags,
|
|
provider configuration lookups, and LangSmith config guards behave predictably.
|
|
|
|
Scenario: data_dir default is the spec-required home directory path
|
|
Given no environment variables are set
|
|
When I load the settings with defaults
|
|
Then the data directory should equal the home cleveragents path
|
|
|
|
Scenario: data_dir env var override takes precedence over default
|
|
Given no environment variables are set
|
|
And the environment variable "CLEVERAGENTS_DATA_DIR" is set to "/tmp/custom-data"
|
|
When I load the settings with defaults
|
|
Then the data directory should be "/tmp/custom-data"
|
|
|
|
Scenario: Environment alias setter syncs env field
|
|
Given no environment variables are set
|
|
When I load the settings with defaults
|
|
And I set the environment alias to "staging"
|
|
Then the environment alias should be "staging"
|
|
And the environment should be "staging"
|
|
|
|
Scenario: Storage base path trims whitespace
|
|
Given the storage base path is "/tmp/storage"
|
|
When I compute the storage path for " logs "
|
|
Then the storage path should be "/tmp/storage/logs"
|
|
|
|
Scenario: Storage base path returns default when type missing
|
|
Given the storage base path is "/tmp/storage"
|
|
When I compute the default storage path
|
|
Then the storage path should be "/tmp/storage"
|
|
|
|
Scenario: SQLite URLs derive implicit test database
|
|
Given no environment variables are set
|
|
When I derive the test database URL from "sqlite:///primary.db"
|
|
Then the derived test database URL should be "sqlite:///primary_test.db"
|
|
|
|
Scenario: Non-SQLite URLs fall back to original string
|
|
When I derive the test database URL from "postgresql://localhost/app"
|
|
Then the derived test database URL should be "postgresql://localhost/app"
|
|
|
|
Scenario: Production mode helpers honour flags
|
|
When I evaluate production mode with env "production", debug "false", and reload "false"
|
|
Then is_production should be True
|
|
And is_production_mode should be True
|
|
|
|
Scenario: Unknown provider alias short-circuits lookups
|
|
Given no API keys are set
|
|
When I check if "unknown" provider is configured
|
|
Then has_provider_configured should be False for "unknown"
|
|
|
|
Scenario: Explicit provider values skip env overrides
|
|
Given the environment variable "OPENAI_API_KEY" is set to "env-key"
|
|
When I instantiate settings with explicit openai API key "constructor-key"
|
|
Then the openai API key should be "constructor-key"
|
|
|
|
Scenario: Provider env vars satisfy configuration checks
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
When I check if "openai" provider is configured
|
|
Then has_provider_configured should be True for "openai"
|
|
|
|
Scenario: Provider fallback reads env vars when attribute unset
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
When I clear the explicit "openai" key and recheck configuration
|
|
Then has_provider_configured should be True for "openai"
|
|
|
|
Scenario: Provider defaults honour environment overrides
|
|
Given no API keys are set
|
|
And the openai API key is set to "env-key"
|
|
And the environment variable "CLEVERAGENTS_DEFAULT_PROVIDER" is set to "openai"
|
|
And the environment variable "CLEVERAGENTS_DEFAULT_MODEL" is set to "gpt-4.1-mini"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "openai" from source "env"
|
|
And the provider defaults should report model "gpt-4.1-mini" from source "env"
|
|
|
|
Scenario: Provider defaults fall back to configured providers
|
|
Given no environment variables are set
|
|
And the anthropic API key is set to "anthropic-key"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "anthropic" from source "auto"
|
|
And the provider defaults should report model "claude-sonnet-4-20250514" from source "provider-default"
|
|
|
|
Scenario: Provider validation surfaces Azure endpoint requirements
|
|
Given no environment variables are set
|
|
And the azure API key is set to "azure-key"
|
|
When I load the settings
|
|
And I validate provider configuration for "azure"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "Azure provider requires AZURE_OPENAI_ENDPOINT"
|
|
|
|
Scenario: Provider validation passes when Azure configuration complete
|
|
Given no environment variables are set
|
|
And the azure API key is set to "azure-key"
|
|
And the environment variable "AZURE_OPENAI_ENDPOINT" is set to "https://azure.example.com"
|
|
And the environment variable "AZURE_OPENAI_DEPLOYMENT" is set to "gpt4o"
|
|
When I load the settings
|
|
And I validate provider configuration for "azure"
|
|
Then provider validation should pass with no errors
|
|
|
|
Scenario: LangSmith config builder returns None when disabled
|
|
When I build the LangSmith config
|
|
Then the LangSmith config should be absent
|
|
|
|
Scenario: Relative vector store path resolves against cwd
|
|
Given the vector store path override is "relative/vector"
|
|
When I resolve the vector store path
|
|
Then the resolved vector store path should equal the override under the current working directory
|
|
|
|
Scenario: LangSmith validation surfaces missing credentials
|
|
Given LangSmith is enabled without credentials
|
|
When I validate the LangSmith configuration
|
|
Then LangSmith validation should fail with 2 errors
|
|
|
|
Scenario: LangSmith endpoint synchronization updates LangChain environment
|
|
Given LangSmith is configured with API key "abc123", project "coverage-runner", and endpoint "https://trace.local"
|
|
When I check if LangSmith is enabled
|
|
Then the "LANGCHAIN_ENDPOINT" environment variable should equal "https://trace.local"
|
|
|
|
Scenario: Provider lookup trims api key suffix before resolving
|
|
Given the environment variable "OPENAI_API_KEY" is set to "env-key"
|
|
When I check if "openai_api_key" provider is configured
|
|
Then has_provider_configured should be True for "openai_api_key"
|
|
|
|
Scenario: Provider lookup falls back to direct map entries
|
|
Given the environment variable "HF_TOKEN" is set to "hf-secret"
|
|
When I check if "hf_token" provider is configured
|
|
Then has_provider_configured should be True for "hf_token"
|
|
|
|
Scenario: Custom provider candidate resolves env mapping
|
|
Given a temporary provider "custom" uses env "CUSTOM_API_KEY"
|
|
And the environment variable "CUSTOM_API_KEY" is set to "dynamic-key"
|
|
When I check if "custom_api_key" provider is configured using a constructed instance
|
|
Then has_provider_configured should be True for "custom_api_key"
|
|
|
|
Scenario: Settings default provider populates provider defaults
|
|
Given no environment variables are set
|
|
And no API keys are set
|
|
When I construct settings with default provider "groq"
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "groq" from source "settings"
|
|
|
|
Scenario: Settings default model populates provider defaults
|
|
Given no environment variables are set
|
|
And no API keys are set
|
|
When I construct settings with default model "custom-model"
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report model "custom-model" from source "settings"
|
|
|
|
Scenario: Provider defaults fall back to first configured provider outside priority list
|
|
Given no environment variables are set
|
|
And the environment variable "HF_TOKEN" is set to "hf-secret"
|
|
When I load the settings
|
|
And I resolve provider defaults
|
|
Then the provider defaults should report provider "hf" from source "auto"
|
|
|
|
Scenario: Provider validation surfaces missing providers
|
|
Given no environment variables are set
|
|
When I load the settings
|
|
And I validate provider configuration for "none"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "No AI providers configured."
|
|
|
|
Scenario: Provider validation requires specific environment variables
|
|
Given no environment variables are set
|
|
When I load the settings
|
|
And I validate provider configuration for "openai"
|
|
Then provider validation should fail
|
|
And provider validation errors should include "openai provider requires OPENAI_API_KEY"
|
|
|
|
Scenario: Provider normalization rejects whitespace-only input
|
|
When I normalize the provider name " "
|
|
Then the normalized provider name should be absent
|
|
|
|
Scenario: Provider normalization returns canonical names for unknown providers
|
|
When I normalize the provider name "Custom-LLM "
|
|
Then the normalized provider name should be "customllm"
|
|
|
|
Scenario: Settings reset clears singleton cache
|
|
Given no environment variables are set
|
|
And the environment variable "CLEVERAGENTS_ENV" is set to "production"
|
|
When I load singleton settings via get_settings
|
|
And I reset singleton settings
|
|
And the environment variable "CLEVERAGENTS_ENV" is set to "staging"
|
|
And I load singleton settings via get_settings
|
|
Then singleton settings instances should be different
|
|
And singleton environment should be "staging"
|
|
|
|
Scenario: Default database_url resolves to spec-required home directory path
|
|
Given no environment variables are set
|
|
And "CLEVERAGENTS_DATABASE_URL" is not set
|
|
When I load the settings with defaults
|
|
Then the database URL should point to "~/.cleveragents/cleveragents.db"
|
|
And the database URL should start with "sqlite:///"
|
|
And the database URL should contain ".cleveragents/cleveragents.db"
|
|
|
|
Scenario: CLEVERAGENTS_DATABASE_URL env var overrides the default
|
|
Given no environment variables are set
|
|
And the environment variable "CLEVERAGENTS_DATABASE_URL" is set to "sqlite:///custom/path/mydb.db"
|
|
When I load the settings with defaults
|
|
Then the database URL should be "sqlite:///custom/path/mydb.db"
|
|
|
|
Scenario: Default database_url uses absolute path not relative path
|
|
Given no environment variables are set
|
|
And "CLEVERAGENTS_DATABASE_URL" is not set
|
|
When I load the settings with defaults
|
|
Then the database URL should be an absolute path
|