forked from cleveragents/cleveragents-core
freemo
7c4663b8ee
Implement the complete configuration system with multi-level resolution chain, typed key registry, and CLI integration per specification. ConfigService changes: - Expand _build_catalog() to register all 102 spec-aligned config keys across 8 groups: core (14), server (4), actor (5), plan (8), sandbox (5), index (12), context (43), provider (11) - Each key carries exact dotted-dash name, Python type, default value, explicit env var name per spec, project-scopability flag, and description - Fix _env_name() to convert dots and dashes to underscores - Provider keys use standard env var names (e.g., OPENAI_API_KEY) CLI commands rewiring: - Rewrite config set/get/list to use ConfigService instead of Settings - Add --verbose flag to config get showing full 5-level resolution chain - Add --project flag to config set/get/list for project-scoped overrides - Support both glob and regex patterns in config list - Validate keys against ConfigService registry with actionable errors - Retain backward-compatible helper functions delegating to ConfigService Documentation: - Add docs/reference/config_resolution.md covering resolution chain, all 102 config keys, CLI commands, TOML format, and provider credentials Testing: - Update all 4 Behave feature files and step definitions to use new spec-aligned key names, env vars, and defaults (119 scenarios passing) - Add robot/config_resolution.robot with 10 integration test cases - Add benchmarks/config_resolution_bench.py with 8 time + 2 memory suites ISSUES CLOSED: #258
287 lines
15 KiB
Gherkin
287 lines
15 KiB
Gherkin
Feature: Config CLI safety-net coverage
|
|
As a developer
|
|
I want thorough safety-net tests for every function in config.py
|
|
So that 100% line and branch coverage is maintained as the code evolves
|
|
|
|
# =====================================================================
|
|
# _normalize_key (L91-96)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _normalize_key returns registry key as-is
|
|
When the safety-net normalizer processes key "core.log.level"
|
|
Then the safety-net normalized result should be "core.log.level"
|
|
|
|
Scenario: safety-net _normalize_key converts underscores to dots for registry match
|
|
When the safety-net normalizer processes key "plan_concurrency"
|
|
Then the safety-net normalized result should be "plan.concurrency"
|
|
|
|
Scenario: safety-net _normalize_key strips whitespace
|
|
When the safety-net normalizer processes key " core.log.level "
|
|
Then the safety-net normalized result should be "core.log.level"
|
|
|
|
Scenario: safety-net _normalize_key returns unrecognized key as-is
|
|
When the safety-net normalizer processes key "unknown_random_key"
|
|
Then the safety-net normalized result should be "unknown_random_key"
|
|
|
|
# =====================================================================
|
|
# _is_secret_key (L116-118)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _is_secret_key detects api_key pattern
|
|
When the safety-net secret checker inspects key "openai_api_key"
|
|
Then the safety-net secret check result should be true
|
|
|
|
Scenario: safety-net _is_secret_key detects token pattern
|
|
When the safety-net secret checker inspects key "auth_token"
|
|
Then the safety-net secret check result should be true
|
|
|
|
Scenario: safety-net _is_secret_key detects password pattern
|
|
When the safety-net secret checker inspects key "db_password"
|
|
Then the safety-net secret check result should be true
|
|
|
|
Scenario: safety-net _is_secret_key detects secret pattern
|
|
When the safety-net secret checker inspects key "client_secret"
|
|
Then the safety-net secret check result should be true
|
|
|
|
Scenario: safety-net _is_secret_key rejects non-secret key
|
|
When the safety-net secret checker inspects key "log_level"
|
|
Then the safety-net secret check result should be false
|
|
|
|
Scenario: safety-net _is_secret_key is case insensitive
|
|
When the safety-net secret checker inspects key "API_KEY"
|
|
Then the safety-net secret check result should be true
|
|
|
|
# =====================================================================
|
|
# _mask_value (L121-123)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _mask_value always returns four asterisks
|
|
When the safety-net masker masks value "super-secret-123"
|
|
Then the safety-net masked output should be "****"
|
|
|
|
Scenario: safety-net _mask_value masks empty string
|
|
When the safety-net masker masks an empty string value
|
|
Then the safety-net masked output should be "****"
|
|
|
|
# =====================================================================
|
|
# _env_var_for_key (L158-160)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _env_var_for_key builds correct env var name
|
|
When the safety-net env var builder processes key "log_level"
|
|
Then the safety-net env var name should be "CLEVERAGENTS_LOG_LEVEL"
|
|
|
|
Scenario: safety-net _env_var_for_key uppercases the key
|
|
When the safety-net env var builder processes key "server_port"
|
|
Then the safety-net env var name should be "CLEVERAGENTS_SERVER_PORT"
|
|
|
|
# =====================================================================
|
|
# _read_config_file (L126-131)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _read_config_file returns empty dict when file absent
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net reader reads the config file
|
|
Then the safety-net read result should be an empty dict
|
|
|
|
Scenario: safety-net _read_config_file reads existing toml data
|
|
Given a safety-net isolated temp config directory
|
|
And a safety-net toml config file containing key "debug_enabled" with value "true"
|
|
When the safety-net reader reads the config file
|
|
Then the safety-net read result should contain key "debug_enabled"
|
|
|
|
# =====================================================================
|
|
# _write_config_file (L134-155) - create new file
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _write_config_file creates file when absent
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net writer writes key "log_level" with value "INFO"
|
|
Then the safety-net config file should exist and contain key "log_level"
|
|
|
|
# =====================================================================
|
|
# _settings_fields (L62-70)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _settings_fields returns populated dictionary
|
|
When the safety-net fields loader retrieves all settings fields
|
|
Then the safety-net fields result should be a non-empty dict
|
|
|
|
# =====================================================================
|
|
# _validate_key (L99-113) - unknown key path
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _validate_key raises on unknown key with helpful message
|
|
When the safety-net validator checks unknown key "zzz_nonexistent_xxy"
|
|
Then the safety-net validator should raise BadParameter with "Unknown configuration key"
|
|
|
|
Scenario: safety-net _validate_key accepts valid key and returns normalized form
|
|
When the safety-net validator checks valid key "core.log.level"
|
|
Then the safety-net validator should return "core.log.level"
|
|
|
|
# =====================================================================
|
|
# _resolve_source (L163-174) - default path
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _resolve_source returns default when no env or file
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net source resolver checks key "core.log.level"
|
|
Then the safety-net resolved source should be "default"
|
|
|
|
Scenario: safety-net _resolve_source returns env_var when env var is set
|
|
Given a safety-net isolated temp config directory
|
|
And the safety-net env var "CLEVERAGENTS_LOG_LEVEL" is set to "WARNING"
|
|
When the safety-net source resolver checks key "core.log.level"
|
|
Then the safety-net resolved source should be "env_var"
|
|
|
|
# =====================================================================
|
|
# _resolution_chain (L177-209)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net _resolution_chain returns five-entry list
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net chain builder builds chain for key "core.log.level"
|
|
Then the safety-net chain should have exactly 5 entries
|
|
And the safety-net chain sources should be "cli_flag, env_var, project, global, default"
|
|
|
|
# =====================================================================
|
|
# config_set - type coercion paths (L241-248)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config set coerces boolean true
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "index.auto-reindex" to value "true" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON field "value" should be boolean true
|
|
|
|
Scenario: safety-net config set coerces boolean false
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "index.auto-reindex" to value "false" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON field "value" should be boolean false
|
|
|
|
Scenario: safety-net config set coerces integer value
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "plan.concurrency" to value "8080" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON field "value" should be integer 8080
|
|
|
|
Scenario: safety-net config set coerces float value
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "plan.budget.warn-threshold" to value "3.14" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON field "value" should be float 3.14
|
|
|
|
Scenario: safety-net config set keeps string when not numeric or bool
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "core.log.level" to value "DEBUG" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON field "value" should be string "DEBUG"
|
|
|
|
Scenario: safety-net config set captures previous value field
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "core.log.level" to value "DEBUG" with format "json"
|
|
Then the safety-net set output should be valid JSON
|
|
And the safety-net set JSON should contain a "previous_value" field
|
|
|
|
Scenario: safety-net config set rich format shows panel output
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI sets key "core.log.level" to value "DEBUG" with format "rich"
|
|
Then the safety-net set rich output should contain "Configuration Updated"
|
|
And the safety-net set rich output should contain "core.log.level"
|
|
|
|
# =====================================================================
|
|
# config_get - rich format (L319-335)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config get rich format displays panel and chain
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI gets key "core.log.level" with format "rich" and verbose
|
|
Then the safety-net get rich output should contain "Configuration Value"
|
|
And the safety-net get rich output should contain "Resolution chain"
|
|
|
|
Scenario: safety-net config get yaml format produces valid YAML
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI gets key "core.log.level" with format "yaml"
|
|
Then the safety-net get output should be valid YAML
|
|
And the safety-net get YAML should contain key "key"
|
|
|
|
Scenario: safety-net config get json format includes type field
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI gets key "core.log.level" with format "json" and verbose
|
|
Then the safety-net get output should be valid JSON with type field
|
|
|
|
# =====================================================================
|
|
# config_list - YAML and plain formats (L426-431)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config list yaml format produces valid YAML
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI lists config with format "yaml"
|
|
Then the safety-net list output should be valid YAML list
|
|
|
|
Scenario: safety-net config list plain format produces text output
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI lists config with format "plain"
|
|
Then the safety-net list plain output should contain key-value lines
|
|
|
|
Scenario: safety-net config list table format produces table output
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI lists config with format "table"
|
|
Then the safety-net list table output should contain column headers
|
|
|
|
# =====================================================================
|
|
# config_list - combined key + value filters (L391-399)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config list with both key and value filter
|
|
Given a safety-net isolated temp config directory
|
|
When the safety-net CLI lists config with key pattern "log" and value filter "INFO"
|
|
Then the safety-net combined filter result should succeed
|
|
|
|
# =====================================================================
|
|
# config_list - Path serialisation in non-rich (L428-429)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config list json format serialises Path values to strings
|
|
Given a safety-net isolated temp config directory
|
|
And safety-net settings fields include a Path-typed value
|
|
When the safety-net CLI lists config with format "json"
|
|
Then the safety-net list JSON output should not contain "PosixPath" or "WindowsPath"
|
|
|
|
# =====================================================================
|
|
# config_list - secret masking in list (L408-411)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config list masks secret values by default
|
|
Given a safety-net isolated temp config directory
|
|
And the safety-net env var "AZURE_OPENAI_API_KEY" is set to "test-secret-value"
|
|
When the safety-net CLI lists all config in json format
|
|
Then the safety-net list JSON should contain masked "****" for key "provider.azure.api-key"
|
|
|
|
Scenario: safety-net config list reveals secrets with show-secrets flag
|
|
Given a safety-net isolated temp config directory
|
|
And the safety-net env var "AZURE_OPENAI_API_KEY" is set to "test-secret-value"
|
|
When the safety-net CLI lists all config in json format with show-secrets
|
|
Then the safety-net list JSON should contain value "test-secret-value" for key "provider.azure.api-key"
|
|
|
|
# =====================================================================
|
|
# config_list - modified flag (L405-407)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config list marks modified values
|
|
Given a safety-net isolated temp config directory
|
|
And safety-net settings fields have a value different from default
|
|
When the safety-net CLI lists all config in json format
|
|
Then the safety-net list JSON should include a modified flag set to true
|
|
|
|
# =====================================================================
|
|
# config_get - resolution chain active marker (L333)
|
|
# =====================================================================
|
|
|
|
Scenario: safety-net config get rich format marks active source in chain
|
|
Given a safety-net isolated temp config directory
|
|
And a safety-net patched console for capturing rich output
|
|
When the safety-net CLI gets key "core.log.level" with format "rich" and verbose
|
|
Then the safety-net captured console output should contain "active"
|