cleveragents-core/tests/features/configuration_management.feature

Feature: Configuration Management
  As a CleverAgents developer
  I want robust configuration loading and validation
  So that the application handles various configuration scenarios correctly

  Background:
    Given the configuration manager is initialized

  Scenario: Basic configuration loading and validation
    Given I have a configuration file "basic.yaml":
      """
      agents:
        test_agent:
          type: llm
          config:
            provider: openai
            model: gpt-3.5-turbo

      routes:
        test_route:
          type: stream
          stream_type: cold
          operators:
            - type: map
              params:
                agent: test_agent
          publications:
            - __output__

      cleveragents:
        default_router: test_route

      merges:
        - sources: [__input__]
          target: test_route
      """
    When I load the configuration from "basic.yaml"
    Then the configuration should be loaded successfully
    And the configuration should contain 1 agent
    And the configuration should contain 1 route
    And validation should pass

  Scenario: Environment variable interpolation
    Given I have a configuration file "env_vars.yaml":
      """
      agents:
        llm_agent:
          type: llm
          config:
            provider: ${LLM_PROVIDER}
            model: ${LLM_MODEL}
            api_key: ${API_KEY}
            temperature: ${TEMPERATURE:0.7}
            max_tokens: ${MAX_TOKENS:1000}

      routes:
        main_route:
          type: stream
          operators:
            - type: map
              params:
                agent: llm_agent
          publications:
            - __output__

      cleveragents:
        default_router: main_route
        debug: ${DEBUG_MODE:false}
        timeout: ${TIMEOUT:2}

      merges:
        - sources: [__input__]
          target: main_route
      """
    And I set environment variables:
      | variable     | value           |
      | LLM_PROVIDER | openai          |
      | LLM_MODEL    | gpt-4           |
      | API_KEY      | sk-test123      |
      | DEBUG_MODE   | true            |
    When I load the configuration from "env_vars.yaml"
    Then the configuration should be loaded successfully
    And the agent config should have provider "openai"
    And the agent config should have model "gpt-4"
    And the agent config should have api_key "sk-test123"
    And the agent config should have temperature 0.7
    And the agent config should have max_tokens 1000
    And the cleveragents config should have debug true
    And the cleveragents config should have timeout 30

  Scenario: Missing environment variable error handling
    Given I have a configuration file "missing_env.yaml":
      """
      agents:
        env_agent:
          type: llm
          config:
            api_key: ${MISSING_API_KEY}

      routes:
        test_route:
          type: stream
          operators: []
          publications: []

      cleveragents:
        default_router: test_route
      """
    When I attempt to load the configuration from "missing_env.yaml"
    Then the configuration loading should fail
    And the configuration error should mention "Environment variable 'MISSING_API_KEY' is not set"

  Scenario: Multiple configuration file merging
    Given I have configuration files:
      | filename      | content                                    |
      | base.yaml     | base configuration with agents section    |
      | routes.yaml   | routes configuration section              |
      | overrides.yaml| configuration overrides and customizations|
    And the base configuration contains:
      """
      agents:
        base_agent:
          type: llm
          config:
            provider: openai
            model: gpt-3.5-turbo
            temperature: 0.5

      cleveragents:
        debug: false
        timeout: 2
      """
    And the routes configuration contains:
      """
      routes:
        main_stream:
          type: stream
          stream_type: cold
          operators:
            - type: map
              params:
                agent: base_agent
          publications:
            - __output__

      cleveragents:
        default_router: main_stream

      merges:
        - sources: [__input__]
          target: main_stream
      """
    And the overrides configuration contains:
      """
      agents:
        base_agent:
          config:
            temperature: 0.8
            max_tokens: 2000
        
        additional_agent:
          type: tool
          config:
            tools: ["echo"]

      cleveragents:
        debug: true
      """
    When I load configurations from ["base.yaml", "routes.yaml", "overrides.yaml"]
    Then the configuration should be loaded successfully
    And the merged configuration should contain 2 agents
    And agent "base_agent" should have temperature 0.8
    And agent "base_agent" should have max_tokens 2000
    And agent "additional_agent" should exist
    And the cleveragents config should have debug true
    And the cleveragents config should have timeout 30

  Scenario: Deep merge behavior validation
    Given I have a base configuration:
      """
      agents:
        complex_agent:
          type: llm
          config:
            provider: openai
            model: gpt-3.5-turbo
            parameters:
              temperature: 0.7
              max_tokens: 1000
              stop_sequences: ["STOP"]
            metadata:
              version: "1.0"
              tags: ["base"]
      """
    And I have an override configuration:
      """
      agents:
        complex_agent:
          config:
            parameters:
              temperature: 0.9
              top_p: 0.95
              stop_sequences: ["END", "FINISH"]
            metadata:
              tags: ["override", "production"]
              environment: "prod"
      """
    When I merge the configurations
    Then the merged agent should have:
      | path                          | value                      |
      | config.provider               | openai                     |
      | config.model                  | gpt-3.5-turbo             |
      | config.parameters.temperature | 0.9                        |
      | config.parameters.max_tokens  | 1000                       |
      | config.parameters.top_p       | 0.95                       |
      | config.parameters.stop_sequences | ["END", "FINISH"]      |
      | config.metadata.version       | 1.0                        |
      | config.metadata.tags          | ["override", "production"] |
      | config.metadata.environment   | prod                       |

  Scenario: Configuration validation error scenarios
    Given I have invalid configuration scenarios:
      | scenario                | config_content                          | expected_error                           |
      | missing_agents_section  | routes: {}                             | Configuration must contain 'agents'      |
      | empty_agents           | agents: {}; routes: {}                  | Configuration must have 'routes'         |
      | invalid_agent_type     | agents: {bad: {type: invalid}}         | Agent 'bad' is missing required 'type'  |
      | missing_default_router | agents: {a: {type: llm}}; routes: {r: {}}| must specify 'default_router'          |
      | router_not_found       | agents: {a: {type: llm}}; routes: {r: {}}; cleveragents: {default_router: missing} | Default router 'missing' not found |
    When I attempt to load each invalid configuration
    Then each should fail with the corresponding error message

  Scenario: Configuration path-based access
    Given I have a loaded configuration with nested structure
    When I access configuration values using dot notation:
      | path                               | expected_value |
      | agents.test_agent.type            | llm            |
      | agents.test_agent.config.provider | openai         |
      | routes.main_route.type            | stream         |
      | cleveragents.default_router       | main_route     |
      | nonexistent.path                  | None           |
    Then each path should return the expected value
    When I set a configuration value using path "agents.test_agent.config.temperature"
    Then the value should be updated in the configuration
    And subsequent access should return the new value

  Scenario: Configuration schema validation edge cases
    Given I have edge case configurations:
      | case                    | description                                    |
      | null_config_section     | Agent with null config section               |
      | string_instead_of_dict  | Agent config as string instead of dictionary |
      | nested_null_values      | Deep nested structure with null values       |
      | mixed_data_types        | Configuration with mixed data types          |
      | unicode_characters      | Configuration with Unicode and special chars |
    When I validate each edge case configuration
    Then validation should handle each case appropriately
    And error messages should be clear and actionable

  Scenario: Configuration serialization and deserialization
    Given I have a complex loaded configuration
    When I serialize the configuration to JSON
    Then the JSON should be valid and complete
    When I convert the configuration to dictionary
    Then all nested structures should be preserved
    And sensitive information should be handled appropriately

  Scenario: Configuration hot-reloading and updates
    Given I have a loaded configuration
    When I modify the source configuration file
    And I reload the configuration
    Then the changes should be reflected
    And dependent components should be notified
    And the system should remain in a consistent state

  Scenario: Configuration validation with templates
    Given I have a configuration with template references:
      """
      templates:
        agents:
          standard_llm:
            type: llm
            config:
              provider: openai
              model: "{{ model_name }}"
              temperature: "{{ temperature | default(0.7) }}"

      agents:
        primary:
          template: standard_llm
          params:
            model_name: gpt-4
            temperature: 0.8
        
        secondary:
          template: standard_llm
          params:
            model_name: gpt-3.5-turbo

      routes:
        test_route:
          type: stream
          operators:
            - type: map
              params:
                agent: primary
          publications:
            - __output__

      cleveragents:
        default_router: test_route
      """
    When I load and validate the configuration
    Then the template should be resolved correctly
    And agent "primary" should have model "gpt-4" and temperature 0.8
    And agent "secondary" should have model "gpt-3.5-turbo" and temperature 0.7
    And the configuration should pass validation