@retry @wiring
Feature: Retry Policy Wiring for Services
  As a developer
  I want retry policies wired into service layer operations
  So that services are resilient to transient failures

  Background:
    Given I have the retry policy wiring module imported

  # -------------------------------------------------------------------
  # Registry Configuration
  # -------------------------------------------------------------------

  @registry
  Scenario: RetryPolicyConfig validates default construction
    When I create a default RetryPolicyConfig
    Then the retry policy should have max_attempts 3
    And the retry policy should have base_delay 1.0
    And the retry policy should have max_delay 60.0
    And the retry policy should have jitter True
    And the retry policy backoff_strategy should be exponential

  @registry
  Scenario: RetryPolicyConfig enforces max_delay >= base_delay
    When I create a RetryPolicyConfig with base_delay 10.0 and max_delay 5.0
    Then a validation error should be raised for max_delay

  @registry
  Scenario: CircuitBreakerConfig validates default construction
    When I create a default CircuitBreakerConfig
    Then the circuit breaker config should have failure_threshold 5
    And the circuit breaker config should have recovery_timeout 60.0
    And the circuit breaker config should have half_open_max_successes 2
    And the circuit breaker config should have cooldown_seconds 30.0
    And the circuit breaker config should be enabled

  @registry
  Scenario: ServiceRetryPolicy binds retry and circuit breaker to service
    When I create a ServiceRetryPolicy for "plan_service"
    Then the policy service_name should be "plan_service"
    And the policy should have a retry config
    And the policy should have a circuit breaker config

  @registry
  Scenario: ServiceRetryPolicyRegistry returns defaults for known services
    Given I have a ServiceRetryPolicyRegistry
    When I query the policy for "plan_service"
    Then the returned policy should have service_name "plan_service"
    And the returned policy should have a non-empty description

  @registry
  Scenario: ServiceRetryPolicyRegistry returns auto-generated default for unknown services
    Given I have a ServiceRetryPolicyRegistry
    When I query the policy for "unknown_service"
    Then the returned policy should have service_name "unknown_service"
    And the returned policy description should contain "Auto-generated"

  @registry
  Scenario: ServiceRetryPolicyRegistry applies overrides
    Given I have a ServiceRetryPolicyRegistry
    When I apply overrides setting plan_service max_attempts to 7
    Then the plan_service policy should have max_attempts 7

  @registry
  Scenario: ServiceRetryPolicyRegistry register adds new policy
    Given I have a ServiceRetryPolicyRegistry
    When I register a custom policy for "custom_service"
    Then "custom_service" should appear in registered_services

  @registry
  Scenario: ServiceRetryPolicyRegistry all_policies returns snapshot
    Given I have a ServiceRetryPolicyRegistry
    When I call all_policies
    Then the all_policies snapshot should contain known service names

  @registry
  Scenario: Per-service policy defaults with config overrides
    Given I have Settings with retry_service_overrides setting plan_service max_attempts to 10
    When I create a ServiceRetryWiring from those retry Settings
    Then the plan_service policy should have max_attempts 10

  @registry
  Scenario: Unknown service policy is cached on first get
    Given I have a ServiceRetryPolicyRegistry
    When I query the policy for "brand_new_service" twice
    Then both queries should return the same policy object

  @registry
  Scenario: apply_overrides rejects service_name mutation
    Given I have a ServiceRetryPolicyRegistry
    When I apply overrides with service_name mutation for "plan_service"
    Then the plan_service policy service_name should still be "plan_service"

  @registry
  Scenario: apply_overrides catches invalid override gracefully
    Given I have a ServiceRetryPolicyRegistry
    When I apply overrides with invalid data for "plan_service"
    Then the plan_service policy should retain its original max_attempts

  @registry
  Scenario: apply_overrides handles non-dict override values gracefully
    Given I have a ServiceRetryPolicyRegistry
    When I apply non-dict overrides for "plan_service"
    Then the plan_service policy should be unchanged after non-dict override

  @registry
  Scenario: Deep-copied defaults prevent cross-policy mutation
    Given I have a fresh ServiceRetryPolicyRegistry
    When I mutate one service policy retry config directly
    Then other service policies should not be affected by the mutation

  @registry
  Scenario: Auto-generated unknown service policies do not share mutable defaults
    Given I have a fresh ServiceRetryPolicyRegistry
    When I get auto-generated policies for two unknown services
    And I mutate the first unknown service retry config
    Then the second unknown service retry config should be unaffected
    And the module-level DEFAULT_DATABASE_RETRY should be unaffected

  # -------------------------------------------------------------------
  # Settings Integration
  # -------------------------------------------------------------------

  @settings
  Scenario: ServiceRetryWiring initialises from Settings
    Given I have default Settings
    When I create a ServiceRetryWiring from Settings
    Then the wiring should have circuit breakers for known services
    And the wiring registry should contain known service policies

  @settings
  Scenario: Settings contain retry configuration keys
    Given I have default Settings
    Then the settings should have retry_max_attempts field
    And the settings should have retry_base_delay field
    And the settings should have retry_max_delay field
    And the settings should have retry_jitter field
    And the settings should have retry_backoff_strategy field
    And the settings should have circuit_breaker_failure_threshold field
    And the settings should have circuit_breaker_recovery_timeout field
    And the settings should have circuit_breaker_cooldown field
    And the settings should have retry_service_overrides field

  @settings
  Scenario: Invalid JSON in retry_service_overrides is ignored gracefully
    Given I have Settings with retry_service_overrides set to "not-valid-json"
    When I create a ServiceRetryWiring from those gracefully-invalid Settings
    Then the wiring should initialise without errors

  @settings
  Scenario: Non-dict JSON in retry_service_overrides is ignored gracefully
    Given I have Settings with retry_service_overrides set to "[1, 2, 3]"
    When I create a ServiceRetryWiring from those gracefully-invalid Settings
    Then the wiring should initialise without errors

  @settings
  Scenario: Global retry_backoff_strategy propagated via settings defaults
    Given I have Settings with non-default retry_backoff_strategy "fixed"
    When I create a ServiceRetryWiring from those backoff-strategy Settings
    Then all service policies should have backoff_strategy "fixed"

  # -------------------------------------------------------------------
  # Retry Execution
  # -------------------------------------------------------------------

  @retry-execution
  Scenario: ServiceRetryWiring executes idempotent operation with retry
    Given I have a ServiceRetryWiring instance
    And I have a function that fails once then succeeds
    When I execute the function through the wiring as an idempotent operation
    Then the function should succeed after retry
    And the function should have been called 2 times

  @retry-execution
  Scenario: ServiceRetryWiring skips retry for non-idempotent operations
    Given I have a ServiceRetryWiring instance
    And I have a function that fails on first call
    When I execute the function through the wiring as a non-idempotent operation
    Then the function should fail immediately
    And the function should have been called exactly 1 time

  @retry-execution
  Scenario: Structured logging emitted on retry attempts
    Given I have a ServiceRetryWiring instance
    And I have a function that fails once then succeeds
    When I execute the function through the wiring with logging captured
    Then structured retry log messages should be present

  @retry-execution
  Scenario: Retry exhaustion propagates the last exception
    Given I have a ServiceRetryWiring instance
    And I have a function that always fails with RuntimeError
    When I execute the always-failing function through the wiring as idempotent
    Then the RuntimeError should propagate after exhausting retries
    And the always-failing function should have been called max_attempts times

  @retry-execution
  Scenario: Retry amplification guard prevents nested retries
    Given I have a ServiceRetryWiring instance
    When I execute a nested retry operation through the wiring
    Then the inner function should have been called only once per outer attempt

  @retry-execution
  Scenario: Decorator nesting guard prevents retry amplification
    Given I have a ServiceRetryWiring instance
    When I call a decorated function that nests another decorated call
    Then the inner decorated call should execute without retries

  @retry-execution
  Scenario: Decorator total timeout parameter is honoured
    Given I have a ServiceRetryWiring instance
    When I create a retry_service_operation decorator with total_timeout 1.0
    Then the decorator should accept the total_timeout parameter

  @retry-execution
  Scenario: wrap_service_method decorator executes the wrapped function
    Given I have a ServiceRetryWiring instance
    When I decorate and call a function via wrap_service_method
    Then the wrapped function should return its expected value

  @retry-execution
  Scenario: wrap_service_method returns cached decorator on second call
    Given I have a ServiceRetryWiring instance
    When I call wrap_service_method twice with the same arguments
    Then both wrap_service_method calls should return the same object

  @retry-execution
  Scenario: _get_wait_strategy lazy-builds for unregistered service name
    Given I have a ServiceRetryWiring instance
    When I execute a succeeding function for an unregistered service name
    Then the wait strategy cache should contain the unregistered service

  # -------------------------------------------------------------------
  # Circuit Breaker
  # -------------------------------------------------------------------

  @circuit-breaker
  Scenario: ServiceRetryWiring respects circuit breaker open state
    Given I have a ServiceRetryWiring instance
    And the circuit breaker for "plan_service" is open
    When I execute a function through the wiring for "plan_service"
    Then a CircuitBreakerOpen exception should be raised

  @circuit-breaker
  Scenario: ServiceRetryWiring resets circuit breaker
    Given I have a ServiceRetryWiring instance
    And the circuit breaker for "plan_service" is open
    When I reset the circuit breaker for "plan_service"
    Then the circuit breaker for "plan_service" should be closed

  @circuit-breaker
  Scenario: ServiceRetryWiring checks circuit open status
    Given I have a ServiceRetryWiring instance
    When I check the circuit status for "plan_service"
    Then the circuit should not be open

  @circuit-breaker
  Scenario: Circuit breaker half-open recovery with consecutive successes
    Given I have a ServiceRetryWiring instance
    And the circuit breaker for "plan_service" is in half-open state
    When I execute two successful calls through the wiring for "plan_service"
    Then the circuit breaker for "plan_service" should be closed

  @circuit-breaker
  Scenario: Circuit breaker wires cooldown_seconds from config
    Given I have a ServiceRetryWiring instance
    Then the circuit breaker for "plan_service" should have cooldown_seconds 30.0

  @circuit-breaker
  Scenario: Circuit breaker wires half_open_max_successes from config
    Given I have a ServiceRetryWiring instance
    Then the circuit breaker for "plan_service" should have half_open_max_successes 2

  @circuit-breaker
  Scenario: Circuit breaker with circuit_breaker=None path executes without retry protection
    Given I have a ServiceRetryWiring instance for a service without circuit breaker
    When I execute a function through the wiring for the unprotected service
    Then the function should execute successfully without circuit breaker

  @circuit-breaker
  Scenario: reset_circuit clears last_failure_time
    Given I have a ServiceRetryWiring instance
    And the circuit breaker for "plan_service" is open
    When I reset the circuit breaker for "plan_service"
    Then the circuit breaker for "plan_service" last_failure_time should be None

  @circuit-breaker
  Scenario: _on_success does not close circuit opened by concurrent failure
    Given I have a standalone CircuitBreaker instance
    And the standalone circuit breaker state is set to "open"
    When I call _on_success on the standalone circuit breaker
    Then the standalone circuit breaker state should remain "open"

  @circuit-breaker
  Scenario: Half-open probe limit blocks excess concurrent requests
    Given I have a standalone CircuitBreaker instance in half-open with 0 permits
    When I call the standalone circuit breaker with a test function
    Then a CircuitBreakerOpen exception should be raised for probe limit

  @circuit-breaker
  Scenario: Circuit breaker only counts expected_exception for failure tracking
    Given I have a standalone CircuitBreaker with expected_exception IOError
    When I call it with a function that raises RuntimeError
    Then the standalone circuit breaker failure_count should be 0

  @circuit-breaker
  Scenario: Circuit breaker counts matching expected_exception failures
    Given I have a standalone CircuitBreaker with expected_exception IOError
    When I call it with a function that raises IOError
    Then the standalone circuit breaker failure_count should be 1

  @circuit-breaker
  Scenario: Half-open failure re-opens the circuit breaker
    Given I have a standalone CircuitBreaker instance in half-open state with permits
    When I call it with a function that raises an expected exception
    Then the standalone circuit breaker state should be "open"

  @circuit-breaker
  Scenario: Cooldown prevents premature half-open transitions
    Given I have a standalone CircuitBreaker with a recent half-open attempt
    When I check _should_attempt_reset on the standalone circuit breaker
    Then the reset should be denied due to cooldown

  @circuit-breaker
  Scenario: failure_count is reset when entering half-open from open
    Given I have a standalone CircuitBreaker that transitions to half-open
    When the circuit breaker transitions from open to half-open
    Then the circuit breaker failure_count should be 0

  @circuit-breaker @negative
  Scenario: Disabled circuit breaker does not block calls
    Given I have a ServiceRetryWiring instance with disabled circuit breaker
    When I execute a function through the wiring for the disabled-CB service
    Then the function should execute successfully without circuit breaker protection

  @circuit-breaker @negative
  Scenario: Concurrent failures open the circuit breaker
    Given I have a standalone CircuitBreaker with threshold 3
    When 3 concurrent threads trigger failures on the circuit breaker
    Then the standalone circuit breaker state should be "open"

  @circuit-breaker @negative
  Scenario: Retry exhaustion opens the circuit breaker
    Given I have a ServiceRetryWiring instance with low CB threshold
    And I have a function that always fails with RuntimeError
    When I execute the always-failing function through the wiring as idempotent
    Then the circuit breaker for "plan_service" should be open after exhaustion

  # -------------------------------------------------------------------
  # Sanitization
  # -------------------------------------------------------------------

  @sanitization
  Scenario Outline: Sensitive values are sanitized from error messages
    When I sanitize an error containing <sensitive_type> with secret "<secret>"
    Then the sanitized output should not contain "<secret>"
    And the sanitized output should contain "<redacted_marker>"

    Examples:
      | sensitive_type   | secret                  | redacted_marker       |
      | database_url     | s3cret                  | ***@                  |
      | api_key          | sk-12345abcdef          | api_key=***           |
      | auth_header      | sk-live-abc123xyz       | Authorization: ***    |
      | private_key      | AAAA-BBBB-CCCC-DDDD     | private_key=***       |
      | connection_string| Server=db;Password=s3cret| connection_string=*** |
      | access_key       | AKIAIOSFODNN7EXAMPLE    | access_key=***        |

  @sanitization
  Scenario: Long error messages are truncated at 200 characters
    When I sanitize an error message longer than 200 characters
    Then the sanitized message should be truncated with ellipsis
    And the sanitized message length should not exceed 203

  @sanitization
  Scenario: _sanitize_error_message returns None for None input
    When I sanitize a None error
    Then the sanitized result should be None

  # -------------------------------------------------------------------
  # Read-Only Operations
  # -------------------------------------------------------------------

  @read-only
  Scenario: Guard prevents retries on tool writes in read-only plans
    Given I have the read-only plan operation guard
    When I check a read_only=True operation
    Then the retry guard should indicate read-only
    When I check a read_only=False operation
    Then the retry guard should indicate not read-only

  @read-only
  Scenario: plan_phase guard returns True for strategize phase
    Given I have the read-only plan operation guard
    When I check an operation with plan_phase "strategize"
    Then the retry guard should indicate read-only

  @read-only
  Scenario: plan_phase guard returns False for execute phase
    Given I have the read-only plan operation guard
    When I check an operation with plan_phase "execute"
    Then the retry guard should indicate not read-only

  @read-only
  Scenario: plan_phase guard returns False for empty phase
    Given I have the read-only plan operation guard
    When I check an operation with empty plan_phase
    Then the retry guard should indicate not read-only

  @read-only
  Scenario: plan_phase guard returns False for apply phase
    Given I have the read-only plan operation guard
    When I check an operation with plan_phase "apply"
    Then the retry guard should indicate not read-only

  @read-only
  Scenario: plan_phase guard returns False for non-string truthy values
    Given I have the read-only plan operation guard
    When I check an operation with non-string plan_phase True
    Then the retry guard should indicate not read-only

  # -------------------------------------------------------------------
  # Enums and Defaults
  # -------------------------------------------------------------------

  @enums
  Scenario: RetryCategory enum covers all categories
    When I list all RetryCategory values
    Then the categories should include "network"
    And the categories should include "provider"
    And the categories should include "database"
    And the categories should include "file_operation"

  @enums
  Scenario: RetryStrategy enum covers all strategies
    When I list all RetryStrategy values
    Then the strategies should include "exponential"
    And the strategies should include "linear"
    And the strategies should include "fixed"
    And the strategies should include "jitter"
    And the strategies should include "none"

  @enums
  Scenario: Category defaults map all categories
    When I inspect CATEGORY_DEFAULTS
    Then CATEGORY_DEFAULTS should have entries for all RetryCategory values

  # -------------------------------------------------------------------
  # Backoff Strategy Builders
  # -------------------------------------------------------------------

  @backoff
  Scenario: Exponential backoff with jitter builds correctly
    When I build a wait strategy with strategy "exponential" and jitter True
    Then the wait strategy should be an exponential jitter strategy

  @backoff
  Scenario: Exponential backoff without jitter builds correctly
    When I build a wait strategy with strategy "exponential" and jitter False
    Then the wait strategy should be an exponential strategy

  @backoff
  Scenario: Exponential backoff enforces minimum base delay
    When I build a wait strategy with strategy "exponential" base_delay 0.0 and jitter False
    Then the effective base delay should be at least 0.1

  @backoff
  Scenario: Linear backoff enforces 2-second minimum delay
    When I build a linear wait strategy with base_delay 0.5
    Then the linear wait value should be at least 2.0

  @backoff
  Scenario: Fixed backoff enforces 0.1-second minimum delay
    When I build a fixed wait strategy with base_delay 0.0
    Then the fixed wait value should be at least 0.1

  @backoff
  Scenario: _build_wait_strategy handles jitter strategy
    When I build a jitter wait strategy with base_delay 1.0
    Then the jitter wait strategy should not be None

  @backoff
  Scenario: _build_wait_strategy handles none strategy
    When I build a none wait strategy
    Then the none wait strategy should produce zero-second wait

  @backoff
  Scenario: Jitter strategy enforces minimum delay floor
    When I build a jitter wait strategy with base_delay 0.0
    Then the jitter wait value should be at least 0.1

  # -------------------------------------------------------------------
  # Async Operations
  # -------------------------------------------------------------------

  @async
  Scenario: Async retry_service_operation retries correctly
    Given I have a ServiceRetryWiring instance
    And I have an async function that fails once then succeeds
    When I execute the async function through async_execute as idempotent
    Then the async function should succeed after retry
    And the async function should have been called 2 times

  @async
  Scenario: Async execute skips retry for non-idempotent operations
    Given I have a ServiceRetryWiring instance
    And I have an async function that always fails
    When I execute the async function through async_execute as non-idempotent
    Then the async function should fail immediately with RuntimeError
    And the async function call count should be 1

  @async
  Scenario: async_execute raises CircuitBreakerOpen for open circuit
    Given I have a ServiceRetryWiring instance
    And the circuit breaker for "plan_service" is open
    When I async_execute a function for "plan_service" with open circuit
    Then a CircuitBreakerOpen exception should be raised for async_execute

  @async
  Scenario: async_execute nesting guard prevents retry amplification
    Given I have a ServiceRetryWiring instance
    When I async_execute a nested retry operation through the wiring
    Then the async inner function should have been called only once

  @async
  Scenario: Async retry exhaustion propagates the last exception
    Given I have a ServiceRetryWiring instance
    And I have an async function that always fails
    When I async_execute the always-failing function as idempotent
    Then the async RuntimeError should propagate after exhausting retries
    And the async always-failing function should have been called max_attempts times

  # -------------------------------------------------------------------
  # Async Callable Detection
  # -------------------------------------------------------------------

  @async
  Scenario: _is_async_callable detects partial-wrapped async functions
    When I check _is_async_callable with a partial-wrapped async function
    Then _is_async_callable should return True

  @async
  Scenario: _is_async_callable detects callable objects with async __call__
    When I check _is_async_callable with an async callable object
    Then _is_async_callable should return True

  @async
  Scenario: _is_async_callable returns False for sync functions
    When I check _is_async_callable with a regular sync function
    Then _is_async_callable should return False

  @async
  Scenario: _is_async_callable returns False for non-callable objects
    When I check _is_async_callable with a non-callable object
    Then _is_async_callable should return False

  # -------------------------------------------------------------------
  # Error Handling & Validation
  # -------------------------------------------------------------------

  @error-handling @negative
  Scenario: RetryPolicyConfig rejects max_attempts below 1
    When I attempt to create a RetryPolicyConfig with max_attempts 0
    Then a validation error should be raised for invalid retry config

  @error-handling @negative
  Scenario: CircuitBreakerConfig rejects failure_threshold below 1
    When I attempt to create a CircuitBreakerConfig with failure_threshold 0
    Then a validation error should be raised for invalid circuit breaker config

  @error-handling @negative
  Scenario: CircuitBreakerConfig rejects recovery_timeout below 1
    When I attempt to create a CircuitBreakerConfig with recovery_timeout 0.0
    Then a validation error should be raised for invalid circuit breaker config

  @error-handling @negative
  Scenario: ServiceRetryPolicy rejects empty service_name
    When I attempt to create a ServiceRetryPolicy with empty service_name
    Then a validation error should be raised for invalid service policy

  @error-handling @negative
  Scenario: Negative base_delay is rejected
    When I attempt to create a RetryPolicyConfig with base_delay -1.0
    Then a validation error should be raised for invalid retry config

  @retry-execution
  Scenario: ServiceRetryWiring wrap_service_method produces a decorator
    Given I have a ServiceRetryWiring instance
    When I get a decorator from wrap_service_method for "session_service"
    Then the decorator should be callable
