@retry @patterns Feature: Retry Patterns Implementation As a developer I want retry patterns implemented with tenacity So that operations can be resilient to transient failures Background: Given I have the retry patterns module imported # ------------------------------------------------------------------- # Basic Retry Strategies # ------------------------------------------------------------------- @exponential Scenario: Basic exponential backoff retry works Given I have a function that fails 2 times then succeeds When I apply exponential backoff retry with max 3 attempts Then the function should eventually succeed And the function should be called 3 times @exponential @async Scenario: Async exponential backoff retry works Given I have an async function that fails 2 times then succeeds When I apply async exponential backoff retry with max 3 attempts Then the async function should eventually succeed And the function should be called 3 times @timeout Scenario: Retry with timeout retries transient failures without waiting Given I have a function that fails 1 times then succeeds When I apply retry with timeout of 3 attempts and 0.01 seconds Then the function should eventually succeed And the function should be called 2 times @jitter Scenario: Retry with jitter prevents thundering herd Given I have multiple concurrent operations When I apply retry with jitter Then the retries should have random delays And operations should not retry simultaneously @result-retry Scenario: Retry on result retries when predicate flags response Given I have sequential result payloads requiring a retry And I have a retry predicate that checks for retry flag When I apply retry on result with max 3 attempts Then the decorated function should return the successful payload And the function should be called 2 times # ------------------------------------------------------------------- # Network / Provider / File Retry Patterns # ------------------------------------------------------------------- @network Scenario: Network operation retry handles connection errors Given I have a function that raises NetworkError twice When I apply network retry pattern Then the function should retry on NetworkError And the function should be called 3 times maximum @provider Scenario: Provider operation retry handles rate limits Given I have a function that raises RateLimitError When I apply provider retry pattern Then the function should retry with exponential jitter And the function should respect rate limit backoff @file Scenario: File operation retry recovers from transient OSError Given I have a file operation that fails with OSError once When I apply file retry pattern Then the file operation should eventually succeed And the file operation should be invoked 2 times # ------------------------------------------------------------------- # Circuit Breaker # ------------------------------------------------------------------- @circuit-breaker Scenario: Circuit breaker opens after threshold Given I have a circuit breaker with threshold 3 When a function fails 3 times Then the circuit breaker should open And subsequent calls should fail immediately @circuit-breaker Scenario: Circuit breaker resets after timeout Given I have an open circuit breaker When the recovery timeout expires Then the circuit breaker should enter half-open state And successful calls should close the circuit @circuit-breaker Scenario: Circuit breaker resets failure counters after closed success Given I have a circuit breaker with threshold 2 And the circuit breaker has recorded failures When I execute a successful call while the circuit is closed Then the circuit breaker should reset failure tracking after success @circuit-breaker Scenario: Circuit breaker half-open closes after consecutive successes Given I have a half-open circuit breaker with threshold 2 When I record a successful half-open call And I record another successful half-open call Then the circuit breaker should close after consecutive successes @circuit-breaker Scenario: Circuit breaker half-open reopens after failure Given I have a half-open circuit breaker with threshold 2 When I record a successful half-open call And I simulate a half-open failure Then the circuit breaker should reopen after half-open failure @circuit-breaker Scenario: Circuit breaker rejects invalid half_open_max_successes When I create a CircuitBreaker with half_open_max_successes 0 Then a ValueError should be raised for invalid half_open_max_successes @circuit-breaker Scenario: Circuit breaker _should_attempt_reset returns False with no prior failures Given I have a fresh circuit breaker with no failures When I check _should_attempt_reset on the fresh circuit breaker Then _should_attempt_reset should return False # ------------------------------------------------------------------- # Async Circuit Breaker # ------------------------------------------------------------------- @circuit-breaker @async Scenario: Async circuit breaker raises immediately while open Given I have an async circuit breaker with threshold 1 When I trigger an async failure on the circuit breaker Then the async circuit breaker should raise immediately while open @circuit-breaker @async Scenario: Async circuit breaker closes after recovery and successful calls Given I have an async circuit breaker with threshold 1 When I trigger an async failure on the circuit breaker And the recovery timeout expires And I perform two async successful calls after recovery Then the async circuit breaker should close after async successes @circuit-breaker @async Scenario: Async circuit breaker half-open probe limit blocks excess calls Given I have an async circuit breaker in half-open with 0 permits When I make an async call on the half-open circuit breaker Then a CircuitBreakerOpen should be raised for async probe limit @circuit-breaker @async Scenario: Async circuit breaker restores permits on CancelledError Given I have an async circuit breaker in half-open with 1 permit When the async call is cancelled mid-execution Then the half-open permit should be restored # ------------------------------------------------------------------- # Retry Context # ------------------------------------------------------------------- @retry-context Scenario: Retry context tracks attempts Given I have a retry context for "test operation" When I execute a function that fails twice Then the retry context should track 3 attempts And the errors should be recorded @retry-context Scenario: Retry context manager records exceptions raised in block Given I have a retry context for "context block" When I execute a failing block under the retry context manager Then the retry context manager should capture the exception @retry-context @async Scenario: Async retry context manager records exceptions raised in block Given I have a retry context for "async context block" When I execute a failing block under the async retry context manager Then the async retry context manager should capture the exception @retry-context Scenario: Retry context manager leaves errors untouched on success Given I have a retry context for "successful context block" When I execute a successful block under the retry context manager Then the retry context manager should not record errors @retry-context @async Scenario: Async retry context manager leaves errors untouched on success Given I have a retry context for "async successful context block" When I execute a successful block under the async retry context manager Then the async retry context manager should not record errors @retry-context @async Scenario: Retry context async execute handles transient async failures Given I have a retry context for "async execute operation" And I have an async function that fails 2 times then succeeds When I execute the async function with the retry context Then the async retry context execute should succeed after 3 attempts # ------------------------------------------------------------------- # should_retry_result # ------------------------------------------------------------------- @result-retry Scenario: should_retry_result handles error payloads Given I have a response payload with error "timeout detected" When I evaluate the retry decision Then the retry decision should be True @result-retry Scenario: should_retry_result handles server error status codes Given I have a response payload with status code 503 When I evaluate the retry decision Then the retry decision should be True @result-retry Scenario: should_retry_result returns False for None payloads Given I have a result payload set to None When I evaluate the retry decision Then the retry decision should be False @result-retry Scenario: should_retry_result returns False for non-error dictionaries Given I have a response payload with status code 200 When I evaluate the retry decision Then the retry decision should be False @result-retry Scenario: should_retry_result returns False for non-dictionary results Given I have a result payload set to the string "ok" When I evaluate the retry decision Then the retry decision should be False # ------------------------------------------------------------------- # Retry Categories & Logging # ------------------------------------------------------------------- @categories Scenario: Retry categories have correct configurations Given I have the retry categories defined Then network operations should retry 5 times And provider operations should retry 3 times And database operations should retry 3 times And file operations should retry 3 times @categories Scenario: Unknown retry category falls back to default decorator When I request a retry decorator for "unknown-category" And I decorate a function returning "decorated result" Then the decorated function should execute successfully @categories Scenario: Known retry category returns a configured decorator When I request a retry decorator for "network" And I decorate a function returning "network result" Then the decorated function should execute successfully @logging Scenario: Retry logging handles loggers without keyword support Given the retry logger rejects keyword arguments When I invoke logging callbacks for failed and successful attempts Then the fallback logging messages should be recorded @logging Scenario: configure_retry_logging sets tenacity logger levels When I configure retry logging to warning level Then the tenacity loggers should be set to warning # ------------------------------------------------------------------- # Auto-Debug Retry # ------------------------------------------------------------------- @auto-debug Scenario: Auto-debug retry pattern works Given I have a function that fails with errors And I have a debug callback that fixes issues When I apply auto-debug retry Then the function should retry with debug fixes And eventually succeed after debugging @auto-debug Scenario: Auto-debug retry raises the last error message when unresolved Given I have a function that returns persistent error payloads And I have a debug callback that never fixes issues When I apply auto-debug retry expecting failure Then the auto-debug retry should raise the last error message @auto-debug Scenario: Auto-debug retry logs debug callback failures Given I have a function that returns persistent error payloads And I have a debug callback that raises errors When I apply auto-debug retry expecting failure Then the auto-debug retry should raise the debug callback failure And the debug callback failures should be counted @auto-debug Scenario: Auto-debug retry returns result for non-dict success Given I have an async function that returns a non-dict value When I apply auto-debug retry to the non-dict function Then the auto-debug should return the non-dict value directly @auto-debug Scenario: Auto-debug retry returns None when no errors occur and no last_error Given I have an async function that returns error-free dict then stops And I have no debug callback When I apply auto-debug retry to the error-free function Then the auto-debug should return the dict result @auto-debug Scenario: Auto-debug with fix callback that resolves the error Given I have an async function that returns error then succeeds after fix And I have a debug callback that returns fixed When I apply auto-debug retry with fix callback Then the auto-debug should eventually succeed # ------------------------------------------------------------------- # Retry Service Operation Decorators # ------------------------------------------------------------------- @decorator @async Scenario: Async retry_service_operation decorator retries correctly Given I have an async function that fails once then returns ok When I apply retry_service_operation as async and call Then the async decorated function should succeed And the async decorated function should have been called 2 times @decorator @async Scenario: Async retry_service_operation nesting guard skips retries When I call nested async retry_service_operation decorators Then the inner async decorator should execute without retries @decorator Scenario: Sync decorator with circuit breaker handles open Given I have a sync function guarded by retry_service_operation with CB When the circuit breaker opens during sync decorated retry Then the sync decorator should propagate CircuitBreakerOpen @decorator @async Scenario: Async retry_service_operation with circuit breaker and CB open Given I have an async function guarded by retry_service_operation with CB When the circuit breaker opens during async decorated retry Then the async decorator should propagate CircuitBreakerOpen @decorator Scenario: Sync decorator nesting guard delegates to circuit breaker When I call a sync decorated function at max nesting depth with CB Then the sync decorator nesting guard should delegate to CB @decorator Scenario: retry_service_operation non-idempotent returns function as-is When I create a non-idempotent retry_service_operation decorator Then the decorator should return the original function unchanged @decorator @async Scenario: Async decorator nesting guard delegates to circuit breaker When I call an async decorated function at max nesting depth with CB Then the async nesting CB guard should delegate successfully # ------------------------------------------------------------------- # _is_async_callable # ------------------------------------------------------------------- @async Scenario: _is_async_callable detects partial wrapping async callable object When I check _is_async_callable with partial wrapping an async callable object Then _is_async_callable should return True for partial async callable @async Scenario: _is_async_callable detects partial wrapping plain async function When I check _is_async_callable with partial wrapping a plain async function Then _is_async_callable should return True for partial plain async # ------------------------------------------------------------------- # Structured Log Fallbacks # ------------------------------------------------------------------- @logging Scenario: Structured log TypeError fallback for service retry When I trigger the structlog TypeError fallback for service retry logging Then the fallback logging should not raise any exception