cleveragents-core/docs/architecture/decisions/ADR-005-error-handling.md

ADR-005: Error Handling Hierarchy and Exception Strategy

Status

Accepted

Context

CleverAgents needs a structured approach to error handling that:

• Maps cleanly from Go's error patterns
• Supports fail-fast principles from CONTRIBUTING.md
• Provides clear error messages for users
• Enables proper error recovery and retry logic
• Maintains traceability for debugging
• Differentiates between recoverable and fatal errors

The discovery phase identified various error scenarios:

• 33 retry/backoff patterns indicating recoverable errors
• Authentication and authorization failures
• Network and timeout errors
• Validation failures
• Resource conflicts
• System errors

Decision

We will implement a hierarchical exception strategy with domain-specific exceptions that propagate to top-level handlers.

Exception Hierarchy

# cleveragents.core.exceptions

class CleverAgentsError(Exception):
    """Base exception for all CleverAgents errors"""
    def __init__(self, message: str, details: dict = None):
        super().__init__(message)
        self.message = message
        self.details = details or {}

# Domain Exceptions
class DomainError(CleverAgentsError):
    """Base for domain/business logic errors"""
    pass

class ValidationError(DomainError):
    """Data validation failures"""
    pass

class BusinessRuleViolation(DomainError):
    """Business rule violations"""
    pass

class ResourceNotFoundError(DomainError):
    """Requested resource doesn't exist"""
    pass

class ResourceConflictError(DomainError):
    """Resource state conflict"""
    pass

# Infrastructure Exceptions
class InfrastructureError(CleverAgentsError):
    """Base for infrastructure errors"""
    pass

class DatabaseError(InfrastructureError):
    """Database operation failures"""
    pass

class NetworkError(InfrastructureError):
    """Network communication failures"""
    pass

class ExternalServiceError(InfrastructureError):
    """External service failures"""
    pass

# Provider Exceptions
class ProviderError(CleverAgentsError):
    """Base for AI provider errors"""
    pass

class RateLimitError(ProviderError):
    """Rate limit exceeded"""
    def __init__(self, message: str, retry_after: int = None):
        super().__init__(message)
        self.retry_after = retry_after

class ModelNotAvailableError(ProviderError):
    """Model not available or deprecated"""
    pass

class TokenLimitExceededError(ProviderError):
    """Context window exceeded"""
    pass

# Authentication/Authorization
class AuthenticationError(CleverAgentsError):
    """Authentication failures"""
    pass

class AuthorizationError(CleverAgentsError):
    """Authorization failures"""
    pass

# Configuration Errors
class ConfigurationError(CleverAgentsError):
    """Configuration issues"""
    pass

class MissingConfigurationError(ConfigurationError):
    """Required configuration missing"""
    pass

Error Handling Patterns

1. Fail-Fast Principle (from CONTRIBUTING.md)

# DON'T suppress errors
def bad_example():
    try:
        result = risky_operation()
    except Exception:
        return None  # DON'T DO THIS

# DO let exceptions propagate
def good_example():
    # Let exceptions bubble up to handler
    result = risky_operation()
    return process(result)

2. Recovery Patterns

# Retry with backoff for recoverable errors
async def with_retry(func, max_attempts=3):
    for attempt in range(max_attempts):
        try:
            return await func()
        except (NetworkError, RateLimitError) as e:
            if attempt == max_attempts - 1:
                raise
            if isinstance(e, RateLimitError) and e.retry_after:
                await asyncio.sleep(e.retry_after)
            else:
                await asyncio.sleep(2 ** attempt)

3. Context Enhancement

# Add context when re-raising
class PlanService:
    async def create_plan(self, name: str):
        try:
            plan = await self.repository.create(name)
        except DatabaseError as e:
            # Add context and re-raise
            raise DomainError(
                f"Failed to create plan '{name}'",
                details={'original_error': str(e)}
            ) from e

4. Top-Level Handlers

CLI Handler

# cleveragents.cli.error_handler
def handle_cli_error(e: Exception) -> int:
    """Convert exceptions to exit codes and user messages"""
    if isinstance(e, ValidationError):
        click.echo(f"Validation error: {e.message}", err=True)
        return 1
    elif isinstance(e, AuthenticationError):
        click.echo(f"Authentication failed: {e.message}", err=True)
        return 2
    elif isinstance(e, ResourceNotFoundError):
        click.echo(f"Not found: {e.message}", err=True)
        return 3
    elif isinstance(e, NetworkError):
        click.echo(f"Network error: {e.message}", err=True)
        click.echo("Check your internet connection and try again", err=True)
        return 4
    else:
        # Unexpected error - log full traceback
        logger.exception("Unexpected error")
        click.echo(f"Error: {e}", err=True)
        return 99

Server Handler

# cleveragents.runtime.error_handler
from fastapi import Request, status
from fastapi.responses import JSONResponse

async def handle_api_error(request: Request, exc: Exception):
    """Convert exceptions to HTTP responses"""
    if isinstance(exc, ValidationError):
        return JSONResponse(
            status_code=status.HTTP_400_BAD_REQUEST,
            content={"error": exc.message, "details": exc.details}
        )
    elif isinstance(exc, AuthenticationError):
        return JSONResponse(
            status_code=status.HTTP_401_UNAUTHORIZED,
            content={"error": exc.message}
        )
    elif isinstance(exc, AuthorizationError):
        return JSONResponse(
            status_code=status.HTTP_403_FORBIDDEN,
            content={"error": exc.message}
        )
    elif isinstance(exc, ResourceNotFoundError):
        return JSONResponse(
            status_code=status.HTTP_404_NOT_FOUND,
            content={"error": exc.message}
        )
    elif isinstance(exc, ResourceConflictError):
        return JSONResponse(
            status_code=status.HTTP_409_CONFLICT,
            content={"error": exc.message}
        )
    elif isinstance(exc, RateLimitError):
        return JSONResponse(
            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
            content={"error": exc.message},
            headers={"Retry-After": str(exc.retry_after)} if exc.retry_after else {}
        )
    else:
        # Log unexpected errors
        logger.exception("Unexpected API error")
        return JSONResponse(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            content={"error": "Internal server error"}
        )

Logging Integration

import structlog

logger = structlog.get_logger()

class ErrorLoggingMiddleware:
    async def __call__(self, request, call_next):
        try:
            response = await call_next(request)
            return response
        except CleverAgentsError as e:
            logger.warning(
                "handled_error",
                error_type=type(e).__name__,
                message=e.message,
                details=e.details
            )
            raise
        except Exception as e:
            logger.exception(
                "unhandled_error",
                error_type=type(e).__name__
            )
            raise

Consequences

Positive

• Clear error categorization
• Consistent error handling patterns
• Proper error context preservation
• Supports fail-fast principle
• Easy to test error scenarios
• Good debugging experience

Negative

• More exception classes to maintain
• Need discipline to use correct exception types
• Potential for over-specific exceptions

Neutral

• Requires documentation of error scenarios
• Need to map from Go error patterns
• Testing must cover error paths

Implementation Guidelines

1. Never catch Exception broadly - Be specific about what you handle
2. Always preserve stack traces - Use raise ... from e when re-raising
3. Add context when crossing boundaries - Enhance errors with domain context
4. Log at appropriate levels - Warnings for handled errors, errors for unexpected
5. Test error paths - Ensure error handling is covered by tests

Testing Strategy

import pytest

def test_validation_error_handling():
    with pytest.raises(ValidationError) as exc:
        validate_plan_name("")
    
    assert "Name cannot be empty" in str(exc.value)
    assert exc.value.details.get('field') == 'name'

@pytest.mark.asyncio
async def test_retry_on_network_error():
    call_count = 0
    
    async def flaky_operation():
        nonlocal call_count
        call_count += 1
        if call_count < 3:
            raise NetworkError("Connection failed")
        return "success"
    
    result = await with_retry(flaky_operation)
    assert result == "success"
    assert call_count == 3

References

• Python Exception Hierarchy
• Effective Python Item 87: Define a Root Exception
• Go Error Handling