cleveragents-core/docs/architecture/decisions/ADR-008-provider-plugin-architecture.md

ADR-008: Provider Plugin Architecture

Status

Accepted

Context

The discovery phase identified complex provider management needs:

• Multiple AI providers (OpenAI, Anthropic, Google, Azure, etc.)
• 8 model management workflows
• Fallback chains and retry logic
• Custom model support
• Provider-specific capabilities (chat, embeddings, vision, tools)
• Dynamic model catalog updates

Decision

We will implement a plugin-based provider architecture with a common interface and provider-specific adapters.

Provider Interface

# cleveragents.domain.providers
from abc import ABC, abstractmethod
from typing import AsyncIterator, Optional

class ModelProvider(ABC):
    """Base interface for all model providers"""
    
    @abstractmethod
    async def chat_completion(
        self,
        messages: List[Message],
        model: str,
        temperature: float = 0.7,
        max_tokens: Optional[int] = None,
        stream: bool = False
    ) -> Union[ChatResponse, AsyncIterator[ChatChunk]]:
        pass
    
    @abstractmethod
    async def embeddings(
        self,
        texts: List[str],
        model: str
    ) -> EmbeddingsResponse:
        pass
    
    @abstractmethod
    def supports_feature(self, feature: str) -> bool:
        pass
    
    @abstractmethod
    async def list_models(self) -> List[ModelInfo]:
        pass

Provider Registry

# cleveragents.providers.registry
class ProviderRegistry:
    """Central registry for all providers"""
    
    def __init__(self):
        self._providers: Dict[str, ModelProvider] = {}
        self._fallback_chains: Dict[str, List[str]] = {}
    
    def register(self, name: str, provider: ModelProvider):
        self._providers[name] = provider
    
    def get_provider(self, name: str) -> ModelProvider:
        if name not in self._providers:
            raise ProviderNotFoundError(f"Provider {name} not registered")
        return self._providers[name]
    
    def set_fallback_chain(self, primary: str, fallbacks: List[str]):
        self._fallback_chains[primary] = fallbacks

2025-12-10 update — implemented runtime defaults

The concrete registry (src/cleveragents/providers/registry.py:1-357) and settings layer (src/cleveragents/config/settings.py:20-343) now realize this ADR with:

• Deterministic precedence (actor-first): CLI requires --actor (or a default actor). Actor configs carry provider/model; built-in actors are seeded from CLEVERAGENTS_DEFAULT_PROVIDER / CLEVERAGENTS_DEFAULT_MODEL and Settings defaults, then follow the fallback order openai → anthropic → google → azure → openrouter → groq → together → cohere → gemini.

• Diagnostics + CLI surfacing: PlanService and Typer commands enforce actor selection, log resolved provider/model from the actor config, and bubble actionable errors when actors are missing or misconfigured.

• Mock isolation: CLEVERAGENTS_TESTING_USE_MOCK_AI remains the only bypass inside application/container.py, keeping mocks out of production code per the implementation plan.

• Actor registry boundaries: The registry persists canonical config blobs/hashes/graph descriptors, seeds immutable built-in actors named <provider>/<model> from settings defaults, enforces local/<id> for custom actors, blocks removal of built-ins or the default actor, and requires --unsafe confirmation when an actor config is marked unsafe; CLI help/docs mirror these guards.

These details are mirrored in docs/reference/providers.md and docs/reference/env_variables.yaml so the ADR now points at the actual runtime behavior.

Provider Implementations

# cleveragents.providers.openai
class OpenAIProvider(ModelProvider):
    def __init__(self, api_key: str, base_url: Optional[str] = None):
        self.client = AsyncOpenAI(api_key=api_key, base_url=base_url)
    
    async def chat_completion(self, messages, model, **kwargs):
        try:
            if kwargs.get('stream'):
                return self._stream_chat(messages, model, **kwargs)
            
            response = await self.client.chat.completions.create(
                messages=messages,
                model=model,
                **kwargs
            )
            return ChatResponse.from_openai(response)
        except RateLimitError as e:
            raise ProviderRateLimitError(retry_after=e.retry_after)

Fallback Strategy

# cleveragents.providers.fallback
class FallbackProvider:
    """Handles fallback chains and retry logic"""
    
    async def execute_with_fallback(
        self,
        operation: Callable,
        providers: List[str],
        max_retries: int = 3
    ):
        for provider_name in providers:
            provider = self.registry.get_provider(provider_name)
            
            for attempt in range(max_retries):
                try:
                    return await operation(provider)
                except ProviderRateLimitError as e:
                    if attempt < max_retries - 1:
                        await asyncio.sleep(e.retry_after or 2 ** attempt)
                    continue
                except ProviderError:
                    break  # Try next provider
        
        raise AllProvidersFailed("All providers in chain failed")

Model Catalog

# cleveragents.providers.catalog
class ModelCatalog:
    """Manages model metadata and capabilities"""
    
    def __init__(self, catalog_path: Path):
        self.catalog = self._load_catalog(catalog_path)
    
    def get_model_info(self, provider: str, model: str) -> ModelInfo:
        return self.catalog[provider][model]
    
    def find_models_with_capability(self, capability: str) -> List[ModelInfo]:
        return [
            model for models in self.catalog.values()
            for model in models.values()
            if capability in model.capabilities
        ]

Consequences

Positive

• Easy to add new providers
• Consistent interface across providers
• Centralized fallback and retry logic
• Testable with mock providers
• Dynamic provider loading possible

Negative

• Abstraction may hide provider-specific features
• Need to maintain provider compatibility
• Catalog synchronization complexity

Implementation Priority

1. OpenAI provider (most common)
2. Anthropic provider
3. Local/Ollama provider
4. Google/Azure providers

References

• Strategy Pattern
• Plugin Architecture
• OpenAI API Documentation