From a17848064db2190b22cf3928a8d3bd0603c49933 Mon Sep 17 00:00:00 2001 From: Jeffrey Phillips Freeman Date: Sun, 5 Apr 2026 07:03:54 +0000 Subject: [PATCH] docs: add providers API reference, update index/architecture/changelog MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/api/providers.md: new comprehensive API reference for cleveragents.providers — ProviderRegistry, ProviderType, ProviderCapabilities, ProviderInfo, LangChainChatProvider, module-level helpers, environment variables, and ASV benchmarks - docs/api/index.md: add providers module to module index table - docs/architecture.md: add Provider Registry section with discovery, selection, and factory usage examples - CHANGELOG.md: add [Unreleased] entries for: - agents plan list --namespace/-n option (#2616) - ASV benchmark suite for providers module (#3022) - MCP error extraction from content[0].text per MCP 1.4.0 (#2600) - CI quality gates restored to passing on master (#2629) --- docs/api/providers.md | 285 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 285 insertions(+) create mode 100644 docs/api/providers.md diff --git a/docs/api/providers.md b/docs/api/providers.md new file mode 100644 index 000000000..8835f475c --- /dev/null +++ b/docs/api/providers.md @@ -0,0 +1,285 @@ +# `cleveragents.providers` — AI Provider Registry + +The `providers` package implements the AI provider registry that discovers, +selects, and instantiates LLM backends. It follows the provider plugin +architecture described in ADR-008 and wires into the actor system via +`AIProviderInterface`. + +--- + +## Overview + +The registry discovers configured providers from environment variables and +`Settings`, selects a default using a configurable fallback chain, and +creates `LangChain` LLM instances or `AIProviderInterface` wrappers on +demand. + +```python +from cleveragents.providers.registry import get_provider_registry + +registry = get_provider_registry() +provider = registry.create_ai_provider() # uses default provider +llm = registry.create_llm() # returns a LangChain BaseLanguageModel +``` + +--- + +## `ProviderType` + +```python +class ProviderType(StrEnum): + OPENAI = "openai" + ANTHROPIC = "anthropic" + GOOGLE = "google" + AZURE = "azure" + OPENROUTER = "openrouter" + GEMINI = "gemini" + COHERE = "cohere" + GROQ = "groq" + TOGETHER = "together" + MOCK = "mock" +``` + +String enum of all supported provider identifiers. Values are used in +`CLEVERAGENTS_DEFAULT_PROVIDER` and in actor YAML `provider:` fields. + +--- + +## `ProviderCapabilities` + +```python +@dataclass(frozen=True) +class ProviderCapabilities: + supports_streaming: bool = True + supports_tool_calls: bool = False + supports_vision: bool = False + max_context_length: int = 4096 + supports_json_mode: bool = False +``` + +Immutable capability metadata for a provider. Used by the actor system to +gate features (e.g., tool calls, vision inputs) at runtime. + +### Default capabilities by provider + +| Provider | Streaming | Tool calls | Vision | Context (tokens) | JSON mode | +|----------|-----------|------------|--------|-----------------|-----------| +| OpenAI | ✓ | ✓ | ✓ | 128 000 | ✓ | +| Anthropic | ✓ | ✓ | ✓ | 200 000 | ✗ | +| Google / Gemini | ✓ | ✓ | ✓ | 1 000 000 | ✓ | +| Azure OpenAI | ✓ | ✓ | ✓ | 128 000 | ✓ | +| OpenRouter | ✓ | ✓ | ✓ | 128 000 | ✓ | +| Cohere | ✓ | ✓ | ✗ | 128 000 | ✗ | +| Groq | ✓ | ✓ | ✗ | 32 000 | ✓ | +| Together | ✓ | ✓ | ✗ | 32 000 | ✗ | +| Mock | ✓ | ✗ | ✗ | 4 096 | ✗ | + +--- + +## `ProviderInfo` + +```python +class ProviderInfo(BaseModel): + provider_type: ProviderType + name: str + api_key_env_var: str + default_model: str + capabilities: ProviderCapabilities + is_configured: bool +``` + +Read-only snapshot of a provider's registration state. Returned by +`ProviderRegistry.get_provider_info()` and `get_all_providers()`. + +--- + +## `ProviderRegistry` + +Central registry for discovering and managing AI providers. + +```python +from cleveragents.providers.registry import ProviderRegistry + +registry = ProviderRegistry() # uses global Settings +registry = ProviderRegistry(settings=my_cfg) # override settings +``` + +### Default models + +| Provider | Default model | +|----------|--------------| +| OpenAI | `gpt-4o` | +| Anthropic | `claude-sonnet-4-20250514` | +| Google / Gemini | `gemini-2.0-flash` | +| Azure | `gpt-4o` | +| OpenRouter | `anthropic/claude-sonnet-4-20250514` | +| Cohere | `command-r-plus` | +| Groq | `llama-3.1-70b-versatile` | +| Together | `meta-llama/Llama-3.1-70B-Instruct-Turbo` | +| Mock | `mock-gpt` | + +### Provider fallback order + +When `CLEVERAGENTS_DEFAULT_PROVIDER` is not set, the registry selects the +first configured provider in this order: + +`openai → anthropic → google → azure → openrouter → groq → together → cohere` + +### Methods + +#### `get_configured_providers() → list[ProviderInfo]` + +Returns only providers that have valid API credentials configured. + +#### `get_all_providers() → list[ProviderInfo]` + +Returns all known providers regardless of configuration state. + +#### `get_provider_info(provider_type) → ProviderInfo | None` + +Look up a provider by `ProviderType` or string name. Returns `None` if +the provider is unknown. + +#### `is_provider_configured(provider_type) → bool` + +Returns `True` if the provider has a non-empty API key in `Settings`. + +#### `get_default_provider_type() → ProviderType | None` + +Resolves the default provider using this precedence: + +1. `CLEVERAGENTS_DEFAULT_PROVIDER` environment variable +2. `Settings.default_provider` +3. First configured provider in fallback order + +Returns `None` if no provider is configured. + +#### `get_default_model(provider_type=None) → str | None` + +Resolves the default model using this precedence: + +1. `CLEVERAGENTS_DEFAULT_MODEL` environment variable +2. `Settings.default_model` +3. Provider's built-in default model + +#### `create_llm(provider_type=None, model_id=None, **kwargs) → BaseLanguageModel` + +Creates a LangChain `BaseLanguageModel` for the specified provider and model. + +```python +llm = registry.create_llm("anthropic", model_id="claude-3-haiku-20240307") +``` + +Raises `ValueError` if no provider is configured or the provider name is +unknown. + +#### `create_ai_provider(provider_type=None, model_id=None, max_retries=3) → AIProviderInterface` + +Creates an `AIProviderInterface` implementation. For most providers this +returns a `LangChainChatProvider`; Google and OpenRouter return their own +specialized implementations. + +```python +provider = registry.create_ai_provider("openai", model_id="gpt-4o-mini") +``` + +--- + +## `LangChainChatProvider` + +**Module:** `cleveragents.providers.llm.langchain_chat_provider` + +Wraps a LangChain chat model with the `AIProviderInterface` protocol and +wires it into `PlanGenerationGraph`. + +```python +class LangChainChatProvider(AIProviderInterface): + def __init__( + self, + *, + name: str, + model_id: str, + llm_factory: Callable[[str], BaseLanguageModel], + max_retries: int = 3, + supports_streaming: bool = True, + progress_map: dict[str, int] | None = None, + ) -> None: ... + + @property + def name(self) -> str: ... + + @name.setter + def name(self, value: str) -> None: ... + + @property + def model_id(self) -> str: ... + + @model_id.setter + def model_id(self, value: str) -> None: ... +``` + +> **Note (v3.7.0+):** `name` and `model_id` are now mutable properties with +> setters, fixing an `AttributeError` when `PlanService` attempted to resolve +> provider names after instantiation. (#1553) + +--- + +## Module-level helpers + +### `get_provider_registry(settings=None) → ProviderRegistry` + +Returns the global singleton `ProviderRegistry`. Pass `settings` to force +a fresh instance (useful in tests). + +### `reset_provider_registry() → None` + +Clears the global singleton. Call this between tests to ensure clean state. + +### `resolve_provider_by_name(name, settings=None) → AIProviderInterface` + +Convenience function that resolves a provider by string name and raises a +descriptive `ValueError` if the provider is not configured. + +```python +from cleveragents.providers.registry import resolve_provider_by_name + +provider = resolve_provider_by_name("anthropic") +``` + +--- + +## Environment variables + +| Variable | Description | +|----------|-------------| +| `CLEVERAGENTS_DEFAULT_PROVIDER` | Pin the global provider (e.g. `openai`) | +| `CLEVERAGENTS_DEFAULT_MODEL` | Pin the global model ID | +| `OPENAI_API_KEY` | OpenAI credentials | +| `ANTHROPIC_API_KEY` | Anthropic credentials | +| `GOOGLE_API_KEY` / `GOOGLE_GENAI_API_KEY` | Google credentials | +| `AZURE_OPENAI_API_KEY` | Azure OpenAI credentials | +| `AZURE_OPENAI_ENDPOINT` | Azure OpenAI endpoint URL | +| `AZURE_OPENAI_DEPLOYMENT` | Azure deployment name | +| `OPENROUTER_API_KEY` | OpenRouter credentials | +| `GEMINI_API_KEY` / `GOOGLE_GEMINI_API_KEY` | Gemini credentials | +| `COHERE_API_KEY` | Cohere credentials | +| `GROQ_API_KEY` | Groq credentials | +| `TOGETHER_API_KEY` | Together AI credentials | +| `CLEVERAGENTS_TESTING_USE_MOCK_AI` | Force mock provider in tests | + +--- + +## ASV Performance Benchmarks + +The `benchmarks/` directory contains ASV (airspeed velocity) benchmark +suites for the providers module. Run them with: + +```bash +nox -s benchmarks +# or directly: +asv run --config asv.conf.json +``` + +Benchmarks cover provider discovery, registry initialization, and LLM +factory creation latency. Results are tracked over time to detect +performance regressions.