Compare commits

..

2 Commits

Author SHA1 Message Date
HAL9000 806c1f34f1 style: fix code formatting with ruff
Fix formatting issues in OpenRouter provider registry implementation to pass CI lint checks.
2026-04-19 06:11:57 +00:00
HAL9000 c228ad546e feat(providers): implement OpenRouter provider support in ProviderRegistry
CI / lint (pull_request) Failing after 56s
CI / helm (pull_request) Successful in 51s
CI / push-validation (pull_request) Successful in 22s
CI / quality (pull_request) Successful in 4m36s
CI / typecheck (pull_request) Successful in 4m38s
CI / security (pull_request) Successful in 5m11s
CI / coverage (pull_request) Has been skipped
CI / build (pull_request) Successful in 3m47s
CI / integration_tests (pull_request) Successful in 7m36s
CI / e2e_tests (pull_request) Successful in 7m4s
CI / unit_tests (pull_request) Successful in 8m42s
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Failing after 3s
Added handling for ProviderType.OPENROUTER in _create_provider_llm in src/cleveragents/providers/registry.py to resolve ValueError: Unsupported provider type when using OpenRouter via create_llm.

Created features/openrouter_provider_registry.feature with 11 scenarios validating OpenRouter provider behavior in ProviderRegistry.

Created features/steps/openrouter_provider_registry_steps.py with step definitions for the new feature file.

Created docs/reference/providers.md with comprehensive documentation including the OpenRouter configuration guide.

ISSUES CLOSED: #8907
2026-04-19 02:44:16 +00:00
8 changed files with 6101 additions and 481 deletions
-8
View File
@@ -28,14 +28,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
correctly in all deployment modes: Docker containers, local pip installs
(wheel or editable), and development environments.
- **TDD Non-AssertionError Guard Visibility** (#8294): `apply_tdd_inversion` in
- **bug-hunt-pool-supervisor Non-Blocking Tracking** (#8835): The automation-tracking-manager
call in step 5 was blocking the main loop indefinitely, causing 3+ consecutive initialization
failures. Step 5 now explicitly marks tracking as best-effort -- if the call does not complete
within a reasonable time or fails, it is skipped and the supervisor continues to the next
cycle. A new Rule 9 reinforces that tracking must never block the main loop; core
functionality (module mapping, worker dispatch, monitoring) takes priority over status
reporting.
`features/environment.py` now emits its non-assertion exception guard warning to
both the structured logger and `stderr` via a new `_warning_with_stderr` helper.
This makes the guard firing visible in standard Behave console output and CI log
-453
View File
@@ -1,453 +0,0 @@
# Configuration Reference Guide
This guide provides a comprehensive reference for configuring CleverAgents. Configuration is primarily managed through environment variables, with sensible defaults for most settings.
## Overview
CleverAgents uses a **Pydantic-based settings system** that reads configuration from environment variables with the `CLEVERAGENTS_` prefix. The configuration system is designed to be:
- **Environment-first**: All configuration comes from environment variables, making it ideal for containerized and cloud-native deployments
- **Hierarchical**: Settings are organized by functional area (runtime, providers, storage, observability, etc.)
- **Validated**: All configuration values are validated at startup with clear error messages
- **Sensible defaults**: Most settings have reasonable defaults suitable for development and production
### Configuration Loading Order
1. **Environment variables** (highest priority) — `CLEVERAGENTS_*` prefixed variables
2. **Provider-specific environment variables** — Native provider variables (e.g., `OPENAI_API_KEY`)
3. **Default values** — Hardcoded defaults in the settings schema
When a setting is not explicitly configured, the system falls back to the next level in the order above.
### Configuration File Locations
CleverAgents stores configuration and data in the following locations:
| Purpose | Default Location | Environment Variable |
|---------|------------------|----------------------|
| User data (sessions, personas, etc.) | `~/.cleveragents/` | `CLEVERAGENTS_DATA_DIR` |
| Database | `~/.cleveragents/cleveragents.db` | `CLEVERAGENTS_DATABASE_URL` |
| Logs | `./logs/` | `CLEVERAGENTS_LOG_DIR` |
| Storage (plans, resources, etc.) | `./data/` | `CLEVERAGENTS_STORAGE_BASE_PATH` |
| Vector store | `./.cleveragents/vector_store/` | `CLEVERAGENTS_VECTOR_STORE_PATH` |
## Environment Variables Reference
All environment variables are organized by functional area. Each entry includes:
- **Variable name** — The full `CLEVERAGENTS_*` environment variable name
- **Type** — The expected data type (string, integer, boolean, etc.)
- **Default** — The default value if not set
- **Description** — What the setting controls
- **Example** — A practical example value
### Runtime & Server Configuration
#### `CLEVERAGENTS_ENV`
- **Type:** String
- **Default:** `development`
- **Valid values:** `development`, `production`, `staging`, `test`
- **Description:** The runtime environment. Controls logging verbosity, debug features, and validation strictness.
- **Example:** `export CLEVERAGENTS_ENV=production`
#### `CLEVERAGENTS_DEBUG_ENABLED`
- **Type:** Boolean
- **Default:** `false`
- **Description:** Enable debug mode with verbose logging and additional runtime checks. Should never be enabled in production.
- **Example:** `export CLEVERAGENTS_DEBUG_ENABLED=true`
#### `CLEVERAGENTS_SERVER_HOST`
- **Type:** String (IP address)
- **Default:** `0.0.0.0`
- **Description:** The host address for the CleverAgents server to bind to.
- **Example:** `export CLEVERAGENTS_SERVER_HOST=127.0.0.1`
#### `CLEVERAGENTS_SERVER_PORT`
- **Type:** Integer
- **Default:** `8080`
- **Valid range:** 1-65535
- **Description:** The port for the CleverAgents server to listen on.
- **Example:** `export CLEVERAGENTS_SERVER_PORT=9000`
#### `CLEVERAGENTS_SERVER_RELOAD`
- **Type:** Boolean
- **Default:** `false`
- **Description:** Enable auto-reload on code changes (development only). Never use in production.
- **Example:** `export CLEVERAGENTS_SERVER_RELOAD=true`
#### `CLEVERAGENTS_SERVER_URL`
- **Type:** String (URL)
- **Default:** `None` (local mode)
- **Description:** URL of a remote CleverAgents server to connect to. When set, the CLI operates in server mode.
- **Example:** `export CLEVERAGENTS_SERVER_URL=https://cleveragents.example.com`
#### `CLEVERAGENTS_SERVER_TOKEN`
- **Type:** String
- **Default:** `None`
- **Description:** Authentication token for connecting to a remote CleverAgents server. Required when `CLEVERAGENTS_SERVER_URL` is set.
- **Example:** `export CLEVERAGENTS_SERVER_TOKEN=sk-1234567890abcdef`
### AI Provider Configuration
#### Provider API Keys
Each AI provider requires its API key to be configured. CleverAgents supports multiple providers simultaneously and automatically selects the best available one.
| Provider | Primary Variable | Fallback Variables | Example |
|----------|------------------|-------------------|---------|
| OpenAI | `OPENAI_API_KEY` | `CLEVERAGENTS_OPENAI_API_KEY` | `sk-proj-...` |
| Anthropic | `ANTHROPIC_API_KEY` | `CLEVERAGENTS_ANTHROPIC_API_KEY` | `sk-ant-...` |
| Google | `GOOGLE_API_KEY` | `GOOGLE_GENAI_API_KEY`, `CLEVERAGENTS_GOOGLE_API_KEY` | `AIza...` |
| Azure OpenAI | `AZURE_OPENAI_API_KEY` | `AZURE_API_KEY`, `CLEVERAGENTS_AZURE_API_KEY` | `...` |
| OpenRouter | `OPENROUTER_API_KEY` | `CLEVERAGENTS_OPENROUTER_API_KEY` | `sk-or-...` |
| Gemini | `GEMINI_API_KEY` | `GOOGLE_GEMINI_API_KEY`, `CLEVERAGENTS_GEMINI_API_KEY` | `AIza...` |
| Cohere | `COHERE_API_KEY` | `CLEVERAGENTS_COHERE_API_KEY` | `...` |
| Groq | `GROQ_API_KEY` | `CLEVERAGENTS_GROQ_API_KEY` | `gsk_...` |
| Together | `TOGETHER_API_KEY` | `CLEVERAGENTS_TOGETHER_API_KEY` | `...` |
| HuggingFace | `HF_TOKEN` | `HUGGINGFACEHUB_API_TOKEN`, `HUGGING_FACE_HUB_TOKEN` | `hf_...` |
| Perplexity | `PERPLEXITY_API_KEY` | `CLEVERAGENTS_PERPLEXITY_API_KEY` | `pplx-...` |
#### `CLEVERAGENTS_DEFAULT_PROVIDER`
- **Type:** String
- **Default:** `None` (auto-detect from configured providers)
- **Valid values:** `openai`, `anthropic`, `google`, `azure`, `openrouter`, `gemini`, `cohere`, `groq`, `together`, `huggingface`, `perplexity`
- **Description:** Pin the default AI provider. When not set, the system automatically selects the first available configured provider in the fallback order.
- **Example:** `export CLEVERAGENTS_DEFAULT_PROVIDER=anthropic`
#### `CLEVERAGENTS_DEFAULT_MODEL`
- **Type:** String
- **Default:** `None` (provider-specific default)
- **Description:** Pin the default model for the selected provider. When not set, each provider's published default is used (e.g., `gpt-4o` for OpenAI, `claude-sonnet-4-20250514` for Anthropic).
- **Example:** `export CLEVERAGENTS_DEFAULT_MODEL=gpt-4-turbo`
#### `CLEVERAGENTS_MOCK_PROVIDERS`
- **Type:** Boolean
- **Default:** `false`
- **Description:** Enable mock AI providers for testing. Must not be used in production. Useful for development and CI/CD pipelines.
- **Example:** `export CLEVERAGENTS_MOCK_PROVIDERS=true`
#### `CLEVERAGENTS_TESTING_USE_MOCK_AI`
- **Type:** Boolean
- **Default:** `false`
- **Description:** Force the in-repo mock provider for Behave/Robot test suites to prevent external API calls.
- **Example:** `export CLEVERAGENTS_TESTING_USE_MOCK_AI=true`
### Logging & Paths
#### `CLEVERAGENTS_LOG_LEVEL`
- **Type:** String
- **Default:** `INFO`
- **Valid values:** `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`
- **Description:** The logging level for standard output and file logs.
- **Example:** `export CLEVERAGENTS_LOG_LEVEL=DEBUG`
#### `CLEVERAGENTS_LOG_DIR`
- **Type:** Path
- **Default:** `./logs/`
- **Description:** Directory where log files are written.
- **Example:** `export CLEVERAGENTS_LOG_DIR=/var/log/cleveragents`
#### `CLEVERAGENTS_DATA_DIR`
- **Type:** Path
- **Default:** `~/.cleveragents/`
- **Description:** Directory for user data, sessions, personas, and other persistent state.
- **Example:** `export CLEVERAGENTS_DATA_DIR=/data/cleveragents`
#### `CLEVERAGENTS_STORAGE_BASE_PATH`
- **Type:** Path
- **Default:** `./data/`
- **Description:** Base directory for plan storage, checkpoints, and other runtime data.
- **Example:** `export CLEVERAGENTS_STORAGE_BASE_PATH=/storage/cleveragents`
### Database & Persistence
#### `CLEVERAGENTS_DATABASE_URL`
- **Type:** String (database URL)
- **Default:** `sqlite:///$HOME/.cleveragents/cleveragents.db`
- **Supported formats:**
- SQLite: `sqlite:///path/to/database.db`
- PostgreSQL: `postgresql://user:password@host:port/database`
- MySQL: `mysql+pymysql://user:password@host:port/database`
- DuckDB: `duckdb:///path/to/database.duckdb`
- **Description:** The database URL for storing sessions, plans, and other persistent data.
- **Example:** `export CLEVERAGENTS_DATABASE_URL=postgresql://user:pass@localhost/cleveragents`
### Cost Controls & Budgets
#### `CLEVERAGENTS_SESSION_MAX_COST_USD`
- **Type:** Float
- **Default:** `None` (unlimited)
- **Valid range:** >= 0.0
- **Description:** Maximum USD spend per session across all plans. `None` means unlimited.
- **Example:** `export CLEVERAGENTS_SESSION_MAX_COST_USD=100.00`
#### `CLEVERAGENTS_ORG_MAX_COST_USD`
- **Type:** Float
- **Default:** `None` (unlimited)
- **Valid range:** >= 0.0
- **Description:** Maximum USD spend per organization across all sessions. `None` means unlimited.
- **Example:** `export CLEVERAGENTS_ORG_MAX_COST_USD=1000.00`
#### `CLEVERAGENTS_BUDGET_PER_PLAN`
- **Type:** Float
- **Default:** `None` (unlimited)
- **Valid range:** >= 0.0
- **Description:** Maximum USD spend per plan execution. `None` means unlimited.
- **Example:** `export CLEVERAGENTS_BUDGET_PER_PLAN=50.00`
#### `CLEVERAGENTS_BUDGET_PER_DAY`
- **Type:** Float
- **Default:** `None` (unlimited)
- **Valid range:** >= 0.0
- **Description:** Maximum USD spend per calendar day across all plans. `None` means unlimited.
- **Example:** `export CLEVERAGENTS_BUDGET_PER_DAY=500.00`
### Observability & Logging
#### LangSmith Tracing
#### `CLEVERAGENTS_LANGSMITH_ENABLED`
- **Type:** Boolean
- **Default:** `false`
- **Aliases:** `LANGCHAIN_TRACING_V2`, `LANGSMITH_TRACING_V2`
- **Description:** Enable LangSmith tracing for LangChain/LangGraph agents.
- **Example:** `export CLEVERAGENTS_LANGSMITH_ENABLED=true`
#### `CLEVERAGENTS_LANGSMITH_API_KEY`
- **Type:** String
- **Default:** `None`
- **Aliases:** `LANGSMITH_API_KEY`, `LANGCHAIN_API_KEY`
- **Description:** API key for LangSmith. Required when tracing is enabled.
- **Example:** `export CLEVERAGENTS_LANGSMITH_API_KEY=ls_...`
#### `CLEVERAGENTS_LANGSMITH_PROJECT`
- **Type:** String
- **Default:** `None`
- **Aliases:** `LANGSMITH_PROJECT`, `LANGCHAIN_PROJECT`
- **Description:** LangSmith project name. Required when tracing is enabled.
- **Example:** `export CLEVERAGENTS_LANGSMITH_PROJECT=my-project`
### Cleanup & Retention Policies
#### `CLEVERAGENTS_CLEANUP_SANDBOX_MAX_AGE_HOURS`
- **Type:** Integer
- **Default:** `48`
- **Valid range:** >= 1
- **Description:** Maximum age (in hours) before stale sandboxes are eligible for cleanup.
- **Example:** `export CLEVERAGENTS_CLEANUP_SANDBOX_MAX_AGE_HOURS=72`
#### `CLEVERAGENTS_CHECKPOINT_MAX`
- **Type:** Integer
- **Default:** `50`
- **Valid range:** >= 2
- **Description:** Maximum number of checkpoints to keep per plan. Oldest checkpoints are pruned first, but the first and most recent are always preserved.
- **Example:** `export CLEVERAGENTS_CHECKPOINT_MAX=100`
### Audit Logging
#### `CLEVERAGENTS_AUDIT_RETENTION_DAYS`
- **Type:** Integer
- **Default:** `0` (keep indefinitely)
- **Valid range:** >= 0
- **Description:** Days to retain audit log entries before pruning. `0` means keep indefinitely (spec default for compliance).
- **Example:** `export CLEVERAGENTS_AUDIT_RETENTION_DAYS=365`
#### `CLEVERAGENTS_AUDIT_ASYNC`
- **Type:** Boolean
- **Default:** `true`
- **Description:** When true, audit entries are written asynchronously via a write-behind queue on a background thread. Set to false for synchronous behavior (useful for debugging).
- **Example:** `export CLEVERAGENTS_AUDIT_ASYNC=false`
### Metrics & Observability
#### `CLEVERAGENTS_METRICS_ENABLED`
- **Type:** Boolean
- **Default:** `true`
- **Description:** Enable structured metric collection and emission.
- **Example:** `export CLEVERAGENTS_METRICS_ENABLED=false`
#### `CLEVERAGENTS_METRICS_EXPORT_PROMETHEUS`
- **Type:** Boolean
- **Default:** `false`
- **Description:** Enable Prometheus metrics export endpoint.
- **Example:** `export CLEVERAGENTS_METRICS_EXPORT_PROMETHEUS=true`
### Retry & Circuit Breaker Configuration
#### `CLEVERAGENTS_RETRY_MAX_ATTEMPTS`
- **Type:** Integer
- **Default:** `3`
- **Valid range:** 1-50
- **Description:** Default maximum retry attempts for service operations.
- **Example:** `export CLEVERAGENTS_RETRY_MAX_ATTEMPTS=5`
#### `CLEVERAGENTS_RETRY_BASE_DELAY`
- **Type:** Float
- **Default:** `1.0`
- **Valid range:** 0.0-300.0
- **Description:** Default base delay in seconds between retries.
- **Example:** `export CLEVERAGENTS_RETRY_BASE_DELAY=0.5`
#### `CLEVERAGENTS_RETRY_MAX_DELAY`
- **Type:** Float
- **Default:** `60.0`
- **Valid range:** 0.0-3600.0
- **Description:** Default maximum delay in seconds between retries.
- **Example:** `export CLEVERAGENTS_RETRY_MAX_DELAY=120.0`
#### `CLEVERAGENTS_RETRY_BACKOFF_STRATEGY`
- **Type:** String
- **Default:** `exponential`
- **Valid values:** `exponential`, `linear`, `fixed`, `jitter`, `none`
- **Description:** Default backoff strategy for retries.
- **Example:** `export CLEVERAGENTS_RETRY_BACKOFF_STRATEGY=linear`
### Security & Secrets
#### `CLEVERAGENTS_SHOW_SECRETS`
- **Type:** Boolean
- **Default:** `false`
- **Description:** When true, secrets are shown in CLI output and logs. Should never be enabled in production.
- **Example:** `export CLEVERAGENTS_SHOW_SECRETS=true`
### Automation & Output
#### `CLEVERAGENTS_AUTOMATION_PROFILE`
- **Type:** String
- **Default:** `` (empty, manual mode)
- **Valid values:** `manual`, `auto`, `full-auto` (or custom profile names)
- **Description:** Default automation profile name controlling plan execution behavior.
- **Example:** `export CLEVERAGENTS_AUTOMATION_PROFILE=auto`
#### `CLEVERAGENTS_FORMAT`
- **Type:** String
- **Default:** `None` (uses default format)
- **Valid values:** `json`, `markdown`, `text`, `yaml`
- **Description:** Default output format for CLI commands.
- **Example:** `export CLEVERAGENTS_FORMAT=json`
## Configuration Best Practices
### Development Environment
For local development, create a `.env` file in your project root:
```bash
# .env
CLEVERAGENTS_ENV=development
CLEVERAGENTS_DEBUG_ENABLED=true
CLEVERAGENTS_LOG_LEVEL=DEBUG
OPENAI_API_KEY=sk-proj-...
CLEVERAGENTS_DATABASE_URL=sqlite:///./dev.db
```
### Production Environment
For production deployments:
```bash
# Use strong, randomly generated secrets
export CLEVERAGENTS_ENV=production
export CLEVERAGENTS_DEBUG_ENABLED=false
export CLEVERAGENTS_LOG_LEVEL=WARNING
export CLEVERAGENTS_SHOW_SECRETS=false
# Use a production database
export CLEVERAGENTS_DATABASE_URL=postgresql://user:password@db.example.com/cleveragents
# Configure cost controls
export CLEVERAGENTS_SESSION_MAX_COST_USD=100
export CLEVERAGENTS_ORG_MAX_COST_USD=1000
# Enable observability
export CLEVERAGENTS_METRICS_ENABLED=true
export CLEVERAGENTS_LANGSMITH_ENABLED=true
export CLEVERAGENTS_LANGSMITH_API_KEY=ls_...
export CLEVERAGENTS_LANGSMITH_PROJECT=production
```
### Multi-Provider Setup
To support multiple AI providers with automatic fallback:
```bash
# Configure multiple providers
export OPENAI_API_KEY=sk-proj-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=AIza...
# Set primary provider (optional)
export CLEVERAGENTS_DEFAULT_PROVIDER=anthropic
```
## Security Considerations
### Secrets Management
1. **Never commit secrets** — Use environment variables or a secrets manager (Vault, AWS Secrets Manager, etc.)
2. **Use strong API keys** — Ensure all provider API keys are strong and rotated regularly
3. **Restrict access** — Limit who can view or modify configuration in your deployment
4. **Audit logging** — Enable audit logging to track configuration changes and API usage
5. **Redaction** — CleverAgents automatically redacts secrets in logs and output
### Database Security
- **Use encrypted connections** — Always use TLS/SSL for database connections
- **Strong credentials** — Use strong, randomly generated database passwords
- **Network isolation** — Restrict database access to authorized services only
### API Key Security
- **Rotate keys regularly** — Implement automated key rotation (e.g., monthly)
- **Use scoped keys** — If your provider supports it, use keys with minimal required permissions
- **Monitor usage** — Track API usage and set up alerts for unusual activity
## Troubleshooting Configuration Issues
### No AI Provider Configured
**Error:** `ValueError: No AI providers configured and mock_providers is disabled.`
**Solution:**
1. Set at least one provider API key: `export OPENAI_API_KEY=sk-proj-...`
2. Or enable mock providers for testing: `export CLEVERAGENTS_MOCK_PROVIDERS=true`
3. Verify the key is set: `echo $OPENAI_API_KEY`
### LangSmith Configuration Incomplete
**Error:** `LangSmith API key is required` or `LangSmith project name is required`
**Solution:**
1. Set both required variables:
```bash
export CLEVERAGENTS_LANGSMITH_API_KEY=ls_...
export CLEVERAGENTS_LANGSMITH_PROJECT=my-project
```
2. Or disable LangSmith: `export CLEVERAGENTS_LANGSMITH_ENABLED=false`
### Database Connection Failed
**Error:** `sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server`
**Solution:**
1. Verify the database URL: `echo $CLEVERAGENTS_DATABASE_URL`
2. Check database credentials and host
3. Ensure the database is running and accessible
### Configuration Not Taking Effect
**Problem:** Environment variable changes not reflected
**Solution:**
1. Verify the variable is set: `env | grep CLEVERAGENTS`
2. Restart the application (singleton settings are cached)
3. Check for typos in variable names (case-sensitive)
4. Ensure the variable is exported: `export CLEVERAGENTS_VAR=value`
## Related Documentation
- [Architecture Guide](../architecture.md) — System design and component overview
- [API Reference - Configuration](../api/config.md) — Detailed API documentation
- [Development Guide](../development/quality-automation.md) — Development setup and workflows
- [Specification](../specification.md) — Authoritative design documentation
- [FAQ](../faq.md) — Frequently asked questions
+120
View File
@@ -0,0 +1,120 @@
# Provider Configuration Reference
CleverAgents supports multiple AI provider backends through the `ProviderRegistry`.
Each provider is configured via environment variables or the `agents config set` command.
## Supported Providers
| Provider | `ProviderType` | API Key Environment Variable |
|----------|---------------|------------------------------|
| OpenAI | `openai` | `OPENAI_API_KEY` |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` |
| Google / Gemini | `google` / `gemini` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` |
| Azure OpenAI | `azure` | `AZURE_OPENAI_API_KEY` |
| **OpenRouter** | **`openrouter`** | **`OPENROUTER_API_KEY`** |
| Groq | `groq` | `GROQ_API_KEY` |
| Together AI | `together` | `TOGETHER_API_KEY` |
| Cohere | `cohere` | `COHERE_API_KEY` |
---
## OpenRouter
[OpenRouter](https://openrouter.ai) provides a unified API gateway to hundreds of
models from different providers (Anthropic, OpenAI, Google, Meta, Mistral, and more).
This makes it ideal for cost optimisation and model diversity without managing
multiple API keys.
### Configuration
Set the following environment variables:
```bash
# Required
export OPENROUTER_API_KEY="sk-or-v1-..."
# Optional: default model (defaults to anthropic/claude-sonnet-4-20250514)
export CLEVERAGENTS_DEFAULT_MODEL="openai/gpt-4o"
# Optional: set OpenRouter as the default provider
export CLEVERAGENTS_DEFAULT_PROVIDER="openrouter"
# Optional: organisation identifier sent in HTTP headers
export OPENROUTER_ORGANIZATION="myapp.example.com"
```
Or use the CLI:
```bash
agents config set provider.openrouter.api_key "sk-or-v1-..."
agents config set provider.openrouter.model "anthropic/claude-3-haiku"
```
### Config Keys
| Config Key | Environment Variable | Description |
|------------|---------------------|-------------|
| `provider.openrouter.api_key` | `OPENROUTER_API_KEY` | OpenRouter API key (required) |
| `provider.openrouter.model` | `CLEVERAGENTS_DEFAULT_MODEL` | Default model slug (optional) |
| `provider.openrouter.organization` | `OPENROUTER_ORGANIZATION` | Organisation name sent in `HTTP-Referer` / `X-Title` headers (optional) |
### Capabilities
| Feature | Supported |
|---------|-----------|
| Streaming | ✅ Yes |
| Tool calls | ✅ Yes |
| Vision / image input | ✅ Yes (model-dependent) |
| JSON mode | ✅ Yes (model-dependent) |
| Max context length | 128 000 tokens (varies by model) |
### Selecting a Model
OpenRouter model slugs follow the format `<provider>/<model-name>`. Examples:
```bash
# Anthropic via OpenRouter
agents config set provider.openrouter.model "anthropic/claude-sonnet-4-20250514"
# OpenAI via OpenRouter
agents config set provider.openrouter.model "openai/gpt-4o"
# Meta Llama via OpenRouter
agents config set provider.openrouter.model "meta-llama/llama-3.1-70b-instruct"
```
Browse the full model catalogue at <https://openrouter.ai/models>.
### Fallback Priority
When no explicit provider is configured, CleverAgents selects the first available
provider in the following fallback order:
1. OpenAI
2. Anthropic
3. Google
4. Azure
5. **OpenRouter** ← position 5
6. Groq
7. Together AI
8. Cohere
### Example Usage
```python
from cleveragents.providers.registry import ProviderRegistry, ProviderType
registry = ProviderRegistry()
# Create a LangChain LLM directly
llm = registry.create_llm(
provider_type=ProviderType.OPENROUTER,
model_id="anthropic/claude-3-haiku",
)
# Or create a full AIProviderInterface adapter
provider = registry.create_ai_provider(
provider_type="openrouter",
model_id="openai/gpt-4o",
)
```
+5494 -18
View File
File diff suppressed because one or more lines are too long
@@ -0,0 +1,87 @@
Feature: OpenRouter provider support in ProviderRegistry
As a developer using CleverAgents
I want ProviderRegistry to handle ProviderType.OPENROUTER in create_llm and _create_provider_llm
So that OpenRouter models are accessible without raising ValueError
# Covers the core acceptance criterion:
# ProviderRegistry.create_llm handles ProviderType.OPENROUTER without raising ValueError
@unit @providers @openrouter @registry
Scenario: create_llm dispatches to OpenRouter via _create_provider_llm
Given I have an openrouter registry with API key "sk-or-test-key"
And the openrouter registry LangChain ChatOpenAI client is stubbed
When I call create_llm on the openrouter registry for provider "openrouter" with model "anthropic/claude-3-haiku"
Then the openrouter registry ChatOpenAI should be called with model "anthropic/claude-3-haiku"
And the openrouter registry ChatOpenAI should use base url "https://openrouter.ai/api/v1"
And the openrouter registry ChatOpenAI should use api key "sk-or-test-key"
@unit @providers @openrouter @registry
Scenario: _create_provider_llm builds ChatOpenAI for OpenRouter with default model
Given I have an openrouter registry with API key "sk-or-default"
And the openrouter registry LangChain ChatOpenAI client is stubbed
When I call _create_provider_llm on the openrouter registry for provider "openrouter" with model None
Then the openrouter registry ChatOpenAI should be called with model "anthropic/claude-sonnet-4-20250514"
And the openrouter registry ChatOpenAI should use base url "https://openrouter.ai/api/v1"
@unit @providers @openrouter @registry
Scenario: _create_provider_llm injects organization headers when configured
Given I have an openrouter registry with API key "sk-or-org" and organization "myapp.example.com"
And the openrouter registry LangChain ChatOpenAI client is stubbed
When I call _create_provider_llm on the openrouter registry for provider "openrouter" with model "openai/gpt-4o"
Then the openrouter registry ChatOpenAI should include header "HTTP-Referer" with value "myapp.example.com"
And the openrouter registry ChatOpenAI should include header "X-Title" with value "myapp.example.com"
@unit @providers @openrouter @registry
Scenario: _create_provider_llm passes custom default_headers through
Given I have an openrouter registry with API key "sk-or-headers"
And the openrouter registry LangChain ChatOpenAI client is stubbed
When I call _create_provider_llm on the openrouter registry for provider "openrouter" with model "openai/gpt-4o" and default_headers "X-Custom=trace-123"
Then the openrouter registry ChatOpenAI should include header "X-Custom" with value "trace-123"
@unit @providers @openrouter @registry
Scenario: create_llm raises ValueError when OpenRouter API key is missing
Given I have an openrouter registry with no API keys
When I try to call create_llm on the openrouter registry for provider "openrouter"
Then an openrouter registry ValueError should be raised
And the openrouter registry error should mention "OPENROUTER_API_KEY"
@unit @providers @openrouter @registry
Scenario: ProviderType.OPENROUTER is in the registry fallback order
Given I have the openrouter ProviderRegistry class
When I check the openrouter FALLBACK_ORDER
Then ProviderType.OPENROUTER should be in the fallback order
@unit @providers @openrouter @registry
Scenario: OpenRouter default model is configured correctly
Given I have the openrouter ProviderRegistry class
When I check the openrouter DEFAULT_MODELS
Then the openrouter default model should be "anthropic/claude-sonnet-4-20250514"
@unit @providers @openrouter @registry
Scenario: OpenRouter capabilities are configured correctly
Given I have the openrouter ProviderRegistry class
When I check the openrouter DEFAULT_CAPABILITIES for OPENROUTER
Then the openrouter provider supports_streaming should be True
And the openrouter provider supports_tool_calls should be True
And the openrouter provider supports_vision should be True
And the openrouter provider max_context_length should be 128000
And the openrouter provider supports_json_mode should be True
@unit @providers @openrouter @registry
Scenario: OpenRouter provider is discovered when API key is set
Given I have an openrouter registry with API key "sk-or-discover"
When I call get_provider_info for OPENROUTER on the openrouter registry
Then the openrouter provider info should be configured
@unit @providers @openrouter @registry
Scenario: OpenRouter provider is not discovered when API key is missing
Given I have an openrouter registry with no API keys
When I call get_provider_info for OPENROUTER on the openrouter registry
Then the openrouter provider info should not be configured
@unit @providers @openrouter @registry
Scenario: create_llm selects OpenRouter as default when it is the only configured provider
Given I have an openrouter registry with API key "sk-or-only"
And the openrouter registry LangChain ChatOpenAI client is stubbed
When I call create_llm on the openrouter registry without specifying a provider
Then the openrouter registry ChatOpenAI should be called with model "anthropic/claude-sonnet-4-20250514"
@@ -0,0 +1,374 @@
"""Step definitions for openrouter_provider_registry.feature.
Tests that ProviderRegistry.create_llm and _create_provider_llm correctly
dispatch to the OpenRouter ChatOpenAI backend without raising ValueError.
"""
from __future__ import annotations
import sys
import types
from typing import Any
from unittest.mock import MagicMock
from behave import given, then, use_step_matcher, when # type: ignore[import-untyped]
from cleveragents.providers.registry import (
ProviderRegistry,
ProviderType,
)
_OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_or_settings(
openrouter: str | None = None,
organization: str | None = None,
) -> MagicMock:
"""Return a minimal fake Settings object for OpenRouter tests."""
settings = MagicMock()
settings.openai_api_key = None
settings.anthropic_api_key = None
settings.google_api_key = None
settings.gemini_api_key = None
settings.azure_api_key = None
settings.openrouter_api_key = openrouter
settings.cohere_api_key = None
settings.groq_api_key = None
settings.together_api_key = None
settings.default_provider = None
settings.default_model = None
settings.azure_openai_endpoint = None
settings.azure_openai_api_version = None
settings.azure_openai_deployment = None
settings.openrouter_organization = organization
return settings
def _stub_chat_openai(context: Any) -> None:
"""Replace langchain_openai.ChatOpenAI with a recording stub."""
original = sys.modules.get("langchain_openai")
calls: list[dict[str, Any]] = []
class FakeChatOpenAI:
def __init__(self, **kwargs: Any) -> None:
calls.append(kwargs)
class FakeAzureChatOpenAI:
def __init__(self, **kwargs: Any) -> None:
pass
stub = types.ModuleType("langchain_openai")
stub.ChatOpenAI = FakeChatOpenAI # type: ignore[attr-defined]
stub.AzureChatOpenAI = FakeAzureChatOpenAI # type: ignore[attr-defined]
sys.modules["langchain_openai"] = stub
context._or_chat_openai_calls = calls
context._or_chat_openai_cls = FakeChatOpenAI
if not hasattr(context, "_or_cleanup"):
context._or_cleanup = []
def _restore() -> None:
if original is None:
sys.modules.pop("langchain_openai", None)
else:
sys.modules["langchain_openai"] = original
context._or_cleanup.append(_restore)
def _parse_headers_string(headers_string: str) -> dict[str, str]:
"""Parse 'Key=Value,Key2=Value2' into a dict."""
result: dict[str, str] = {}
for entry in headers_string.split(","):
entry = entry.strip()
if "=" in entry:
key, value = entry.split("=", 1)
result[key.strip()] = value.strip()
return result
# ---------------------------------------------------------------------------
# Given
# ---------------------------------------------------------------------------
use_step_matcher("re")
@given(
r'I have an openrouter registry with API key "(?P<or_api_key>[^"]+)" and organization "(?P<or_org>[^"]+)"'
)
def step_or_registry_with_key_and_org(
context: Any, or_api_key: str, or_org: str
) -> None:
context.or_registry = ProviderRegistry(
settings=_make_or_settings(openrouter=or_api_key, organization=or_org)
)
@given(r'I have an openrouter registry with API key "(?P<or_api_key>[^"]+)"')
def step_or_registry_with_key(context: Any, or_api_key: str) -> None:
context.or_registry = ProviderRegistry(
settings=_make_or_settings(openrouter=or_api_key)
)
@given(r"I have an openrouter registry with no API keys")
def step_or_registry_no_keys(context: Any) -> None:
context.or_registry = ProviderRegistry(settings=_make_or_settings())
@given(r"I have the openrouter ProviderRegistry class")
def step_or_registry_class(context: Any) -> None:
context.or_registry_class = ProviderRegistry
@given(r"the openrouter registry LangChain ChatOpenAI client is stubbed")
def step_or_stub_chat_openai(context: Any) -> None:
_stub_chat_openai(context)
# ---------------------------------------------------------------------------
# When
# ---------------------------------------------------------------------------
@when(
r'I call create_llm on the openrouter registry for provider "openrouter" with model "(?P<or_model>[^"]+)"'
)
def step_or_create_llm_with_model(context: Any, or_model: str) -> None:
context.or_error = None
try:
context.or_result = context.or_registry.create_llm(
provider_type="openrouter", model_id=or_model
)
except ValueError as exc:
context.or_error = exc
@when(r"I call create_llm on the openrouter registry without specifying a provider")
def step_or_create_llm_default(context: Any) -> None:
context.or_error = None
try:
context.or_result = context.or_registry.create_llm()
except ValueError as exc:
context.or_error = exc
@when(
r'I call _create_provider_llm on the openrouter registry for provider "openrouter" with model None'
)
def step_or_private_llm_none_model(context: Any) -> None:
context.or_error = None
try:
context.or_result = context.or_registry._create_provider_llm(
ProviderType.OPENROUTER, None
)
except ValueError as exc:
context.or_error = exc
@when(
r'I call _create_provider_llm on the openrouter registry for provider "openrouter" with model "(?P<or_model>[^"]+)" and default_headers "(?P<or_headers_string>[^"]+)"'
)
def step_or_private_llm_with_headers(
context: Any, or_model: str, or_headers_string: str
) -> None:
headers = _parse_headers_string(or_headers_string)
context.or_error = None
try:
context.or_result = context.or_registry._create_provider_llm(
ProviderType.OPENROUTER, or_model, default_headers=headers
)
except ValueError as exc:
context.or_error = exc
@when(
r'I call _create_provider_llm on the openrouter registry for provider "openrouter" with model "(?P<or_model>[^"]+)"'
)
def step_or_private_llm_with_model(context: Any, or_model: str) -> None:
context.or_error = None
try:
context.or_result = context.or_registry._create_provider_llm(
ProviderType.OPENROUTER, or_model
)
except ValueError as exc:
context.or_error = exc
@when(r'I try to call create_llm on the openrouter registry for provider "openrouter"')
def step_or_try_create_llm_no_key(context: Any) -> None:
context.or_error = None
try:
context.or_registry.create_llm(provider_type="openrouter")
except ValueError as exc:
context.or_error = exc
@when(r"I check the openrouter FALLBACK_ORDER")
def step_or_check_fallback_order(context: Any) -> None:
context.or_fallback_order = context.or_registry_class.FALLBACK_ORDER
@when(r"I check the openrouter DEFAULT_MODELS")
def step_or_check_default_models(context: Any) -> None:
context.or_default_models = context.or_registry_class.DEFAULT_MODELS
@when(r"I check the openrouter DEFAULT_CAPABILITIES for OPENROUTER")
def step_or_check_capabilities(context: Any) -> None:
context.or_capabilities = context.or_registry_class.DEFAULT_CAPABILITIES.get(
ProviderType.OPENROUTER
)
@when(r"I call get_provider_info for OPENROUTER on the openrouter registry")
def step_or_get_provider_info(context: Any) -> None:
context.or_provider_info = context.or_registry.get_provider_info(
ProviderType.OPENROUTER
)
# ---------------------------------------------------------------------------
# Then
# ---------------------------------------------------------------------------
@then(
r'the openrouter registry ChatOpenAI should be called with model "(?P<or_expected_model>[^"]+)"'
)
def step_or_assert_model(context: Any, or_expected_model: str) -> None:
calls = getattr(context, "_or_chat_openai_calls", [])
assert calls, "Expected ChatOpenAI to be instantiated"
actual_model = calls[-1].get("model")
assert actual_model == or_expected_model, (
f"Expected model={or_expected_model!r}, got {actual_model!r}"
)
@then(
r'the openrouter registry ChatOpenAI should use base url "(?P<or_expected_url>[^"]+)"'
)
def step_or_assert_base_url(context: Any, or_expected_url: str) -> None:
calls = getattr(context, "_or_chat_openai_calls", [])
assert calls, "Expected ChatOpenAI to be instantiated"
actual_url = calls[-1].get("openai_api_base")
assert actual_url == or_expected_url, (
f"Expected openai_api_base={or_expected_url!r}, got {actual_url!r}"
)
@then(
r'the openrouter registry ChatOpenAI should use api key "(?P<or_expected_key>[^"]+)"'
)
def step_or_assert_api_key(context: Any, or_expected_key: str) -> None:
calls = getattr(context, "_or_chat_openai_calls", [])
assert calls, "Expected ChatOpenAI to be instantiated"
actual_key = calls[-1].get("openai_api_key")
assert actual_key == or_expected_key, (
f"Expected openai_api_key={or_expected_key!r}, got {actual_key!r}"
)
@then(
r'the openrouter registry ChatOpenAI should include header "(?P<or_header_name>[^"]+)" with value "(?P<or_expected_value>[^"]+)"'
)
def step_or_assert_header(
context: Any, or_header_name: str, or_expected_value: str
) -> None:
calls = getattr(context, "_or_chat_openai_calls", [])
assert calls, "Expected ChatOpenAI to be instantiated"
headers = calls[-1].get("default_headers")
assert isinstance(headers, dict), (
f"Expected default_headers to be a dict, got {type(headers)}"
)
actual_value = headers.get(or_header_name)
assert actual_value == or_expected_value, (
f"Expected header {or_header_name}={or_expected_value!r}, got {actual_value!r}"
)
@then(r"an openrouter registry ValueError should be raised")
def step_or_assert_value_error(context: Any) -> None:
assert context.or_error is not None, "Expected a ValueError to be raised"
assert isinstance(context.or_error, ValueError), (
f"Expected ValueError, got {type(context.or_error).__name__}"
)
@then(r'the openrouter registry error should mention "(?P<or_fragment>[^"]+)"')
def step_or_assert_error_contains(context: Any, or_fragment: str) -> None:
assert context.or_error is not None, "Expected an error to be set"
message = str(context.or_error)
assert or_fragment in message, (
f"Expected error to contain {or_fragment!r}, got: {message!r}"
)
@then(r"ProviderType\.OPENROUTER should be in the fallback order")
def step_or_assert_in_fallback(context: Any) -> None:
assert ProviderType.OPENROUTER in context.or_fallback_order, (
"Expected ProviderType.OPENROUTER to be in FALLBACK_ORDER"
)
@then(r'the openrouter default model should be "(?P<or_expected_model>[^"]+)"')
def step_or_assert_default_model(context: Any, or_expected_model: str) -> None:
actual = context.or_default_models.get(ProviderType.OPENROUTER)
assert actual == or_expected_model, (
f"Expected default model={or_expected_model!r}, got {actual!r}"
)
@then(r"the openrouter provider supports_streaming should be (?P<or_expected>\w+)")
def step_or_assert_streaming(context: Any, or_expected: str) -> None:
assert context.or_capabilities is not None
assert context.or_capabilities.supports_streaming is (or_expected == "True")
@then(r"the openrouter provider supports_tool_calls should be (?P<or_expected>\w+)")
def step_or_assert_tool_calls(context: Any, or_expected: str) -> None:
assert context.or_capabilities is not None
assert context.or_capabilities.supports_tool_calls is (or_expected == "True")
@then(r"the openrouter provider supports_vision should be (?P<or_expected>\w+)")
def step_or_assert_vision(context: Any, or_expected: str) -> None:
assert context.or_capabilities is not None
assert context.or_capabilities.supports_vision is (or_expected == "True")
@then(r"the openrouter provider max_context_length should be (?P<or_expected>\d+)")
def step_or_assert_context_length(context: Any, or_expected: str) -> None:
assert context.or_capabilities is not None
expected_int = int(or_expected)
assert context.or_capabilities.max_context_length == expected_int, (
f"Expected max_context_length={expected_int}, got {context.or_capabilities.max_context_length}"
)
@then(r"the openrouter provider supports_json_mode should be (?P<or_expected>\w+)")
def step_or_assert_json_mode(context: Any, or_expected: str) -> None:
assert context.or_capabilities is not None
assert context.or_capabilities.supports_json_mode is (or_expected == "True")
@then(r"the openrouter provider info should be configured")
def step_or_assert_provider_configured(context: Any) -> None:
info = context.or_provider_info
assert info is not None, "Expected provider info to exist"
assert info.is_configured is True, "Expected provider to be configured"
@then(r"the openrouter provider info should not be configured")
def step_or_assert_provider_not_configured(context: Any) -> None:
info = context.or_provider_info
assert info is not None, "Expected provider info to exist"
assert info.is_configured is False, "Expected provider to not be configured"
-2
View File
@@ -24,8 +24,6 @@ nav:
- AI Providers: api/providers.md
- TUI: api/tui.md
- ACMS / UKO: api/acms.md
- Guides:
- Configuration Reference: guides/configuration-reference.md
- Modules:
- Shell Safety: modules/shell-safety.md
- UKO Provenance Tracking: modules/uko-provenance.md
+26
View File
@@ -525,6 +525,32 @@ class ProviderRegistry:
**kwargs,
)
if provider_type == ProviderType.OPENROUTER:
from langchain_openai import ChatOpenAI
from cleveragents.providers.llm.openrouter_provider import (
OpenRouterChatProvider,
)
key_attr = self.PROVIDER_KEY_ATTRS.get(provider_type)
api_key = getattr(self._settings, key_attr, None) if key_attr else None
organization = getattr(self._settings, "openrouter_organization", None)
default_headers: dict[str, str] | None = kwargs.pop("default_headers", None)
if organization:
if default_headers is None:
default_headers = {}
default_headers.setdefault("HTTP-Referer", organization)
default_headers.setdefault("X-Title", organization)
openrouter_kwargs: dict[str, Any] = {
"model": model_id or "anthropic/claude-sonnet-4-20250514",
"openai_api_base": OpenRouterChatProvider._BASE_URL,
"openai_api_key": api_key or "",
}
if default_headers:
openrouter_kwargs["default_headers"] = default_headers
openrouter_kwargs.update(kwargs)
return ChatOpenAI(**openrouter_kwargs)
if provider_type == ProviderType.GROQ:
from langchain_groq import ChatGroq