hurui200320
9b7a0543d0
CI / lint (push) Successful in 55s
CI / quality (push) Successful in 1m8s
CI / build (push) Successful in 32s
CI / typecheck (push) Successful in 1m31s
CI / security (push) Successful in 1m31s
CI / helm (push) Successful in 26s
CI / push-validation (push) Successful in 19s
CI / integration_tests (push) Successful in 4m14s
CI / e2e_tests (push) Failing after 4m15s
CI / unit_tests (push) Successful in 6m11s
CI / docker (push) Successful in 1m34s
CI / coverage (push) Successful in 11m44s
CI / status-check (push) Failing after 3s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Successful in 1h30m44s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Failing after 52s
CI / quality (pull_request) Failing after 1m1s
CI / typecheck (pull_request) Failing after 1m3s
CI / lint (pull_request) Failing after 1m5s
CI / security (pull_request) Failing after 1m3s
CI / integration_tests (pull_request) Failing after 1m1s
CI / e2e_tests (pull_request) Failing after 1m1s
CI / build (pull_request) Failing after 1m1s
CI / push-validation (pull_request) Successful in 1m31s
CI / helm (pull_request) Successful in 1m5s
CI / unit_tests (pull_request) Failing after 1m0s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Failing after 6s
Add a ProviderType.OPENROUTER branch to ProviderRegistry._create_provider_llm()
so that create_llm("openrouter") returns a configured LangChain BaseLanguageModel
instead of raising ValueError("Unsupported provider type: openrouter").
The new branch creates a ChatOpenAI instance with openai_api_base set to the
OpenRouter gateway URL and openai_api_key from settings, matching the credentials
used by create_ai_provider(). Optional default_headers are sanitized to
dict[str, str] (string-coerced keys and values) via inline dict comprehension.
Also includes review fixes:
- M1: Added openrouter_organization header support (HTTP-Referer, X-Title)
- m1: Added explicit ValueError when openrouter_api_key is empty/missing
- M2: Added openai_api_key assertions to all OpenRouter BDD scenarios
- m2: Added default model fallback scenario for model_id=None
- m4: Added empty default_headers edge case scenario
- m5: Added extra kwargs forwarding scenario (temperature)
- m6: Added empty API key error scenario
- m7: Added negative assertion verifying original integer key 123 is absent
- m8: Added openai_api_key assertion to sanitized headers scenario
ISSUES CLOSED: #10948
535 lines
25 KiB
Gherkin
535 lines
25 KiB
Gherkin
Feature: Provider Registry Coverage
|
|
As a developer
|
|
I want to ensure the ProviderRegistry class is thoroughly tested
|
|
So that provider discovery and selection work correctly
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create ProviderRegistry with default settings
|
|
Given I import the ProviderRegistry class
|
|
When I create a ProviderRegistry with default settings
|
|
Then the registry should be initialized
|
|
And the registry should have provider information
|
|
|
|
@unit @providers @registry
|
|
Scenario: ProviderType enum values
|
|
Given I import the ProviderType enum
|
|
Then ProviderType should have OPENAI value "openai"
|
|
And ProviderType should have ANTHROPIC value "anthropic"
|
|
And ProviderType should have GOOGLE value "google"
|
|
And ProviderType should have AZURE value "azure"
|
|
And ProviderType should have OPENROUTER value "openrouter"
|
|
And ProviderType should have GEMINI value "gemini"
|
|
And ProviderType should have COHERE value "cohere"
|
|
And ProviderType should have GROQ value "groq"
|
|
And ProviderType should have TOGETHER value "together"
|
|
And ProviderType should have MOCK value "mock"
|
|
|
|
@unit @providers @registry
|
|
Scenario: ProviderCapabilities with all fields
|
|
Given I import the ProviderCapabilities class
|
|
When I create a ProviderCapabilities with all fields
|
|
Then supports_streaming should be True
|
|
And supports_tool_calls should be True
|
|
And supports_vision should be True
|
|
And max_context_length should be 128000
|
|
And supports_json_mode should be True
|
|
|
|
@unit @providers @registry
|
|
Scenario: ProviderCapabilities with default values
|
|
Given I import the ProviderCapabilities class
|
|
When I create a ProviderCapabilities with default values
|
|
Then supports_streaming should be True
|
|
And supports_tool_calls should be False
|
|
And supports_vision should be False
|
|
And max_context_length should be 4096
|
|
And supports_json_mode should be False
|
|
|
|
@unit @providers @registry
|
|
Scenario: ProviderInfo dataclass
|
|
Given I import the ProviderInfo class
|
|
And I import the ProviderType enum
|
|
When I create a ProviderInfo for openai
|
|
Then the provider_type should be ProviderType.OPENAI
|
|
And the provider info name should be "Openai"
|
|
And the is_configured should be False by default
|
|
|
|
@unit @providers @registry
|
|
Scenario: Optional string helper coerces non-string values
|
|
Given I import the optional string coercion helper
|
|
When I coerce optional string with integer 42
|
|
Then the coerced optional string result should be "42"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get all providers from registry
|
|
Given I have a ProviderRegistry instance
|
|
When I call get_all_providers
|
|
Then the provider registry result should be a list of ProviderInfo
|
|
And the provider registry result should include core provider types
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get configured providers when none configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I call get_configured_providers
|
|
Then the provider registry result should be an empty list
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get configured providers with OpenAI key
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
When I call get_configured_providers
|
|
Then the provider registry result should contain OpenAI provider
|
|
And the provider registry OpenAI entry should be configured
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get provider info by ProviderType
|
|
Given I have a ProviderRegistry instance
|
|
When I get provider info for ProviderType.OPENAI
|
|
Then the provider registry result should be a ProviderInfo instance
|
|
And the provider_type should be ProviderType.OPENAI
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get provider info by string
|
|
Given I have a ProviderRegistry instance
|
|
When I get provider info for string "openai"
|
|
Then the provider registry result should be a ProviderInfo instance
|
|
And the provider_type should be ProviderType.OPENAI
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get provider info by uppercase string
|
|
Given I have a ProviderRegistry instance
|
|
When I get provider info for string "ANTHROPIC"
|
|
Then the provider registry result should be a ProviderInfo instance
|
|
And the provider_type should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get provider info for invalid provider
|
|
Given I have a ProviderRegistry instance
|
|
When I get provider info for string "invalid_provider"
|
|
Then the provider registry result should be None
|
|
|
|
@unit @providers @registry
|
|
Scenario: Check if provider is configured - not configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I check if openai is configured
|
|
Then the provider registry result should be False
|
|
|
|
@unit @providers @registry
|
|
Scenario: Check if provider is configured - configured
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
When I check if openai is configured
|
|
Then the provider registry result should be True
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type with env var
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is set to "openai"
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.OPENAI
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type fallback order
|
|
Given I have a ProviderRegistry with Anthropic API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type when none configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be None
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type ignores invalid environment override
|
|
Given I have a ProviderRegistry with Anthropic API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is set to "not_a_real_provider"
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type skips unconfigured provider override
|
|
Given I have a ProviderRegistry with Anthropic API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is set to "openai"
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type honors settings default when valid
|
|
Given I have a ProviderRegistry with "anthropic" API key set and settings default provider "anthropic"
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default provider type ignores invalid settings default
|
|
Given I have a ProviderRegistry with "anthropic" API key set and settings default provider "not_a_real_provider"
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
When I call get_default_provider_type
|
|
Then the provider registry result should be ProviderType.ANTHROPIC
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model with env var
|
|
Given I have a ProviderRegistry instance
|
|
And CLEVERAGENTS_DEFAULT_MODEL is set to "gpt-4-turbo"
|
|
When I call get_default_model
|
|
Then the provider registry result should be "gpt-4-turbo"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model from settings configuration
|
|
Given I have a ProviderRegistry with default model "custom-model" configured
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I call get_default_model
|
|
Then the provider registry result should be "custom-model"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model for provider
|
|
Given I have a ProviderRegistry instance
|
|
When I call get_default_model for ProviderType.OPENAI
|
|
Then the provider registry result should be "gpt-4o"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model for provider by string
|
|
Given I have a ProviderRegistry instance
|
|
When I call get_default_model for string "anthropic"
|
|
Then the provider registry result should be "claude-sonnet-4-20250514"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model for invalid provider
|
|
Given I have a ProviderRegistry instance
|
|
When I call get_default_model for string "invalid"
|
|
Then the provider registry result should be None
|
|
|
|
@unit @providers @registry
|
|
Scenario: Get default model when no providers are configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I call get_default_model
|
|
Then the provider registry result should be None
|
|
|
|
@unit @providers @registry
|
|
Scenario: Default capabilities for OpenAI
|
|
Given I have a ProviderRegistry instance
|
|
When I get capabilities for ProviderType.OPENAI
|
|
Then supports_streaming should be True
|
|
And supports_tool_calls should be True
|
|
And supports_vision should be True
|
|
And max_context_length should be 128000
|
|
And supports_json_mode should be True
|
|
|
|
@unit @providers @registry
|
|
Scenario: Default capabilities for Anthropic
|
|
Given I have a ProviderRegistry instance
|
|
When I get capabilities for ProviderType.ANTHROPIC
|
|
Then supports_streaming should be True
|
|
And supports_tool_calls should be True
|
|
And supports_vision should be True
|
|
And max_context_length should be 200000
|
|
And supports_json_mode should be False
|
|
|
|
@unit @providers @registry
|
|
Scenario: Default capabilities for Google/Gemini
|
|
Given I have a ProviderRegistry instance
|
|
When I get capabilities for ProviderType.GOOGLE
|
|
Then supports_streaming should be True
|
|
And supports_tool_calls should be True
|
|
And supports_vision should be True
|
|
And max_context_length should be 1000000
|
|
And supports_json_mode should be True
|
|
|
|
@unit @providers @registry
|
|
Scenario: Global registry singleton
|
|
Given I import the get_provider_registry function
|
|
When I call get_provider_registry twice
|
|
Then both provider registry calls should return the same instance
|
|
|
|
@unit @providers @registry
|
|
Scenario: Reset global registry
|
|
Given I import the reset_provider_registry function
|
|
When I get the global registry and then reset it
|
|
Then resetting the provider registry should return a new instance
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM raises error when no provider configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an LLM without specifying a provider
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention configuring API keys
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM raises error for invalid provider string
|
|
Given I have a ProviderRegistry instance
|
|
When I try to create an LLM for provider "not_a_provider"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention unknown provider type
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM raises error when provider API key is missing
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an LLM for provider "openai"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention configuring API keys
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM delegates to provider factory for configured string input
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And the provider registry LLM factory is stubbed
|
|
When I call create_llm for provider "openai" and model "gpt-mini"
|
|
Then the stubbed LLM factory should be called for ProviderType.OPENAI with model "gpt-mini"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM without provider uses fallback default
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
And the provider registry LLM factory is stubbed
|
|
When I call create_llm without specifying a provider
|
|
Then the stubbed LLM factory should be called for ProviderType.OPENAI with model "gpt-4o"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM accepts ProviderType argument
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
And the provider registry LLM factory is stubbed
|
|
When I call create_llm for ProviderType.OPENAI
|
|
Then the stubbed LLM factory should be called for ProviderType.OPENAI with model "gpt-4o"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create LLM for mock provider skips API key requirement
|
|
Given I have a ProviderRegistry with no API keys
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
And the provider registry LLM factory is stubbed
|
|
When I call create_llm for provider "mock" without specifying a model
|
|
Then the stubbed LLM factory should be called for ProviderType.MOCK with model "mock-gpt"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm builds stubbed OpenAI client
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And langchain OpenAI client is stubbed
|
|
When I call _create_provider_llm for provider "openai" with model "custom-openai"
|
|
Then the stubbed ChatOpenAI should be created with model "custom-openai"
|
|
|
|
@unit @providers @registry
|
|
Scenario Outline: _create_provider_llm builds <ProviderName> client
|
|
Given I have a ProviderRegistry with "<provider_key>" API key set
|
|
And the "<ProviderName>" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "<provider_key>" with model "<model>"
|
|
Then the stubbed <ProviderName> client should be created with argument "model" value "<model>"
|
|
|
|
Examples:
|
|
| ProviderName | provider_key | model |
|
|
| Anthropic | anthropic | claude-mini-cover |
|
|
| Google | google | gemini-cover |
|
|
| Gemini | gemini | gemini-pro-cover |
|
|
| Groq | groq | llama-cover |
|
|
| Together | together | llama-together-cover |
|
|
| Cohere | cohere | command-cover |
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm builds Azure client with deployment override
|
|
Given I have a ProviderRegistry with "azure" API key set
|
|
And the "Azure" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "azure" with model "azure-cover"
|
|
Then the stubbed Azure client should be created with argument "deployment_name" value "azure-cover"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm raises when Azure endpoint is missing
|
|
Given I have a ProviderRegistry with "azure" API key set
|
|
And the Azure endpoint setting is cleared
|
|
When I try to call _create_provider_llm for provider "azure" with model "azure-cover"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention missing Azure endpoint
|
|
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm builds OpenRouter client with ChatOpenAI
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the "openrouter" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
|
Then the stubbed openrouter client should be created with argument "model" value "claude-openrouter-cover"
|
|
And the stubbed openrouter client should be created with argument "openai_api_base" value "https://openrouter.ai/api/v1"
|
|
And the stubbed openrouter client should be created with argument "openai_api_key" value "sk-openrouter-test"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm builds OpenRouter client with sanitized default headers
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the "openrouter" LangChain client is stubbed
|
|
And the OpenRouter default headers include numeric entries
|
|
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and custom default headers
|
|
Then the stubbed OpenRouter client headers should include "123" value "456"
|
|
And the stubbed OpenRouter client headers should include "X-Debug" value "789"
|
|
And the stubbed openrouter client should be created with argument "openai_api_key" value "sk-openrouter-test"
|
|
And the stubbed OpenRouter client headers should not include key 123
|
|
|
|
@unit @providers @registry
|
|
Scenario: create_llm succeeds for OpenRouter and delegates to _create_provider_llm
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
And the provider registry LLM factory is stubbed
|
|
When I call create_llm for provider "openrouter" and model "claude-openrouter"
|
|
Then the stubbed LLM factory should be called for ProviderType.OPENROUTER with model "claude-openrouter"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm adds organization headers for OpenRouter
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the OpenRouter organization setting is "my-org"
|
|
And the "openrouter" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
|
Then the stubbed OpenRouter client headers should include "HTTP-Referer" value "my-org"
|
|
And the stubbed OpenRouter client headers should include "X-Title" value "my-org"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm uses default model for OpenRouter when model_id is None
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the "openrouter" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "openrouter" without specifying a model
|
|
Then the stubbed openrouter client should be created with argument "model" value "anthropic/claude-sonnet-4-20250514"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm treats empty default_headers as None for OpenRouter
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the "openrouter" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and empty default headers
|
|
Then the stubbed openrouter client should not have default_headers
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm forwards extra kwargs to ChatOpenAI for OpenRouter
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And the "openrouter" LangChain client is stubbed
|
|
When I call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover" and temperature 0.7
|
|
Then the stubbed openrouter client should be created with argument "temperature" value "0.7"
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm raises error when OpenRouter API key is empty
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to call _create_provider_llm for provider "openrouter" with model "claude-openrouter-cover"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention missing "OPENROUTER_API_KEY" environment variable
|
|
|
|
@unit @providers @registry
|
|
Scenario: _create_provider_llm raises error for unsupported provider
|
|
Given I have a ProviderRegistry instance
|
|
When I try to call _create_provider_llm for provider "mock" with model "mock-gpt"
|
|
Then a provider registry ValueError should be raised
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider raises error when no provider configured
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an AI provider without specifying a provider
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention configuring API keys
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider raises error for invalid provider string
|
|
Given I have a ProviderRegistry instance
|
|
When I try to create an AI provider for provider "not_a_provider"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention unknown provider type
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider raises error when Google API key is missing
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an AI provider for provider "google"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention missing "GOOGLE_API_KEY" environment variable
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider succeeds for configured provider string
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider for provider "openai"
|
|
Then the provider registry AI provider should be an OpenAIChatProvider
|
|
And the provider registry AI provider name should be "openai"
|
|
And the provider registry AI provider model_id should be "gpt-4o"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider returns Anthropic adapter
|
|
Given I have a ProviderRegistry with Anthropic API key set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider for provider "anthropic"
|
|
Then the provider registry AI provider should be an AnthropicChatProvider
|
|
And the provider registry AI provider name should be "anthropic"
|
|
And the provider registry AI provider model_id should be "claude-sonnet-4-20250514"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider raises error when OpenAI API key is missing
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an AI provider for provider "openai"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention missing "OPENAI_API_KEY" environment variable
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider raises error when Anthropic API key is missing
|
|
Given I have a ProviderRegistry with no API keys
|
|
When I try to create an AI provider for provider "anthropic"
|
|
Then a provider registry ValueError should be raised
|
|
And the provider registry error should mention missing "ANTHROPIC_API_KEY" environment variable
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider returns Google adapter
|
|
Given I have a ProviderRegistry with "google" API key set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider for provider "google"
|
|
Then the provider registry AI provider should be a GoogleChatProvider
|
|
And the provider registry AI provider name should be "google"
|
|
And the provider registry AI provider model_id should be "gemini-2.0-flash"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider returns OpenRouter adapter
|
|
Given I have a ProviderRegistry with "openrouter" API key set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider for provider "openrouter"
|
|
Then the provider registry AI provider should be an OpenRouterChatProvider
|
|
And the provider registry AI provider name should be "openrouter"
|
|
And the provider registry AI provider model_id should be "anthropic/claude-sonnet-4-20250514"
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider without specifying provider uses fallback
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider without specifying a provider
|
|
Then the provider registry AI provider should be an OpenAIChatProvider
|
|
And the provider registry AI provider name should be "openai"
|
|
|
|
|
|
@unit @providers @registry
|
|
Scenario: Create AI provider honors explicit model overrides
|
|
Given I have a ProviderRegistry with OpenAI API key set
|
|
And CLEVERAGENTS_DEFAULT_MODEL is not set
|
|
When I create an AI provider specifying provider "openai" and model "custom-model"
|
|
Then the provider registry AI provider model_id should be "custom-model"
|
|
|
|
@unit @providers @registry
|
|
Scenario: AI provider exposes working llm factory
|
|
Given I have a ProviderRegistry with "groq" API key set
|
|
And the provider registry LLM factory is stubbed
|
|
When I create an AI provider for provider "groq"
|
|
And I call the AI provider llm factory with model "llama-mini-cover"
|
|
Then the stubbed LLM factory should be called for ProviderType.GROQ with model "llama-mini-cover"
|
|
|
|
@unit @providers @registry
|
|
Scenario: ProviderCapabilities is immutable
|
|
Given I import the ProviderCapabilities class
|
|
When I create a ProviderCapabilities instance
|
|
Then the instance should be frozen
|
|
And attempting to modify it should raise an error
|
|
|
|
@unit @providers @registry
|
|
Scenario: Provider fallback order
|
|
Given I have the ProviderRegistry class
|
|
When I check the FALLBACK_ORDER
|
|
Then OpenAI should be first in the order
|
|
And Anthropic should be second in the order
|
|
And Google should be third in the order
|
|
|
|
@unit @providers @registry
|
|
Scenario: Default models for all providers
|
|
Given I have the ProviderRegistry class
|
|
When I check the DEFAULT_MODELS
|
|
Then OpenAI default should be "gpt-4o"
|
|
And Anthropic default should be "claude-sonnet-4-20250514"
|
|
And Google default should be "gemini-2.0-flash"
|
|
And Groq default should be "llama-3.1-70b-versatile"
|