forked from cleveragents/cleveragents-core
106 lines
4.8 KiB
Gherkin
106 lines
4.8 KiB
Gherkin
Feature: Provider configuration fixes remove FakeListLLM defaults
|
|
As a developer
|
|
I want providers to require explicit LLM configuration
|
|
So that FakeListLLM is never silently used as a production fallback
|
|
|
|
@phase1
|
|
Scenario: Provider auto-detection selects first configured provider
|
|
Given a provider-fix settings instance with openai configured
|
|
When I create a provider-fix registry from those settings
|
|
Then the provider-fix default provider type should be "openai"
|
|
|
|
@phase1
|
|
Scenario: No providers configured and mock_providers off raises error
|
|
Given a provider-fix settings instance with no providers configured
|
|
And provider-fix mock_providers is disabled
|
|
When I call provider-fix validate_provider_availability
|
|
Then a provider-fix ValueError should be raised containing "No AI providers configured"
|
|
|
|
@phase1
|
|
Scenario: mock_providers flag enables mock provider
|
|
Given a provider-fix settings instance with mock_providers enabled
|
|
When I build the provider-fix AI provider from the container helper
|
|
Then a provider-fix mock AI provider should be returned
|
|
|
|
@phase1
|
|
Scenario: FakeListLLM is not used as default fallback in PlanGenerationGraph
|
|
When I attempt to create a provider-fix PlanGenerationGraph without an LLM
|
|
Then a provider-fix ValueError should be raised containing "No LLM provider configured"
|
|
|
|
@phase1
|
|
Scenario: FakeListLLM is not used as default fallback in ContextAnalysisAgent
|
|
When I attempt to create a provider-fix ContextAnalysisAgent without an LLM
|
|
Then a provider-fix ValueError should be raised containing "No LLM provider configured"
|
|
|
|
@phase1
|
|
Scenario: FakeListLLM is not used as default fallback in AutoDebugAgent
|
|
When I attempt to create a provider-fix AutoDebugAgent without an LLM
|
|
Then a provider-fix ValueError should be raised containing "No LLM provider configured"
|
|
|
|
@phase1
|
|
Scenario: Provider selection trace logging outputs chosen provider
|
|
Given a provider-fix settings instance with openai configured
|
|
When I create a provider-fix registry and request the default provider
|
|
Then the provider-fix selection should be logged
|
|
|
|
@phase1
|
|
Scenario: resolve_provider_by_name returns provider for configured name
|
|
Given a provider-fix settings instance with openai configured
|
|
When I provider-fix resolve provider by name "openai"
|
|
Then I should receive a provider-fix AI provider instance
|
|
|
|
@phase1
|
|
Scenario: resolve_provider_by_name raises error for unconfigured name
|
|
Given a provider-fix settings instance with no providers configured
|
|
When I provider-fix resolve provider by name "openai"
|
|
Then a provider-fix ValueError should be raised containing "not configured"
|
|
|
|
@phase1
|
|
Scenario: resolve_provider_by_name raises error for unknown name
|
|
Given a provider-fix settings instance with no providers configured
|
|
When I provider-fix resolve provider by name "nonexistent_provider"
|
|
Then a provider-fix ValueError should be raised containing "Unknown provider"
|
|
|
|
@phase1
|
|
Scenario: validate_provider_availability passes with provider configured
|
|
Given a provider-fix settings instance with openai configured
|
|
And provider-fix mock_providers is disabled on that instance
|
|
When I call provider-fix validate_provider_availability
|
|
Then no provider-fix error should be raised
|
|
|
|
@phase1
|
|
Scenario: mock_providers warns outside test mode
|
|
Given a provider-fix settings instance with mock_providers enabled
|
|
And the provider-fix environment is set to "production"
|
|
When I call provider-fix validate_provider_availability
|
|
Then a provider-fix warning should be logged about mock_providers
|
|
|
|
@phase1
|
|
Scenario: Settings mock_providers field defaults to False
|
|
Given I create a provider-fix fresh settings instance
|
|
Then provider-fix mock_providers should be False by default
|
|
|
|
@phase1
|
|
Scenario: Container get_ai_provider respects mock_providers setting
|
|
Given a provider-fix settings instance with mock_providers enabled
|
|
When I call provider-fix get_ai_provider with those settings
|
|
Then a provider-fix mock provider or None should be returned
|
|
|
|
@phase1
|
|
Scenario: PlanGenerationGraph works with explicit LLM
|
|
Given I have a provider-fix FakeListLLM instance for testing
|
|
When I create a provider-fix PlanGenerationGraph with that LLM
|
|
Then the provider-fix graph should be created successfully
|
|
|
|
@phase1
|
|
Scenario: ContextAnalysisAgent works with explicit LLM
|
|
Given I have a provider-fix FakeListLLM instance for testing
|
|
When I create a provider-fix ContextAnalysisAgent with that LLM
|
|
Then the provider-fix agent should be created successfully
|
|
|
|
@phase1
|
|
Scenario: AutoDebugAgent works with explicit LLM
|
|
Given I have a provider-fix FakeListLLM instance for testing
|
|
When I create a provider-fix AutoDebugAgent with that LLM
|
|
Then the provider-fix agent should be created successfully
|