184 lines
8.1 KiB
Gherkin
184 lines
8.1 KiB
Gherkin
Feature: AI Models Errors Coverage
|
|
As a developer
|
|
I want to test the AI models error classes
|
|
So that error handling is properly validated and coverage is improved
|
|
|
|
Scenario: Create ModelError with overloaded error type
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "OVERLOADED"
|
|
And I set retriable to true
|
|
And I set retryafterseconds to 30
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrOverloaded"
|
|
And the retriable flag should be true
|
|
And the retry after seconds should be 30
|
|
|
|
Scenario: Create ModelError with context too long error
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "CONTEXT_TOO_LONG"
|
|
And I set retriable to false
|
|
And I set retryafterseconds to 0
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrContextTooLong"
|
|
And the retriable flag should be false
|
|
And the retry after seconds should be 0
|
|
|
|
Scenario: Create ModelError with rate limited error
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "RATE_LIMITED"
|
|
And I set retriable to true
|
|
And I set retryafterseconds to 60
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrRateLimited"
|
|
And the retriable flag should be true
|
|
And the retry after seconds should be 60
|
|
|
|
Scenario: Create ModelError with subscription quota exhausted
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "SUBSCRIPTION_QUOTA_EXHAUSTED"
|
|
And I set retriable to false
|
|
And I set retryafterseconds to 3600
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrSubscriptionQuotaExhausted"
|
|
And the retriable flag should be false
|
|
And the retry after seconds should be 3600
|
|
|
|
Scenario: Create ModelError with other error type
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "OTHER"
|
|
And I set retriable to true
|
|
And I set retryafterseconds to 10
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrOther"
|
|
And the retriable flag should be true
|
|
And the retry after seconds should be 10
|
|
|
|
Scenario: Create ModelError with cache support error
|
|
Given I import the ModelError class
|
|
When I create a ModelError with kind "CACHE_SUPPORT"
|
|
And I set retriable to false
|
|
And I set retryafterseconds to 0
|
|
Then the ModelError instance should be created successfully
|
|
And the kind should be "ErrCacheSupport"
|
|
And the retriable flag should be false
|
|
And the retry after seconds should be 0
|
|
|
|
Scenario: Validate ModelError configuration
|
|
Given I import the ModelError class
|
|
When I create a ModelError with valid data
|
|
Then the model configuration should have str_strip_whitespace as true
|
|
And the model configuration should have validate_assignment as true
|
|
And the model configuration should have arbitrary_types_allowed as false
|
|
And the model configuration should have populate_by_name as true
|
|
And the model configuration should have use_enum_values as true
|
|
|
|
Scenario: Create FallbackResult with error fallback type
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult with fallback type "ERROR"
|
|
And I set isfallback to true
|
|
And I set modelroleconfig to None
|
|
And I set basemodelconfig to None
|
|
Then the FallbackResult instance should be created successfully
|
|
And the fallback type should be "FallbackTypeError"
|
|
And the is fallback flag should be true
|
|
And the model role config should be None
|
|
And the base model config should be None
|
|
|
|
Scenario: Create FallbackResult with context fallback type
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult with fallback type "CONTEXT"
|
|
And I set isfallback to true
|
|
And I set modelroleconfig to None
|
|
And I set basemodelconfig to None
|
|
Then the FallbackResult instance should be created successfully
|
|
And the fallback type should be "FallbackTypeContext"
|
|
And the is fallback flag should be true
|
|
|
|
Scenario: Create FallbackResult with provider fallback type
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult with fallback type "PROVIDER"
|
|
And I set isfallback to false
|
|
And I set modelroleconfig to None
|
|
And I set basemodelconfig to None
|
|
Then the FallbackResult instance should be created successfully
|
|
And the fallback type should be "FallbackTypeProvider"
|
|
And the is fallback flag should be false
|
|
|
|
Scenario: Create FallbackResult with model role config
|
|
Given I import the FallbackResult class
|
|
And I have a valid ModelRoleConfig instance
|
|
When I create a FallbackResult with fallback type "ERROR"
|
|
And I set isfallback to true
|
|
And I set modelroleconfig to the valid instance
|
|
And I set basemodelconfig to None
|
|
Then the FallbackResult instance should be created successfully
|
|
And the model role config should not be None
|
|
And the model role config role should be "assistant"
|
|
|
|
Scenario: Create FallbackResult with base model config
|
|
Given I import the FallbackResult class
|
|
And I have a valid BaseModelConfig instance
|
|
When I create a FallbackResult with fallback type "CONTEXT"
|
|
And I set isfallback to true
|
|
And I set modelroleconfig to None
|
|
And I set basemodelconfig to the valid instance
|
|
Then the FallbackResult instance should be created successfully
|
|
And the base model config should not be None
|
|
And the base model config model_tag should be "gpt-4"
|
|
|
|
Scenario: Create FallbackResult with both configs
|
|
Given I import the FallbackResult class
|
|
And I have a valid ModelRoleConfig instance
|
|
And I have a valid BaseModelConfig instance
|
|
When I create a FallbackResult with fallback type "PROVIDER"
|
|
And I set isfallback to false
|
|
And I set modelroleconfig to the valid instance
|
|
And I set basemodelconfig to the valid instance
|
|
Then the FallbackResult instance should be created successfully
|
|
And the model role config should not be None
|
|
And the base model config should not be None
|
|
|
|
Scenario: Validate FallbackResult configuration
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult with valid data
|
|
Then the model configuration should have str_strip_whitespace as true
|
|
And the model configuration should have validate_assignment as true
|
|
And the model configuration should have arbitrary_types_allowed as false
|
|
And the model configuration should have populate_by_name as true
|
|
And the model configuration should have use_enum_values as true
|
|
|
|
Scenario: ModelError with invalid kind raises validation error
|
|
Given I import the ModelError class
|
|
When I try to create a ModelError with invalid kind "INVALID_KIND"
|
|
Then a validation error should be raised
|
|
And the error should mention "Input should be"
|
|
|
|
Scenario: FallbackResult with invalid fallback type raises error
|
|
Given I import the FallbackResult class
|
|
When I try to create a FallbackResult with invalid fallback type "INVALID_TYPE"
|
|
Then a validation error should be raised
|
|
And the error should mention "Input should be"
|
|
|
|
Scenario: Test ModelError field aliases
|
|
Given I import the ModelError class
|
|
When I create a ModelError using field aliases
|
|
Then the ModelError should accept the aliased fields
|
|
And the values should be properly mapped
|
|
|
|
Scenario: Test FallbackResult field aliases
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult using field aliases
|
|
Then the FallbackResult should accept the aliased fields
|
|
And the values should be properly mapped
|
|
|
|
Scenario: ModelError strips whitespace from string fields
|
|
Given I import the ModelError class
|
|
When I create a ModelError with whitespace in enum values
|
|
Then the whitespace should be stripped from strings
|
|
And the model should be created successfully
|
|
|
|
Scenario: FallbackResult strips whitespace from string fields
|
|
Given I import the FallbackResult class
|
|
When I create a FallbackResult with whitespace in enum values
|
|
Then the whitespace should be stripped from strings
|
|
And the model should be created successfully |