fix(providers): add ProviderType.GEMINI to ProviderRegistry.FALLBACK_ORDER
CI / benchmark-publish (pull_request) Has been skipped
CI / helm (pull_request) Successful in 32s
CI / build (pull_request) Successful in 53s
CI / lint (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 1m15s
CI / typecheck (pull_request) Successful in 1m31s
CI / security (pull_request) Successful in 1m34s
CI / push-validation (pull_request) Successful in 21s
CI / integration_tests (pull_request) Successful in 3m30s
CI / e2e_tests (pull_request) Successful in 3m54s
CI / unit_tests (pull_request) Failing after 4m42s
CI / docker (pull_request) Has been skipped
CI / coverage (pull_request) Successful in 11m25s
CI / status-check (pull_request) Failing after 3s

ProviderType.GEMINI was missing from FALLBACK_ORDER, causing users who
configure only GEMINI_API_KEY (without GOOGLE_API_KEY) to never have
Gemini auto-selected as the default provider. get_default_provider_type()
would return None instead of ProviderType.GEMINI for Gemini-only setups.

Fix: add ProviderType.GEMINI to FALLBACK_ORDER after ProviderType.GOOGLE,
since GEMINI and GOOGLE share the same underlying model family but use
different API keys (GEMINI_API_KEY vs GOOGLE_API_KEY).

Also adds:
- TDD regression test (tdd_gemini_fallback_order_4750.feature) with
  @tdd_issue @tdd_issue_4750 tags (without @tdd_expected_fail since fix applied)
- New BDD scenarios in provider_registry_coverage.feature verifying GEMINI
  is in FALLBACK_ORDER and that Gemini-only users get GEMINI as default
- New step definitions for GEMINI fallback order verification

ISSUES CLOSED: #4750
This commit is contained in:
2026-04-28 09:37:24 +00:00
parent 98bc7c6b5d
commit be4dee2a5f
5 changed files with 114 additions and 0 deletions
@@ -466,3 +466,17 @@ Feature: Provider Registry Coverage
And Anthropic default should be "claude-sonnet-4-20250514"
And Google default should be "gemini-2.0-flash"
And Groq default should be "llama-3.1-70b-versatile"
@unit @providers @registry
Scenario: GEMINI is included in provider fallback order
Given I have the ProviderRegistry class
When I check the FALLBACK_ORDER
Then GEMINI should be in the fallback order
And GEMINI should appear after GOOGLE in the fallback order
@unit @providers @registry
Scenario: Gemini-only user gets GEMINI as default provider via fallback order
Given I have a ProviderRegistry with "gemini" API key set
And CLEVERAGENTS_DEFAULT_PROVIDER is not set
When I call get_default_provider_type
Then the provider registry result should be ProviderType.GEMINI
+20
View File
@@ -922,3 +922,23 @@ def step_impl_default_models(context: Any, provider: str, model: str) -> None:
# -- Hooks ------------------------------------------------------------------
# Hook logic lives in features/environment.py to ensure scenarios reset state.
@then("GEMINI should be in the fallback order")
def step_impl_gemini_in_fallback_order(context: Any) -> None:
provider_enum = _ensure_provider_type(context)
assert provider_enum.GEMINI in context.fallback_order, (
f"Expected ProviderType.GEMINI to be in FALLBACK_ORDER but it was not found. "
f"FALLBACK_ORDER: {context.fallback_order}"
)
@then("GEMINI should appear after GOOGLE in the fallback order")
def step_impl_gemini_after_google(context: Any) -> None:
provider_enum = _ensure_provider_type(context)
google_idx = context.fallback_order.index(provider_enum.GOOGLE)
gemini_idx = context.fallback_order.index(provider_enum.GEMINI)
assert gemini_idx > google_idx, (
f"Expected ProviderType.GEMINI (index {gemini_idx}) to appear after "
f"ProviderType.GOOGLE (index {google_idx}) in FALLBACK_ORDER."
)
@@ -0,0 +1,68 @@
"""Step definitions for TDD test: ProviderType.GEMINI missing from FALLBACK_ORDER.
Issue: #4750 — ProviderType.GEMINI missing from ProviderRegistry.FALLBACK_ORDER
TDD Issue: #10896
This test captures the bug: when only GEMINI_API_KEY is set (without GOOGLE_API_KEY),
get_default_provider_type() returns None instead of ProviderType.GEMINI because
GEMINI is absent from FALLBACK_ORDER.
The @tdd_expected_fail tag has been removed because the fix has been applied
(ProviderType.GEMINI added to FALLBACK_ORDER). The test now passes normally.
"""
from __future__ import annotations
import os
from typing import Any
from unittest.mock import MagicMock
from behave import given, then, when # type: ignore[import-untyped]
from cleveragents.providers.registry import ProviderRegistry, ProviderType
def _make_gemini_only_settings() -> Any:
"""Return a fake Settings-like object with only gemini_api_key set."""
settings = MagicMock()
settings.openai_api_key = None
settings.anthropic_api_key = None
settings.google_api_key = None
settings.gemini_api_key = "sk-gemini-test-key"
settings.azure_api_key = None
settings.openrouter_api_key = None
settings.cohere_api_key = None
settings.groq_api_key = None
settings.together_api_key = None
settings.default_provider = None
settings.default_model = None
settings.azure_openai_endpoint = None
settings.azure_openai_api_version = None
settings.azure_openai_deployment = None
settings.openrouter_organization = None
return settings
@given("a ProviderRegistry configured with only gemini_api_key set")
def step_tdd_registry_gemini_only(context: Any) -> None:
context.tdd_registry = ProviderRegistry(settings=_make_gemini_only_settings())
@given("CLEVERAGENTS_DEFAULT_PROVIDER is not set for tdd test")
def step_tdd_unset_default_provider(context: Any) -> None:
context.original_provider_env = os.environ.get("CLEVERAGENTS_DEFAULT_PROVIDER")
os.environ.pop("CLEVERAGENTS_DEFAULT_PROVIDER", None)
@when("I call get_default_provider_type on the tdd registry")
def step_tdd_call_get_default_provider_type(context: Any) -> None:
context.tdd_result = context.tdd_registry.get_default_provider_type()
@then("the tdd result should be ProviderType.GEMINI")
def step_tdd_result_should_be_gemini(context: Any) -> None:
assert context.tdd_result == ProviderType.GEMINI, (
f"Expected ProviderType.GEMINI but got {context.tdd_result!r}. "
f"ProviderType.GEMINI should be in ProviderRegistry.FALLBACK_ORDER "
f"so that Gemini-only users get auto-selected provider."
)
@@ -0,0 +1,11 @@
Feature: TDD: ProviderType.GEMINI missing from ProviderRegistry.FALLBACK_ORDER
As a user who configures only GEMINI_API_KEY
I want get_default_provider_type() to return ProviderType.GEMINI via the fallback chain
So that Gemini-only users can use auto-discovery without setting GOOGLE_API_KEY
@tdd_issue @tdd_issue_4750
Scenario: Gemini-only user gets GEMINI as default provider via fallback order
Given a ProviderRegistry configured with only gemini_api_key set
And CLEVERAGENTS_DEFAULT_PROVIDER is not set for tdd test
When I call get_default_provider_type on the tdd registry
Then the tdd result should be ProviderType.GEMINI
+1
View File
@@ -205,6 +205,7 @@ class ProviderRegistry:
ProviderType.OPENAI,
ProviderType.ANTHROPIC,
ProviderType.GOOGLE,
ProviderType.GEMINI,
ProviderType.AZURE,
ProviderType.OPENROUTER,
ProviderType.GROQ,