freemo
2917aa7ddb
CI / lint (pull_request) Successful in 24s
CI / typecheck (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 46s
CI / security (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 38s
CI / helm (pull_request) Successful in 24s
CI / unit_tests (pull_request) Successful in 6m47s
CI / e2e_tests (pull_request) Successful in 18m26s
CI / docker (pull_request) Successful in 1m29s
CI / integration_tests (pull_request) Successful in 22m58s
CI / coverage (pull_request) Successful in 10m55s
CI / status-check (pull_request) Successful in 2s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 56m49s
Extend _check_providers() in system.py to report diagnostic status for all 9 providers supported by ProviderRegistry: OpenAI, Anthropic, Google, Azure, OpenRouter, Gemini, Cohere, Groq, and Together AI. Previously only 4 providers (OpenAI, Anthropic, Google, OpenRouter) were checked, leaving users of Groq, Together AI, Cohere, Azure, and Gemini with no diagnostic feedback about their provider configuration. Changes: - Add Azure (AZURE_OPENAI_API_KEY), Gemini (GEMINI_API_KEY), Cohere (COHERE_API_KEY), Groq (GROQ_API_KEY), and Together AI (TOGETHER_API_KEY) to the provider_checks list - Add Behave feature file with 11 scenarios covering all 9 providers (presence, OK status when configured, WARN with recommendation when not) ISSUES CLOSED: #3422
102 lines
6.0 KiB
Gherkin
102 lines
6.0 KiB
Gherkin
@diagnostics
|
|
Feature: Diagnostics provider coverage — all 9 providers checked
|
|
As a developer ensuring thorough diagnostic coverage
|
|
I want _check_providers() to report status for all 9 supported providers
|
|
So that users of Groq, Together AI, Cohere, Azure, and Gemini see diagnostic feedback
|
|
|
|
# ===========================================================================
|
|
# _check_providers — all 9 providers present
|
|
# ===========================================================================
|
|
|
|
Scenario: check_providers returns results for all 9 supported providers
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check results should contain exactly 9 entries
|
|
And the provider check results should include provider "openai"
|
|
And the provider check results should include provider "anthropic"
|
|
And the provider check results should include provider "google"
|
|
And the provider check results should include provider "azure"
|
|
And the provider check results should include provider "openrouter"
|
|
And the provider check results should include provider "gemini"
|
|
And the provider check results should include provider "cohere"
|
|
And the provider check results should include provider "groq"
|
|
And the provider check results should include provider "together"
|
|
|
|
# ===========================================================================
|
|
# _check_providers — configured provider shows OK
|
|
# ===========================================================================
|
|
|
|
Scenario: check_providers shows OK status for a configured provider
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with "groq" configured
|
|
Then the provider check result for "groq" should have status "ok"
|
|
And the provider check result for "groq" should have details "configured"
|
|
And the provider check result for "groq" should have no recommendation
|
|
|
|
Scenario: check_providers shows OK status for azure when configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with "azure" configured
|
|
Then the provider check result for "azure" should have status "ok"
|
|
And the provider check result for "azure" should have details "configured"
|
|
And the provider check result for "azure" should have no recommendation
|
|
|
|
Scenario: check_providers shows OK status for together when configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with "together" configured
|
|
Then the provider check result for "together" should have status "ok"
|
|
And the provider check result for "together" should have details "configured"
|
|
And the provider check result for "together" should have no recommendation
|
|
|
|
Scenario: check_providers shows OK status for cohere when configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with "cohere" configured
|
|
Then the provider check result for "cohere" should have status "ok"
|
|
And the provider check result for "cohere" should have details "configured"
|
|
And the provider check result for "cohere" should have no recommendation
|
|
|
|
Scenario: check_providers shows OK status for gemini when configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with "gemini" configured
|
|
Then the provider check result for "gemini" should have status "ok"
|
|
And the provider check result for "gemini" should have details "configured"
|
|
And the provider check result for "gemini" should have no recommendation
|
|
|
|
# ===========================================================================
|
|
# _check_providers — unconfigured provider shows WARN with recommendation
|
|
# ===========================================================================
|
|
|
|
Scenario: check_providers shows WARN for groq when not configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check result for "groq" should have status "warn"
|
|
And the provider check result for "groq" should have details "missing"
|
|
And the provider check result for "groq" should have a recommendation mentioning "GROQ_API_KEY"
|
|
|
|
Scenario: check_providers shows WARN for together when not configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check result for "together" should have status "warn"
|
|
And the provider check result for "together" should have details "missing"
|
|
And the provider check result for "together" should have a recommendation mentioning "TOGETHER_API_KEY"
|
|
|
|
Scenario: check_providers shows WARN for cohere when not configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check result for "cohere" should have status "warn"
|
|
And the provider check result for "cohere" should have details "missing"
|
|
And the provider check result for "cohere" should have a recommendation mentioning "COHERE_API_KEY"
|
|
|
|
Scenario: check_providers shows WARN for azure when not configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check result for "azure" should have status "warn"
|
|
And the provider check result for "azure" should have details "missing"
|
|
And the provider check result for "azure" should have a recommendation mentioning "AZURE_OPENAI_API_KEY"
|
|
|
|
Scenario: check_providers shows WARN for gemini when not configured
|
|
Given the diagnostics provider check module is loaded
|
|
When I call check_providers with no providers configured
|
|
Then the provider check result for "gemini" should have status "warn"
|
|
And the provider check result for "gemini" should have details "missing"
|
|
And the provider check result for "gemini" should have a recommendation mentioning "GEMINI_API_KEY"
|