forked from cleveragents/cleveragents-core
freemo
a074b4846f
Remove FakeListLLM as a silent fallback in agent graph constructors (plan_generation.py, context_analysis.py, auto_debug.py). All three now raise ValueError when llm=None, making missing-provider errors explicit. Add Settings.mock_providers flag and validate_provider_availability() method. Update container.get_ai_provider() to check Settings.mock_providers first, with env-var fallback for backward compatibility. Add resolve_provider_by_name() helper to the provider registry and export it from cleveragents.providers. Add structlog trace logging to ProviderRegistry.get_default_provider_type() to record selection reasoning. Update all existing behave step files, robot tests, and benchmarks that relied on the implicit FakeListLLM default to pass an explicit LLM instance instead. Add new BDD tests (features/provider_fixes.feature with 17 scenarios), Robot Framework integration tests (robot/provider_detection_smoke.robot), and ASV benchmarks (benchmarks/provider_selection_bench.py). ISSUES CLOSED: #323
42 lines
1.6 KiB
Plaintext
42 lines
1.6 KiB
Plaintext
*** Settings ***
|
|
Resource ${CURDIR}/common.resource
|
|
Library OperatingSystem
|
|
Library Process
|
|
Suite Setup Setup Test Environment
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
${SRC_DIR} ${CURDIR}/..
|
|
|
|
*** Test Cases ***
|
|
Verify Provider Detection With Mock
|
|
[Documentation] When mock_providers is enabled the container returns a mock provider
|
|
${result}= Run Process ${PYTHON} robot/helper_provider_detection.py mock-detection
|
|
... cwd=${SRC_DIR}
|
|
... env:CLEVERAGENTS_MOCK_PROVIDERS=true
|
|
... env:CLEVERAGENTS_TESTING_USE_MOCK_AI=
|
|
Log Process Failure ${result}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} mock-provider-ok
|
|
|
|
Verify Provider Detection Without Config Fails
|
|
[Documentation] With no API keys and mock_providers off validate_provider_availability raises
|
|
${result}= Run Process ${PYTHON} robot/helper_provider_detection.py no-config-fail
|
|
... cwd=${SRC_DIR}
|
|
... env:CLEVERAGENTS_MOCK_PROVIDERS=
|
|
... env:OPENAI_API_KEY=
|
|
... env:ANTHROPIC_API_KEY=
|
|
... env:GOOGLE_API_KEY=
|
|
... env:CLEVERAGENTS_TESTING_USE_MOCK_AI=
|
|
Log Process Failure ${result}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} no-config-error-ok
|
|
|
|
*** Keywords ***
|
|
Log Process Failure
|
|
[Arguments] ${result}
|
|
Run Keyword If ${result.rc} == 0 Return From Keyword
|
|
Log To Console Process failed with rc=${result.rc}
|
|
Log To Console STDOUT:${\n}${result.stdout}
|
|
Log To Console STDERR:${\n}${result.stderr}
|