fix(provider): remove FakeListLLM defaults #459

2026-02-27T01:16:29Z

freemo commented

2026-02-27 01:16:29 +00:00

Summary

Remove FakeListLLM as a silent fallback in plan_generation.py, context_analysis.py, and auto_debug.py — all three now raise ValueError when llm=None, making missing-provider errors explicit
Add Settings.mock_providers flag with validate_provider_availability() method; update container.get_ai_provider() to check this flag first (with env-var fallback for backward compatibility)
Add resolve_provider_by_name() helper and structlog trace logging to ProviderRegistry.get_default_provider_type()
Update all existing behave steps, robot tests, and benchmarks that relied on the implicit FakeListLLM default to pass an explicit LLM instance
Add 17 new BDD scenarios (features/provider_fixes.feature), Robot Framework integration tests (robot/provider_detection_smoke.robot), and ASV benchmarks (benchmarks/provider_selection_bench.py)

Quality Gates

Gate	Status
`nox -s lint`	Pass
`nox -s typecheck`	Pass (0 errors)
`nox -s unit_tests`	Pass (315 features, 6655 scenarios, 0 failures)
`nox -s integration_tests`	Pass (854 tests, 854 passed)
`nox -s build`	Pass
`nox -s security_scan`	Pass
`nox -s dead_code`	Pass
`nox -s docs`	Pass
`nox -s coverage_report`	70.06% — pre-existing; unchanged from master (noxfile.py and pyproject.toml coverage config not modified)

Files Changed (28)

Source (8 modified):

src/cleveragents/agents/graphs/plan_generation.py — Removed FakeListLLM import + default methods
src/cleveragents/agents/graphs/context_analysis.py — Removed FakeListLLM fallback
src/cleveragents/agents/graphs/auto_debug.py — Removed FakeListLLM fallback
src/cleveragents/config/settings.py — Added mock_providers field + validate_provider_availability()
src/cleveragents/providers/registry.py — Added structlog logging + resolve_provider_by_name()
src/cleveragents/providers/__init__.py — Exported resolve_provider_by_name
src/cleveragents/application/container.py — Updated get_ai_provider() for mock_providers
src/cleveragents/application/services/plan_service.py — Pass llm=self._llm to AutoDebugAgent

New test/benchmark files (5):

features/provider_fixes.feature — 17 BDD scenarios
features/steps/provider_fixes_steps.py — Step definitions
robot/provider_detection_smoke.robot — 2 Robot tests
robot/helper_provider_detection.py — Robot helper
benchmarks/provider_selection_bench.py — ASV benchmarks

Updated test files (14): All modified to pass explicit FakeListLLM instead of relying on removed defaults.

ISSUES CLOSED: #323

## Summary - Remove FakeListLLM as a silent fallback in `plan_generation.py`, `context_analysis.py`, and `auto_debug.py` — all three now raise `ValueError` when `llm=None`, making missing-provider errors explicit - Add `Settings.mock_providers` flag with `validate_provider_availability()` method; update `container.get_ai_provider()` to check this flag first (with env-var fallback for backward compatibility) - Add `resolve_provider_by_name()` helper and structlog trace logging to `ProviderRegistry.get_default_provider_type()` - Update all existing behave steps, robot tests, and benchmarks that relied on the implicit FakeListLLM default to pass an explicit LLM instance - Add 17 new BDD scenarios (`features/provider_fixes.feature`), Robot Framework integration tests (`robot/provider_detection_smoke.robot`), and ASV benchmarks (`benchmarks/provider_selection_bench.py`) ## Quality Gates | Gate | Status | |------|--------| | `nox -s lint` | Pass | | `nox -s typecheck` | Pass (0 errors) | | `nox -s unit_tests` | Pass (315 features, 6655 scenarios, 0 failures) | | `nox -s integration_tests` | Pass (854 tests, 854 passed) | | `nox -s build` | Pass | | `nox -s security_scan` | Pass | | `nox -s dead_code` | Pass | | `nox -s docs` | Pass | | `nox -s coverage_report` | 70.06% — pre-existing; unchanged from master (noxfile.py and pyproject.toml coverage config not modified) | ## Files Changed (28) **Source (8 modified):** - `src/cleveragents/agents/graphs/plan_generation.py` — Removed FakeListLLM import + default methods - `src/cleveragents/agents/graphs/context_analysis.py` — Removed FakeListLLM fallback - `src/cleveragents/agents/graphs/auto_debug.py` — Removed FakeListLLM fallback - `src/cleveragents/config/settings.py` — Added `mock_providers` field + `validate_provider_availability()` - `src/cleveragents/providers/registry.py` — Added structlog logging + `resolve_provider_by_name()` - `src/cleveragents/providers/__init__.py` — Exported `resolve_provider_by_name` - `src/cleveragents/application/container.py` — Updated `get_ai_provider()` for mock_providers - `src/cleveragents/application/services/plan_service.py` — Pass `llm=self._llm` to AutoDebugAgent **New test/benchmark files (5):** - `features/provider_fixes.feature` — 17 BDD scenarios - `features/steps/provider_fixes_steps.py` — Step definitions - `robot/provider_detection_smoke.robot` — 2 Robot tests - `robot/helper_provider_detection.py` — Robot helper - `benchmarks/provider_selection_bench.py` — ASV benchmarks **Updated test files (14):** All modified to pass explicit `FakeListLLM` instead of relying on removed defaults. ISSUES CLOSED: #323

freemo referenced this pull request

2026-02-27 01:17:15 +00:00

fix(provider): remove FakeListLLM defaults #323

freemo force-pushed feature/m4-provider-fixes from 577031d219 to feeb8a06c1

2026-02-27 02:38:53 +00:00

Compare

freemo scheduled this pull request to auto merge when all checks succeed 2026-02-27 02:39:10 +00:00

freemo force-pushed feature/m4-provider-fixes from feeb8a06c1 to 9c108fa2a5

2026-02-27 14:14:36 +00:00

Compare

freemo force-pushed feature/m4-provider-fixes from 9c108fa2a5 to a074b4846f

2026-02-27 14:47:26 +00:00