[AUTO-INF-2] Coverage gap: providers/fallback_selector.py and providers/cost_table.py lack dedicated BDD feature files #10250

New issue

Open

opened 2026-04-17 10:46:06 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-17 10:46:06 +00:00

Owner

Summary

Two recently added modules in src/cleveragents/providers/ — fallback_selector.py (167 lines) and cost_table.py — were introduced in commit 134dce89 as part of the budget/cost-tracking feature but have no dedicated Behave feature files. Both modules are exercised only indirectly through higher-level cost-controls scenarios, leaving their internal logic paths untested and invisible to the slipcover 97% gate.

Findings

`providers/fallback_selector.py`

167 lines of provider fallback selection logic
No dedicated feature file found under features/
Tested only indirectly via features/cost_controls.feature and features/cost_budgets.feature step files
Internal decision branches (e.g., provider ranking, fallback chain traversal, exhaustion handling) are not explicitly asserted
The module was added in commit 134dce89 alongside cost_tracker.py and cost_table.py

`providers/cost_table.py`

Contains cost-per-token lookup tables for all supported providers
No dedicated feature file found under features/
No unit or integration tests explicitly exercise the table lookup paths
Edge cases (unknown model, zero-cost model, provider not in table) are untested

Impact

Both modules contribute to the 97% slipcover threshold but their internal branches are not explicitly covered
Regressions in fallback selection logic (e.g., wrong provider chosen when primary is exhausted) would go undetected
Cost table lookup errors (e.g., KeyError for unknown models) would surface only at runtime

Recommendations

Add features/provider_fallback_coverage.feature with Behave scenarios covering:
- Happy path: primary provider available, fallback not triggered
- Fallback triggered: primary exhausted, secondary selected
- All providers exhausted: appropriate error raised
- Provider ranking order preserved
Add features/provider_cost_table_coverage.feature with scenarios covering:
- Known model lookup returns correct cost-per-token
- Unknown model lookup raises or returns sentinel value
- All supported providers have entries in the table
Wire new scenarios into the existing coverage_report nox session so slipcover instruments these paths.

Acceptance Criteria

features/provider_fallback_coverage.feature exists with ≥3 passing scenarios covering fallback selection branches
features/provider_cost_table_coverage.feature exists with ≥2 passing scenarios covering table lookup paths
nox -s coverage_report shows providers/fallback_selector.py and providers/cost_table.py branch coverage ≥ 97%
No pragma: no cover annotations added to suppress gaps

Duplicate Check

Check	Query	Result
1. Open issues keyword search	`fallback_selector`, `fallback selector`, `cost_table`, `cost table`	No matches found across pages 1–6 of open issues
2. Cross-area search	`[AUTO-INF-*]` coverage issues (#9939, #9938, #9937, #9893, #9702, #9886, #9790)	None cover `providers/fallback_selector.py` or `providers/cost_table.py`
3. Closed issues search	`fallback`, `cost_table`, `provider coverage`	No matches found across pages 1–5 of closed issues
4. Related issues reviewed	#9893 (CLI envelopes, A2A events, session DI), #9702 (TUI, LSP, MCP), #9886 (benchmark coverage)	None address providers/ module coverage
5. Uncertainty check	Scope is specific to `providers/fallback_selector.py` and `providers/cost_table.py`	Confident this is a new gap not covered by any existing issue

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

## Summary Two recently added modules in `src/cleveragents/providers/` — `fallback_selector.py` (167 lines) and `cost_table.py` — were introduced in commit `134dce89` as part of the budget/cost-tracking feature but have **no dedicated Behave feature files**. Both modules are exercised only indirectly through higher-level cost-controls scenarios, leaving their internal logic paths untested and invisible to the slipcover 97% gate. ## Findings ### `providers/fallback_selector.py` - **167 lines** of provider fallback selection logic - No dedicated feature file found under `features/` - Tested only indirectly via `features/cost_controls.feature` and `features/cost_budgets.feature` step files - Internal decision branches (e.g., provider ranking, fallback chain traversal, exhaustion handling) are not explicitly asserted - The module was added in commit `134dce89` alongside `cost_tracker.py` and `cost_table.py` ### `providers/cost_table.py` - Contains cost-per-token lookup tables for all supported providers - No dedicated feature file found under `features/` - No unit or integration tests explicitly exercise the table lookup paths - Edge cases (unknown model, zero-cost model, provider not in table) are untested ## Impact - Both modules contribute to the 97% slipcover threshold but their internal branches are not explicitly covered - Regressions in fallback selection logic (e.g., wrong provider chosen when primary is exhausted) would go undetected - Cost table lookup errors (e.g., `KeyError` for unknown models) would surface only at runtime ## Recommendations 1. Add `features/provider_fallback_coverage.feature` with Behave scenarios covering: - Happy path: primary provider available, fallback not triggered - Fallback triggered: primary exhausted, secondary selected - All providers exhausted: appropriate error raised - Provider ranking order preserved 2. Add `features/provider_cost_table_coverage.feature` with scenarios covering: - Known model lookup returns correct cost-per-token - Unknown model lookup raises or returns sentinel value - All supported providers have entries in the table 3. Wire new scenarios into the existing `coverage_report` nox session so slipcover instruments these paths. ## Acceptance Criteria - [ ] `features/provider_fallback_coverage.feature` exists with ≥3 passing scenarios covering fallback selection branches - [ ] `features/provider_cost_table_coverage.feature` exists with ≥2 passing scenarios covering table lookup paths - [ ] `nox -s coverage_report` shows `providers/fallback_selector.py` and `providers/cost_table.py` branch coverage ≥ 97% - [ ] No `pragma: no cover` annotations added to suppress gaps ### Duplicate Check | Check | Query | Result | |-------|-------|--------| | 1. Open issues keyword search | `fallback_selector`, `fallback selector`, `cost_table`, `cost table` | No matches found across pages 1–6 of open issues | | 2. Cross-area search | `[AUTO-INF-*]` coverage issues (#9939, #9938, #9937, #9893, #9702, #9886, #9790) | None cover `providers/fallback_selector.py` or `providers/cost_table.py` | | 3. Closed issues search | `fallback`, `cost_table`, `provider coverage` | No matches found across pages 1–5 of closed issues | | 4. Related issues reviewed | #9893 (CLI envelopes, A2A events, session DI), #9702 (TUI, LSP, MCP), #9886 (benchmark coverage) | None address providers/ module coverage | | 5. Uncertainty check | Scope is specific to `providers/fallback_selector.py` and `providers/cost_table.py` | Confident this is a new gap not covered by any existing issue | --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

HAL9000 added the

labels

2026-04-17 11:20:50 +00:00

HAL9000 referenced this issue

2026-04-18 08:16:07 +00:00

[AUTO-INF-4] Flaky test: uko_indexer_watcher_steps.py debounce wait uses patched time.sleep — debounce never fires in CI #10295

HAL9000 referenced this issue

2026-04-18 08:26:53 +00:00