TDD: `FallbackSelector.select()` skips per-plan budget check, allowing plan-level budget overruns #10471

New issue

Open

opened 2026-04-18 10:01:09 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-18 10:01:09 +00:00

Owner

Metadata

Commit message: test(providers): add failing behave scenario for missing per-plan budget check in FallbackSelector.select()
Branch name: tdd/mN-fallback-plan-budget

Background and Context

cleveragents.providers.fallback_selector.FallbackSelector.select() (approximately lines 110–145 in src/cleveragents/providers/fallback_selector.py) only checks the daily budget but never checks the per-plan budget. This means a plan that has exhausted its plan-level budget can still select a provider and continue incurring costs, leading to uncontrolled plan-level budget overruns.

This TDD issue must be created and merged before the corresponding bug fix issue. The failing Behave scenario serves as the executable specification that proves the bug exists and will gate the bug fix work.

Expected Behavior

FallbackSelector.select() should check both the daily budget and the per-plan budget before returning a provider. If the per-plan budget is exhausted, select() should return no provider and indicate the reason as "plan budget exceeded".

Acceptance Criteria

Behave scenario written under features/providers/test_fallback_selector_budget.feature
Scenario tagged with @tdd_issue, @tdd_issue_<N>, and @tdd_expected_fail
Assertion uses AssertionError only (not ValueError, RuntimeError, etc.)
Scenario fails (assertion fires) when run against the current unfixed code
CI passes because @tdd_expected_fail inverts the result
TDD PR merged to master before bug fix work begins

Subtasks

Write the failing Behave scenario in features/providers/test_fallback_selector_budget.feature
Implement step definitions for the new scenario
Tag the scenario with @tdd_issue, @tdd_issue_<BUG_ISSUE_NUMBER>, and @tdd_expected_fail
Verify the scenario fails against current unfixed code (assertion fires)
Verify CI passes due to @tdd_expected_fail tag inverting the result
Open TDD PR targeting master on branch tdd/mN-fallback-plan-budget
Merge TDD PR before bug fix work begins
Update bug issue to depend on this TDD issue once bug issue is created

Summary

Write a failing Behave scenario that proves the missing per-plan budget enforcement in cleveragents.providers.fallback_selector.FallbackSelector.select() — the method only checks the daily budget but never checks the per-plan budget, so a plan that has exhausted its plan-level budget can still select a provider and continue incurring costs.

Scenario to write

@tdd_issue @tdd_issue_<BUG_ISSUE_NUMBER> @tdd_expected_fail
Scenario: FallbackSelector respects per-plan budget limits
  Given a FallbackSelector with a cost tracker where the plan budget is exhausted
  And the daily budget has not been exceeded
  When select() is called
  Then no provider should be returned
  And the skip reason should indicate "plan budget exceeded"

Note: Replace <BUG_ISSUE_NUMBER> with the actual bug issue number once it is created. The bug issue will depend on (be blocked by) this TDD issue.

Location

Module: cleveragents.providers.fallback_selector
Class: FallbackSelector
Method: select (approximately lines 110–145 in src/cleveragents/providers/fallback_selector.py)
Test file: features/providers/test_fallback_selector_budget.feature

Definition of Done

Failing test merged to master on a tdd/mN-fallback-plan-budget branch
This issue closed after TDD PR is merged
Bug issue updated to depend on this TDD issue

Automated by CleverAgents Bot
Agent: new-issue-creator

Automated by CleverAgents Bot
Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor [AUTO-BUG-8]

## Metadata - **Commit message:** `test(providers): add failing behave scenario for missing per-plan budget check in FallbackSelector.select()` - **Branch name:** `tdd/mN-fallback-plan-budget` ## Background and Context `cleveragents.providers.fallback_selector.FallbackSelector.select()` (approximately lines 110–145 in `src/cleveragents/providers/fallback_selector.py`) only checks the daily budget but never checks the per-plan budget. This means a plan that has exhausted its plan-level budget can still select a provider and continue incurring costs, leading to uncontrolled plan-level budget overruns. This TDD issue must be created and merged **before** the corresponding bug fix issue. The failing Behave scenario serves as the executable specification that proves the bug exists and will gate the bug fix work. ## Expected Behavior `FallbackSelector.select()` should check both the daily budget **and** the per-plan budget before returning a provider. If the per-plan budget is exhausted, `select()` should return no provider and indicate the reason as "plan budget exceeded". ## Acceptance Criteria - [ ] Behave scenario written under `features/providers/test_fallback_selector_budget.feature` - [ ] Scenario tagged with `@tdd_issue`, `@tdd_issue_<N>`, and `@tdd_expected_fail` - [ ] Assertion uses `AssertionError` only (not `ValueError`, `RuntimeError`, etc.) - [ ] Scenario fails (assertion fires) when run against the current unfixed code - [ ] CI passes because `@tdd_expected_fail` inverts the result - [ ] TDD PR merged to master before bug fix work begins ## Subtasks - [ ] Write the failing Behave scenario in `features/providers/test_fallback_selector_budget.feature` - [ ] Implement step definitions for the new scenario - [ ] Tag the scenario with `@tdd_issue`, `@tdd_issue_<BUG_ISSUE_NUMBER>`, and `@tdd_expected_fail` - [ ] Verify the scenario fails against current unfixed code (assertion fires) - [ ] Verify CI passes due to `@tdd_expected_fail` tag inverting the result - [ ] Open TDD PR targeting `master` on branch `tdd/mN-fallback-plan-budget` - [ ] Merge TDD PR before bug fix work begins - [ ] Update bug issue to depend on this TDD issue once bug issue is created ## Summary Write a failing Behave scenario that proves the missing per-plan budget enforcement in `cleveragents.providers.fallback_selector.FallbackSelector.select()` — the method only checks the daily budget but never checks the per-plan budget, so a plan that has exhausted its plan-level budget can still select a provider and continue incurring costs. ## Scenario to write ```gherkin @tdd_issue @tdd_issue_<BUG_ISSUE_NUMBER> @tdd_expected_fail Scenario: FallbackSelector respects per-plan budget limits Given a FallbackSelector with a cost tracker where the plan budget is exhausted And the daily budget has not been exceeded When select() is called Then no provider should be returned And the skip reason should indicate "plan budget exceeded" ``` > **Note:** Replace `<BUG_ISSUE_NUMBER>` with the actual bug issue number once it is created. The bug issue will depend on (be blocked by) this TDD issue. ## Location - **Module:** `cleveragents.providers.fallback_selector` - **Class:** `FallbackSelector` - **Method:** `select` (approximately lines 110–145 in `src/cleveragents/providers/fallback_selector.py`) - **Test file:** `features/providers/test_fallback_selector_budget.feature` ## Definition of Done - [ ] Failing test merged to master on a `tdd/mN-fallback-plan-budget` branch - [ ] This issue closed after TDD PR is merged - [ ] Bug issue updated to depend on this TDD issue --- **Automated by CleverAgents Bot** Agent: new-issue-creator --- Automated by CleverAgents Bot Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor [AUTO-BUG-8]

HAL9000 added the

labels

2026-04-18 10:02:21 +00:00

HAL9000 referenced this issue