Implemented 5 new ASV benchmark files under benchmarks/:
- providers_cost_table_bench.py — ProviderCostTable construction (default +
custom entries), iteration throughput across all providers/models, fallback
path for unknown providers
- providers_cost_tracker_bench.py — CostTracker construction with various
budget configurations, accumulation throughput (10/50 calls, mixed
providers), daily spend tracking, get_cost_entry delegation
- providers_fallback_selector_bench.py — FallbackSelector construction with
custom order and cost tracker, selection when no providers configured
(exhausts full list), selection with configured provider at various positions
- providers_registry_bench.py — ProviderRegistry.get_all_providers,
get_provider_info (by enum and string), is_provider_configured,
multi-provider initialization
- providers_llm_adapters_bench.py — LangChainChatProvider,
AnthropicChatProvider, GoogleChatProvider, OpenAIChatProvider,
OpenRouterChatProvider instantiation with various configurations
Key design decisions:
- Carefully audited existing cost_controls_bench.py and
provider_selection_bench.py to avoid duplicating any already-covered
benchmarks
- Used MagicMock for Settings objects to avoid requiring real API keys in
benchmarks
- LLM adapter benchmarks use mock factories to measure pure instantiation
cost without network calls
- All benchmark classes use setup() fixtures to isolate measurement from
fixture construction
- 68 benchmark methods total across 5 files, all verified to execute without
errors
ISSUES CLOSED: #2800
freemo