Commit Graph

3 Commits

Author SHA1 Message Date
freemo b122ec7ed5 fix(test-infra): remove redundant ${PYTHON} variable definitions from robot files
CI / lint (pull_request) Successful in 23s
CI / quality (pull_request) Successful in 51s
CI / typecheck (pull_request) Successful in 1m0s
CI / security (pull_request) Successful in 55s
CI / build (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 32s
CI / push-validation (pull_request) Successful in 26s
CI / e2e_tests (pull_request) Successful in 3m38s
CI / integration_tests (pull_request) Successful in 6m42s
CI / unit_tests (pull_request) Successful in 8m19s
CI / docker (pull_request) Successful in 13s
CI / coverage (pull_request) Successful in 15m23s
CI / status-check (pull_request) Successful in 2s
Remove the local ${PYTHON}    python (and python3) variable definitions from
the *** Variables *** sections of all affected robot files. These local
definitions were overriding the pabot-injected venv Python path passed via
--variable PYTHON:/path/to/venv/python, causing tests to use the system
Python (which lacks required packages like structlog, sqlalchemy, etc.)
instead of the nox venv Python.

The correct ${PYTHON} value is already set by Setup Test Environment in
common.resource via sys.executable, and pabot passes it via --variable.
The local fallback definitions are redundant and harmful in parallel runs.

Audit found 56 robot files with the pattern (more than the 9 originally
identified in the issue). All occurrences have been removed.

ISSUES CLOSED: #1309
2026-04-14 14:50:55 +00:00
freemo 5a3678cd6c fix(concurrency): protect CostTracker._daily_costs with a threading.Lock
CI / typecheck (pull_request) Successful in 59s
CI / security (pull_request) Successful in 54s
CI / quality (pull_request) Successful in 40s
CI / lint (pull_request) Successful in 3m21s
CI / build (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 21s
CI / unit_tests (pull_request) Successful in 6m50s
CI / e2e_tests (pull_request) Successful in 19m16s
CI / docker (pull_request) Successful in 1m21s
CI / integration_tests (pull_request) Successful in 21m37s
CI / coverage (pull_request) Successful in 14m51s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m40s
The record_usage method performed an unprotected read-modify-write on
_daily_costs, a classic TOCTOU race condition. In multi-threaded execution
(ThreadPoolExecutor for parallel plan steps), two threads could interleave
their .get() read and = write, causing one thread's cost increment to be
silently overwritten by the other.

Changes:
- Add _daily_costs_lock: threading.Lock to CostTracker.__init__
- Wrap the read-modify-write in record_usage with self._daily_costs_lock
- Protect reads in check_daily_budget and get_daily_spend with the same lock
- Add Behave @concurrency scenario: 20 threads concurrently call record_usage;
  final daily spend must equal the sum of all individual costs
- Add Robot Framework integration smoke test via helper_cost_controls.py
  cost-tracker-concurrent command

All accesses to _daily_costs are now lock-protected. nox -e typecheck passes
with zero errors. Existing cost_controls tests continue to pass.

ISSUES CLOSED: #1919
2026-04-05 04:09:27 +00:00
freemo 134dce896a feat(provider): add cost controls and fallback
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 17s
CI / lint (pull_request) Successful in 18s
CI / quality (pull_request) Successful in 20s
CI / security (pull_request) Successful in 34s
CI / typecheck (pull_request) Successful in 37s
CI / integration_tests (pull_request) Successful in 3m46s
CI / unit_tests (pull_request) Successful in 8m32s
CI / docker (pull_request) Successful in 39s
CI / benchmark-regression (pull_request) Successful in 19m26s
CI / coverage (pull_request) Successful in 31m23s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 16s
CI / typecheck (push) Successful in 30s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 33s
CI / integration_tests (push) Successful in 3m44s
CI / unit_tests (push) Successful in 8m53s
CI / docker (push) Successful in 1m1s
CI / benchmark-publish (push) Successful in 10m35s
CI / coverage (push) Successful in 37m55s
Add token/cost tracking, budget enforcement (per-plan and per-day),
provider fallback selection with capability filtering, and cost
metadata for plan models.

New modules:
- providers/cost_table.py: CostEntry + ProviderCostTable with default
  pricing for OpenAI, Anthropic, Google, Groq, Together, Cohere, Mock
- providers/cost_tracker.py: BudgetStatus enum, BudgetCheckResult,
  CostTracker with warn-at-90%/block-at-100% enforcement
- providers/fallback_selector.py: FallbackSelector with capability
  filtering (tool_calls, streaming, vision, json_mode) and budget
- domain/models/core/cost_metadata.py: CostMetadata + BudgetExhaustionEvent

Modified:
- config/settings.py: budget_per_plan, budget_per_day, fallback_providers
- domain/models/core/plan.py: cost_metadata field + CLI dict surfacing
- providers/__init__.py: public exports for new classes

Testing:
- 71 Behave scenarios in features/cost_controls.feature (all pass)
- 6 Robot Framework integration tests (all pass)
- ASV benchmarks in benchmarks/cost_controls_bench.py
- nox lint, typecheck, format, docs, build, dead_code, security_scan pass

Closes #324
2026-02-25 12:08:01 -05:00