Files
cleveragents-core/robot/cost_controls.robot
freemo 5a3678cd6c
CI / typecheck (pull_request) Successful in 59s
CI / security (pull_request) Successful in 54s
CI / quality (pull_request) Successful in 40s
CI / lint (pull_request) Successful in 3m21s
CI / build (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 21s
CI / unit_tests (pull_request) Successful in 6m50s
CI / e2e_tests (pull_request) Successful in 19m16s
CI / docker (pull_request) Successful in 1m21s
CI / integration_tests (pull_request) Successful in 21m37s
CI / coverage (pull_request) Successful in 14m51s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m40s
fix(concurrency): protect CostTracker._daily_costs with a threading.Lock
The record_usage method performed an unprotected read-modify-write on
_daily_costs, a classic TOCTOU race condition. In multi-threaded execution
(ThreadPoolExecutor for parallel plan steps), two threads could interleave
their .get() read and = write, causing one thread's cost increment to be
silently overwritten by the other.

Changes:
- Add _daily_costs_lock: threading.Lock to CostTracker.__init__
- Wrap the read-modify-write in record_usage with self._daily_costs_lock
- Protect reads in check_daily_budget and get_daily_spend with the same lock
- Add Behave @concurrency scenario: 20 threads concurrently call record_usage;
  final daily spend must equal the sum of all individual costs
- Add Robot Framework integration smoke test via helper_cost_controls.py
  cost-tracker-concurrent command

All accesses to _daily_costs are now lock-protected. nox -e typecheck passes
with zero errors. Existing cost_controls tests continue to pass.

ISSUES CLOSED: #1919
2026-04-05 04:09:27 +00:00

81 lines
3.4 KiB
Plaintext

*** Settings ***
Resource ${CURDIR}/common.resource
Library OperatingSystem
Library Process
*** Variables ***
${PYTHON} python
${SRC_DIR} ${CURDIR}/..
*** Test Cases ***
Cost Entry Creation And Estimation
[Documentation] Verify CostEntry creation and cost estimation
[Tags] cost_controls provider model
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py cost-entry
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} cost-entry-ok
Provider Cost Table Lookup
[Documentation] Verify ProviderCostTable model lookup
[Tags] cost_controls provider model
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py cost-table
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} cost-table-ok
Cost Tracker Budget Enforcement
[Documentation] Verify CostTracker budget enforcement
[Tags] cost_controls provider budget
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py cost-tracker
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} cost-tracker-budget-ok
Fallback Selector No Providers
[Documentation] Verify FallbackSelector with no configured providers
[Tags] cost_controls provider fallback
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py fallback-selector
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} fallback-selector-ok
Settings Cost Control Defaults
[Documentation] Verify Settings cost control config defaults
[Tags] cost_controls config settings
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py settings-defaults
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} settings-defaults-ok
Cost Metadata Display Dict
[Documentation] Verify CostMetadata display dictionary
[Tags] cost_controls provider model
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py cost-metadata
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} cost-metadata-display-ok
Cost Tracker Concurrent Record Usage Is Thread Safe
[Documentation] Verify CostTracker.record_usage is thread-safe: concurrent calls must not lose cost increments
[Tags] cost_controls provider concurrency
${result}= Run Process ${PYTHON} robot/helper_cost_controls.py cost-tracker-concurrent
... cwd=${SRC_DIR}
Log Process Failure ${result}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} cost-tracker-concurrent-ok
*** Keywords ***
Log Process Failure
[Arguments] ${result}
Run Keyword If ${result.rc} == 0 Return From Keyword
Log To Console Process failed with rc=${result.rc}
Log To Console STDOUT:${\n}${result.stdout}
Log To Console STDERR:${\n}${result.stderr}