freemo
876217d0ca
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 24s
CI / typecheck (pull_request) Successful in 37s
CI / security (pull_request) Successful in 44s
CI / unit_tests (pull_request) Successful in 2m40s
CI / integration_tests (pull_request) Successful in 3m19s
CI / docker (pull_request) Successful in 42s
CI / coverage (pull_request) Successful in 5m10s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 16s
CI / quality (push) Successful in 21s
CI / security (push) Successful in 32s
CI / typecheck (push) Successful in 37s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Failing after 2m36s
CI / docker (push) Has been skipped
CI / integration_tests (push) Successful in 3m16s
CI / coverage (push) Successful in 5m3s
CI / benchmark-publish (push) Successful in 17m54s
CI / benchmark-regression (pull_request) Successful in 31m55s
Implements Forgejo issue #584: three-tier budget hierarchy (per-plan -> per-session -> per-org) with tightest limit winning. Domain models: - BudgetLevel enum (PLAN, SESSION, ORG) - BudgetCheckResult (frozen Pydantic model with exceeded_level, warning) - SessionCostBudget (tracks per-session accumulated cost) - OrgCostAccumulator (tracks per-org accumulated cost) - ThreadSafeOrgCostAccumulator (thread-safe wrapper with snapshot) Application services: - CostBudgetService: manages budget state, enforces hierarchy, emits BUDGET_WARNING (once per session) and BUDGET_EXCEEDED events - AutonomyGuardrailService: extended with associate_plan_with_session, check_budget_hierarchy, record_plan_cost_to_session methods Configuration: - Settings: session_max_cost_usd, org_max_cost_usd, budget_warning_threshold Integration: - Session model: cost_budget field, as_cli_dict includes budget data - DI container: CostBudgetService registered as Singleton - CLI session show: cost budget panel display Tests: - 54 Behave scenarios (features/cost_budgets.feature) - 11 Robot Framework integration tests (robot/cost_budgets.robot) - ASV benchmarks (benchmarks/bench_budget_check.py) All nox stages pass: lint, typecheck, unit_tests, coverage_report (98%). CLOSES #584
82 lines
4.0 KiB
Plaintext
82 lines
4.0 KiB
Plaintext
*** Settings ***
|
|
Documentation Smoke tests for per-session and per-org cost budgets
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER_SCRIPT} robot/helper_cost_budgets.py
|
|
|
|
*** Test Cases ***
|
|
Session Cost Budget Records Cost
|
|
[Documentation] Session budget tracks cumulative cost
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} session-record cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} session-record-ok
|
|
|
|
Session Cost Budget Detects Exceeded
|
|
[Documentation] Session budget detects when cap is exceeded
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} session-exceeded cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} session-exceeded-ok
|
|
|
|
Org Cost Accumulator Records Cost
|
|
[Documentation] Org accumulator tracks cumulative cost
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} org-record cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} org-record-ok
|
|
|
|
Org Cost Accumulator Detects Exceeded
|
|
[Documentation] Org accumulator detects when cap is exceeded
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} org-exceeded cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} org-exceeded-ok
|
|
|
|
Thread Safe Org Accumulator Works
|
|
[Documentation] Thread-safe wrapper delegates correctly
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} thread-safe cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} thread-safe-ok
|
|
|
|
Budget Hierarchy Allows Within All Limits
|
|
[Documentation] Budget hierarchy allows when under all caps
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} hierarchy-allow cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} hierarchy-allow-ok
|
|
|
|
Budget Hierarchy Blocks At Session Level
|
|
[Documentation] Budget hierarchy blocks at session tier
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} hierarchy-session-block cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} hierarchy-session-block-ok
|
|
|
|
Budget Hierarchy Blocks At Org Level
|
|
[Documentation] Budget hierarchy blocks at org tier
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} hierarchy-org-block cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} hierarchy-org-block-ok
|
|
|
|
Budget Hierarchy Blocks At Plan Level
|
|
[Documentation] Budget hierarchy blocks at plan tier
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} hierarchy-plan-block cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} hierarchy-plan-block-ok
|
|
|
|
Guardrail Service Budget Hierarchy
|
|
[Documentation] Guardrail service integrates budget hierarchy
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} guardrail-hierarchy cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} guardrail-hierarchy-ok
|
|
|
|
Settings Default Budget Fields
|
|
[Documentation] Settings have correct budget defaults
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} settings-defaults cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} settings-defaults-ok
|
|
|
|
Session Model Cost Budget Field
|
|
[Documentation] Session model includes cost_budget field
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} session-model cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} session-model-ok
|