forked from cleveragents/cleveragents-core
aditya
3f6b255747
Add --no-estimate opt-out flag for plan use command and persist EstimationSkipped reason when estimation is skipped or fails. Add EstimationReport Pydantic domain model (identical to #649) with spec-aligned multi-dimensional output: cost range (min/max USD), expected steps, expected child plans, rollback risk (0.0-1.0), estimated duration in minutes, confidence (0.0-1.0), rationale, and optional historical basis. Add EstimationSkipped domain model with reason, timestamp, and optional actor_name fields for tracking when estimation is skipped via --no-estimate or when the estimation actor fails. Add estimation_produced to DecisionType enum and STRATEGIZE_TYPES. Add Alembic migration m6_006 adding estimation_report_json and estimation_skipped_json columns to v3_plans and updating the ck_decisions_type CHECK constraint. Wire --no-estimate through plan use CLI: when set, creates EstimationSkipped with reason and clears the estimation actor. Update _run_estimation() to persist EstimationSkipped on actor failure instead of only logging. Add docs/reference/estimation.md documenting the estimation feature, EstimationReport schema, EstimationSkipped schema, and CLI examples. Add benchmarks/estimation_actor_bench.py with ASV benchmarks for EstimationReport and EstimationSkipped operations. Add 13 Behave scenarios (estimation_skip.feature) and 6 Robot integration tests (estimation_skip.robot) covering EstimationSkipped model, EstimationReport model, plan field mutual exclusion, and --no-estimate CLI behavior. ISSUES CLOSED: #209
52 lines
2.4 KiB
Plaintext
52 lines
2.4 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for EstimationSkipped and --no-estimate behavior
|
|
Library Process
|
|
Suite Setup Log Estimation skip robot tests starting
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_estimation_skip.py
|
|
${PYTHONPATH} ${CURDIR}/../src
|
|
|
|
*** Test Cases ***
|
|
EstimationSkipped Model Creation
|
|
[Documentation] Verify EstimationSkipped model creation and validation
|
|
${result}= Run Process python ${HELPER} model-create
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} estimation-skipped-model-create-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
EstimationSkipped Frozen Check
|
|
[Documentation] Verify EstimationSkipped model is frozen (immutable)
|
|
${result}= Run Process python ${HELPER} frozen-check
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} estimation-skipped-frozen-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
EstimationSkipped Serialization Round-trip
|
|
[Documentation] Verify EstimationSkipped survives serialization round-trip
|
|
${result}= Run Process python ${HELPER} round-trip
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} estimation-skipped-round-trip-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
EstimationReport Model Creation
|
|
[Documentation] Verify EstimationReport model creation
|
|
${result}= Run Process python ${HELPER} report-create
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} estimation-report-create-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
EstimationReport Validation
|
|
[Documentation] Verify EstimationReport validation rules
|
|
${result}= Run Process python ${HELPER} report-validation
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} estimation-report-validation-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
No Estimate CLI Integration
|
|
[Documentation] Verify --no-estimate sets estimation_skipped on plan
|
|
${result}= Run Process python ${HELPER} no-estimate-integration
|
|
... env:PYTHONPATH=${PYTHONPATH} timeout=30s
|
|
Should Contain ${result.stdout} no-estimate-integration-ok
|
|
Should Be Equal As Integers ${result.rc} 0
|