forked from cleveragents/cleveragents-core
aditya
3f6b255747
Add --no-estimate opt-out flag for plan use command and persist EstimationSkipped reason when estimation is skipped or fails. Add EstimationReport Pydantic domain model (identical to #649) with spec-aligned multi-dimensional output: cost range (min/max USD), expected steps, expected child plans, rollback risk (0.0-1.0), estimated duration in minutes, confidence (0.0-1.0), rationale, and optional historical basis. Add EstimationSkipped domain model with reason, timestamp, and optional actor_name fields for tracking when estimation is skipped via --no-estimate or when the estimation actor fails. Add estimation_produced to DecisionType enum and STRATEGIZE_TYPES. Add Alembic migration m6_006 adding estimation_report_json and estimation_skipped_json columns to v3_plans and updating the ck_decisions_type CHECK constraint. Wire --no-estimate through plan use CLI: when set, creates EstimationSkipped with reason and clears the estimation actor. Update _run_estimation() to persist EstimationSkipped on actor failure instead of only logging. Add docs/reference/estimation.md documenting the estimation feature, EstimationReport schema, EstimationSkipped schema, and CLI examples. Add benchmarks/estimation_actor_bench.py with ASV benchmarks for EstimationReport and EstimationSkipped operations. Add 13 Behave scenarios (estimation_skip.feature) and 6 Robot integration tests (estimation_skip.robot) covering EstimationSkipped model, EstimationReport model, plan field mutual exclusion, and --no-estimate CLI behavior. ISSUES CLOSED: #209
87 lines
3.3 KiB
Markdown
87 lines
3.3 KiB
Markdown
# Estimation Feature Reference
|
|
|
|
The estimation subsystem provides cost and risk estimation for plans
|
|
before the Execute phase begins. Estimation is **informational only** --
|
|
it does not gate execution.
|
|
|
|
## Overview
|
|
|
|
When a plan transitions from Strategize to Execute, the estimation actor
|
|
(if configured) produces an `EstimationReport` containing cost ranges,
|
|
expected work metrics, risk assessment, and rationale.
|
|
|
|
Users can opt out of estimation with the `--no-estimate` flag on
|
|
`plan use`, which records an `EstimationSkipped` record instead.
|
|
|
|
## EstimationReport Schema
|
|
|
|
| Field | Type | Constraints |
|
|
|----------------------------|-------------------|----------------------|
|
|
| `cost_range_usd_min` | `float` | >= 0.0 |
|
|
| `cost_range_usd_max` | `float` | >= 0.0, >= min |
|
|
| `expected_steps` | `int` | >= 0 |
|
|
| `expected_child_plans` | `int` | >= 0 |
|
|
| `rollback_risk` | `float` | 0.0 - 1.0 |
|
|
| `estimated_duration_minutes`| `float` | >= 0.0 |
|
|
| `confidence` | `float` | 0.0 - 1.0 |
|
|
| `rationale` | `str` | 1 - 10,000 chars |
|
|
| `historical_basis` | `tuple[str, ...]` | max 100 items |
|
|
|
|
The model is frozen (immutable) and rejects `inf`/`nan` values.
|
|
|
|
## EstimationSkipped Schema
|
|
|
|
| Field | Type | Description |
|
|
|--------------|------------------|-------------------------------------|
|
|
| `reason` | `str` | Why estimation was skipped |
|
|
| `timestamp` | `datetime` | When estimation was skipped |
|
|
| `actor_name` | `str` or `None` | Estimation actor name if applicable |
|
|
|
|
## Mutual Exclusion
|
|
|
|
A plan should have either `estimation_report` or `estimation_skipped`
|
|
set, but not both. This is enforced at the application layer:
|
|
|
|
- When `--no-estimate` is passed, `estimation_skipped` is set and the
|
|
estimation actor is cleared.
|
|
- When estimation succeeds, `estimation_report` is set.
|
|
- When estimation fails, `estimation_skipped` is set with the failure
|
|
reason.
|
|
|
|
## CLI Usage
|
|
|
|
### Skip estimation
|
|
|
|
```bash
|
|
agents plan use local/code-review my-project --no-estimate
|
|
```
|
|
|
|
When `--no-estimate` is passed:
|
|
- `estimation_skipped` is set with reason "User opted out via --no-estimate"
|
|
- The estimation actor is cleared from the plan
|
|
- No estimation is attempted during the Strategize-to-Execute transition
|
|
|
|
### View estimation data in plan status
|
|
|
|
```bash
|
|
agents plan status <plan_id> --format json | jq '.estimation'
|
|
```
|
|
|
|
The `plan status` command displays estimation data when available:
|
|
- `estimation` section shows the `EstimationReport` fields
|
|
- `estimation_skipped` section shows why estimation was skipped
|
|
|
|
## Database Storage
|
|
|
|
Both `EstimationReport` and `EstimationSkipped` are stored as
|
|
JSON-serialised TEXT columns on the `v3_plans` table:
|
|
|
|
- `estimation_report_json` -- serialised `EstimationReport`
|
|
- `estimation_skipped_json` -- serialised `EstimationSkipped`
|
|
|
|
## Legacy EstimationResult
|
|
|
|
The original `EstimationResult` model (all optional fields) is retained
|
|
for backward compatibility with the `EstimationStubActor`. New code
|
|
should prefer `EstimationReport` for structured estimation data.
|