aditya
50680612d5
Implemented optional estimation_actor role for cost, risk, and duration estimation during plan lifecycle. Estimates are persisted to plan metadata and surfaced in CLI output. Key implementation details: - EstimationOutput (Pydantic models): CostEstimate with currency, token estimates, and confidence ranges; RiskScore with 0-100 scale and factors; DurationEstimate with min/expected/max seconds. All include confidence levels and validation. EstimationSkipped records when estimation is opted out. - EstimationService: stateless async service that invokes estimation actor (stub implementation for M6). Handles actor output parsing, error recovery, and fallback to EstimationSkipped on failure. - Integration: Plan model gains estimation_output and estimation_skipped fields. LifecyclePlanModel adds JSON columns for persistence. PlanLifecycleService invokes estimation during use_action unless skip_estimation is true. - CLI: --no-estimate flag added to 'agents plan use'. plan status displays cost (USD with token estimates), risk (score/100 with confidence), and duration (seconds with ranges) in rich format. - Database: Alembic migration m6_003_estimation_metadata adds estimation_output_json and estimation_skipped_json columns to v3_plans. - Tests: 44 BDD scenarios (features/estimation.feature) covering validation, lifecycle integration, persistence, CLI display, and edge cases. 18 Robot Framework smoke tests. 19 ASV benchmark suites for schema, serialization, validation, and plan integration performance. - Documentation: docs/reference/estimation.md with schema, configuration, and examples. plan_cli.md updated with --no-estimate flag usage. ISSUES CLOSED: #209
72 lines
2.2 KiB
Markdown
72 lines
2.2 KiB
Markdown
# Estimation Reference
|
|
|
|
CleverAgents supports optional pre-execution estimation through the
|
|
`estimation_actor` role. The actor produces a structured `EstimationReport`
|
|
that is persisted on plans and shown in plan status output.
|
|
|
|
## EstimationReport Schema
|
|
|
|
```json
|
|
{
|
|
"cost_range_usd_min": 0.5,
|
|
"cost_range_usd_max": 2.0,
|
|
"token_estimate_input": 1200,
|
|
"token_estimate_output": 600,
|
|
"expected_steps": 4,
|
|
"expected_child_plans": 1,
|
|
"rollback_risk": 0.25,
|
|
"confidence": 0.75,
|
|
"estimated_duration_minutes": 18.0,
|
|
"rationale": "Derived from plan size and constraints",
|
|
"historical_basis": [],
|
|
"actor_used": "local/estimator",
|
|
"generated_at": "2026-03-12T10:00:00Z"
|
|
}
|
|
```
|
|
|
|
### Field Summary
|
|
|
|
- `cost_range_usd_min` / `cost_range_usd_max`: Estimated USD cost band.
|
|
- `token_estimate_input` / `token_estimate_output`: LLM token expectations.
|
|
- `expected_steps`, `expected_child_plans`: Work-size projection.
|
|
- `rollback_risk`: Rollback likelihood in `[0.0, 1.0]`.
|
|
- `confidence`: Model confidence in `[0.0, 1.0]`.
|
|
- `estimated_duration_minutes`: Duration estimate in minutes.
|
|
- `rationale`: Human-readable explanation of estimate.
|
|
- `historical_basis`: Optional prior plan IDs used as basis.
|
|
- `actor_used`: Namespaced actor identifier (`namespace/name`).
|
|
- `generated_at`: Timestamp of estimation generation.
|
|
|
|
### Validation Rules
|
|
|
|
- `cost_range_usd_max >= cost_range_usd_min`
|
|
- `rollback_risk` and `confidence` are bounded to `[0.0, 1.0]`
|
|
- all count/duration/token values are non-negative
|
|
- `actor_used` (when present) must be namespaced
|
|
|
|
## CLI Integration
|
|
|
|
Use `--no-estimate` to explicitly skip estimation:
|
|
|
|
```bash
|
|
agents plan use local/refactor my-project --no-estimate
|
|
```
|
|
|
|
When estimation is skipped, `estimation_skipped` is recorded with the reason.
|
|
|
|
## Persistence
|
|
|
|
Estimation data is stored on `v3_plans`:
|
|
|
|
- `estimation_report`: JSON-serialized `EstimationReport`
|
|
- `estimation_skipped_json`: JSON-serialized `EstimationSkipped`
|
|
|
|
## Runtime Behavior
|
|
|
|
Estimation runs during `plan use`:
|
|
|
|
1. create plan from action
|
|
2. invoke estimation actor if configured and not skipped
|
|
3. store either `estimation_report` or `estimation_skipped`
|
|
4. continue lifecycle (estimation is informational only)
|