Files

T

aditya 50680612d5 feat(estimation): add cost and risk estimation actor

Implemented optional estimation_actor role for cost, risk, and duration
estimation during plan lifecycle. Estimates are persisted to plan metadata
and surfaced in CLI output. Key implementation details:

- EstimationOutput (Pydantic models): CostEstimate with currency, token
  estimates, and confidence ranges; RiskScore with 0-100 scale and factors;
  DurationEstimate with min/expected/max seconds. All include confidence
  levels and validation. EstimationSkipped records when estimation is opted out.

- EstimationService: stateless async service that invokes estimation actor
  (stub implementation for M6). Handles actor output parsing, error recovery,
  and fallback to EstimationSkipped on failure.

- Integration: Plan model gains estimation_output and estimation_skipped fields.
  LifecyclePlanModel adds JSON columns for persistence. PlanLifecycleService
  invokes estimation during use_action unless skip_estimation is true.

- CLI: --no-estimate flag added to 'agents plan use'. plan status displays
  cost (USD with token estimates), risk (score/100 with confidence), and
  duration (seconds with ranges) in rich format.

- Database: Alembic migration m6_003_estimation_metadata adds
  estimation_output_json and estimation_skipped_json columns to v3_plans.

- Tests: 44 BDD scenarios (features/estimation.feature) covering validation,
  lifecycle integration, persistence, CLI display, and edge cases. 18 Robot
  Framework smoke tests. 19 ASV benchmark suites for schema, serialization,
  validation, and plan integration performance.

- Documentation: docs/reference/estimation.md with schema, configuration,
  and examples. plan_cli.md updated with --no-estimate flag usage.

ISSUES CLOSED: #209

2026-03-19 15:07:05 +00:00

2.2 KiB

Raw Blame History

Estimation Reference

CleverAgents supports optional pre-execution estimation through the estimation_actor role. The actor produces a structured EstimationReport that is persisted on plans and shown in plan status output.

EstimationReport Schema

{
  "cost_range_usd_min": 0.5,
  "cost_range_usd_max": 2.0,
  "token_estimate_input": 1200,
  "token_estimate_output": 600,
  "expected_steps": 4,
  "expected_child_plans": 1,
  "rollback_risk": 0.25,
  "confidence": 0.75,
  "estimated_duration_minutes": 18.0,
  "rationale": "Derived from plan size and constraints",
  "historical_basis": [],
  "actor_used": "local/estimator",
  "generated_at": "2026-03-12T10:00:00Z"
}

Field Summary

cost_range_usd_min / cost_range_usd_max: Estimated USD cost band.
token_estimate_input / token_estimate_output: LLM token expectations.
expected_steps, expected_child_plans: Work-size projection.
rollback_risk: Rollback likelihood in [0.0, 1.0].
confidence: Model confidence in [0.0, 1.0].
estimated_duration_minutes: Duration estimate in minutes.
rationale: Human-readable explanation of estimate.
historical_basis: Optional prior plan IDs used as basis.
actor_used: Namespaced actor identifier (namespace/name).
generated_at: Timestamp of estimation generation.

Validation Rules

cost_range_usd_max >= cost_range_usd_min
rollback_risk and confidence are bounded to [0.0, 1.0]
all count/duration/token values are non-negative
actor_used (when present) must be namespaced

CLI Integration

Use --no-estimate to explicitly skip estimation:

agents plan use local/refactor my-project --no-estimate

When estimation is skipped, estimation_skipped is recorded with the reason.

Persistence

Estimation data is stored on v3_plans:

estimation_report: JSON-serialized EstimationReport
estimation_skipped_json: JSON-serialized EstimationSkipped

Runtime Behavior

Estimation runs during plan use:

create plan from action
invoke estimation actor if configured and not skipped
store either estimation_report or estimation_skipped
continue lifecycle (estimation is informational only)

2.2 KiB Raw Blame History