50680612d5
Implemented optional estimation_actor role for cost, risk, and duration estimation during plan lifecycle. Estimates are persisted to plan metadata and surfaced in CLI output. Key implementation details: - EstimationOutput (Pydantic models): CostEstimate with currency, token estimates, and confidence ranges; RiskScore with 0-100 scale and factors; DurationEstimate with min/expected/max seconds. All include confidence levels and validation. EstimationSkipped records when estimation is opted out. - EstimationService: stateless async service that invokes estimation actor (stub implementation for M6). Handles actor output parsing, error recovery, and fallback to EstimationSkipped on failure. - Integration: Plan model gains estimation_output and estimation_skipped fields. LifecyclePlanModel adds JSON columns for persistence. PlanLifecycleService invokes estimation during use_action unless skip_estimation is true. - CLI: --no-estimate flag added to 'agents plan use'. plan status displays cost (USD with token estimates), risk (score/100 with confidence), and duration (seconds with ranges) in rich format. - Database: Alembic migration m6_003_estimation_metadata adds estimation_output_json and estimation_skipped_json columns to v3_plans. - Tests: 44 BDD scenarios (features/estimation.feature) covering validation, lifecycle integration, persistence, CLI display, and edge cases. 18 Robot Framework smoke tests. 19 ASV benchmark suites for schema, serialization, validation, and plan integration performance. - Documentation: docs/reference/estimation.md with schema, configuration, and examples. plan_cli.md updated with --no-estimate flag usage. ISSUES CLOSED: #209
4.7 KiB
4.7 KiB
Decision Domain Model
The decision subsystem records every choice point in a plan's lifecycle as a Decision node in a persistent tree. Decisions are created during the Strategize and Execute phases and form the basis for targeted correction and replay.
Decision Types
| Type | Phase | Description |
|---|---|---|
prompt_definition |
Strategize | Root decision — the plan prompt |
invariant_enforced |
Strategize | An invariant constraint was applied |
strategy_choice |
Strategize | High-level approach chosen |
implementation_choice |
Execute | How to implement a specific task |
resource_selection |
Execute | Which resources to read / modify |
subplan_spawn |
Strategize | Decision to create a child plan |
subplan_parallel_spawn |
Strategize | Spawn a group of child plans in parallel |
tool_invocation |
Execute | Which skill / tool to use |
error_recovery |
Execute | How to handle a failure |
validation_response |
Execute | Response to a validation failure |
user_intervention |
Any | User-provided guidance / correction |
Decision Fields
Identity
- decision_id — ULID, auto-generated
- plan_id — ULID of the parent plan (required)
- parent_decision_id — ULID of the parent decision, or
Nonefor the root - sequence_number — monotonic order within the plan (0-indexed, never reused)
Classification
- decision_type — one of the 12
DecisionTypeenum values
Content
- question — what question was being answered (required, non-empty)
- chosen_option — the option that was selected (required, non-empty)
- alternatives_considered — list of other options evaluated
- confidence_score — float 0.0–1.0, or
Noneif not applicable
Context Snapshot
Every decision captures a ContextSnapshot for replay:
- hot_context_hash — cryptographic hash of the context window
- hot_context_ref — storage pointer to the full serialised context
- relevant_resources — list of
ResourceRefentries (resource_id + optional path) - actor_state_ref — LangGraph actor checkpoint reference
Rationale
- rationale — human-readable explanation
- actor_reasoning — raw LLM reasoning trace, if available
Downstream Impact
- downstream_decision_ids — ULIDs of decisions that depend on this one
- downstream_plan_ids — ULIDs of child plans spawned from this decision
- artifacts_produced — list of
ArtifactRef(artifact_path + artifact_type)
Timestamps
- created_at — UTC datetime, auto-set on creation
Correction Metadata
- is_correction — boolean, True if this decision corrects another
- corrects_decision_id — ULID of the original decision
- correction_reason — why the correction was made
- superseded_by — ULID of the decision that replaced this one
Tree Structure
Decisions form a tree via parent_decision_id:
prompt_definition (root, parent=None)
├── invariant_enforced
├── strategy_choice
│ ├── implementation_choice
│ │ ├── resource_selection
│ │ └── tool_invocation
│ └── subplan_spawn
└── strategy_choice
The prompt_definition type is always the root and must have
parent_decision_id = None. This is enforced by a model validator.
Correction Lifecycle
Corrections never mutate existing decisions. Instead:
- A new
Decisionis created withis_correction=Trueandcorrects_decision_idpointing to the original. - The original decision has its
superseded_byfield set to the new decision's ID. - All downstream decisions of the original are also superseded.
The current tree consists of all decisions where
superseded_by IS NULL.
Validation Rules
plan_idmust be a valid ULIDparent_decision_id,corrects_decision_id,superseded_bymust be valid ULIDs or Noneconfidence_scoremust be in [0.0, 1.0] or Noneprompt_definitiondecisions must haveparent_decision_id = None- If
is_correctionis True,corrects_decision_idmust be set - If
corrects_decision_idis set,is_correctionmust be True
Source
- Domain model:
src/cleveragents/domain/models/core/decision.py - Specification:
docs/specification.mdL18390–L18521 - ADR-007: Decision tree and correction
- ADR-033: Decision recording protocol
- ADR-034: Decision tree versioning and history