Files
cleveragents-core/docs/adr/ADR-033-decision-recording-protocol.md
T
HAL9000 f808abff86 chore(ci): fix pre-commit hook failures
Fix JSON syntax errors in .devcontainer/devcontainer.json (removed
invalid JS-style // comments) and .devcontainer/opencode.json (removed
90+ trailing commas). Apply auto-fixes for end-of-file and trailing
whitespace issues across 100+ files. Fix SIM105 ruff violations in
benchmarks/core_circuit_breaker_bench.py (use contextlib.suppress).

Note: The security fix from issue #7478 (validate_path startswith bypass)
was already delivered to master in commit e18ac5f2. This PR as currently
structured is non-atomic (35 commits across 10+ issues) and needs
significant restructure before merge. This commit only addresses the
CI/pre-commit failures.

ISSUES CLOSED: #7478
2026-06-14 09:51:14 -04:00

18 KiB

adr_number, title, status_history, tier, authors, superseded_by, related_adrs, acceptance
adr_number title status_history tier authors superseded_by related_adrs acceptance
33 Decision Recording Protocol
2026-02-20
Proposed
Jeffrey Phillips Freeman
2026-02-20
Accepted
Jeffrey Phillips Freeman
2
Jeffrey Phillips Freeman
null
number title relationship
6 Plan Lifecycle Defines the phases in which decisions are recorded
number title relationship
7 Decision Tree and Correction Defines the decision record structure and types that this protocol populates
number title relationship
10 Actor and Agent Architecture Actors are the entities that call `record_decision`
number title relationship
11 Tool System `record_decision` is a built-in tool following the tool system conventions
number title relationship
16 Invariant System Invariant Reconciliation Actor uses `record_decision` to record enforced invariants
number title relationship
17 Automation Profiles Confidence thresholds gate `record_decision` calls
number title relationship
34 Decision Tree Versioning and History Recorded decisions form the versioned tree structure
number title relationship
35 Decision Tree Rollback and Replay Context snapshots captured during recording enable rollback and replay
votes_for votes_against abstentions
voter comment
Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> Tool-based decision recording with auto-captured context snapshots provides explicit, auditable, replayable decision creation

Context

ADR-007 defines eleven decision types, the decision record data model, and the correction mechanism. ADR-006 defines the plan lifecycle phases in which decisions are created. However, neither ADR describes the protocol by which actors mechanically create decision records during plan execution. The specification states that "the strategy actor produces the initial decision tree" and that "the execution actor may create additional decisions," but does not specify how an actor identifies a choice point, emits a decision record, captures the context snapshot, or interacts with the automation profile's confidence threshold.

Without a defined recording protocol, implementers must invent their own mechanism for decision creation, leading to inconsistent recording, missing context snapshots, and unreliable correction replay.

Decision Drivers

  • Every decision made during plan execution must be explicitly recorded with type, rationale, and context snapshot to enable auditing and correction replay
  • The recording mechanism must integrate naturally with the existing tool-based architecture, providing clear interception points for automation profile gating and context capture
  • Context snapshots (LangGraph checkpoint, hot context hash, relevant resources) must be captured automatically at recording time without requiring actor cooperation
  • Decision recording must be uniform across all actor roles (strategy, execution, invariant reconciliation) using a single tool interface
  • Confidence thresholds from automation profiles must gate decision persistence, pausing for human review when confidence is below the threshold

Decision

Actors record decisions by calling a built-in record_decision tool. The system prompt for each actor role (strategy, execution, invariant reconciliation) instructs the actor to call this tool whenever it identifies a choice point. Decision types are partitioned into three creation categories: actor-recorded (strategy and execution actors call record_decision), reconciliation-actor-recorded (the Invariant Reconciliation Actor calls record_decision for enforced invariants), and system-created (the plan lifecycle engine creates prompt_definition and user_intervention decisions automatically). All categories produce identical decision records; only the originator differs.

Design

Decision Creation Classification

Decision Type Creator Phase Mechanism
prompt_definition System (plan lifecycle engine) Plan instantiation Created automatically when agents plan use instantiates a plan from an action. The action's description + user arguments become the root decision's chosen_option.
invariant_enforced Invariant Reconciliation Actor Strategize entry The actor evaluates applicable invariants from all four scopes (global, project, action, plan), resolves conflicts using precedence rules, and calls record_decision for each enforced invariant.
strategy_choice Strategy actor Strategize Actor identifies a strategic ambiguity and calls record_decision.
resource_selection Strategy or Execution actor Strategize or Execute Actor selects resources and calls record_decision.
subplan_spawn Strategy actor Strategize Actor decides to decompose work into a child plan and calls record_decision.
subplan_parallel_spawn Strategy actor Strategize Actor decides to group child plans for concurrent execution and calls record_decision.
implementation_choice Execution actor Execute Actor encounters an implementation ambiguity and calls record_decision.
tool_invocation Execution actor Execute Actor selects a tool for a task and calls record_decision.
error_recovery Execution actor Execute Actor encounters an error and decides how to recover, calling record_decision.
validation_response Execution actor Execute Actor responds to a validation failure by calling record_decision.
user_intervention System (plan lifecycle engine) Any Created automatically when the user provides guidance via agents plan correct or agents plan prompt.

The record_decision Tool

The record_decision tool is a built-in tool registered in the Tool Registry. It is included in the local/decision-tools skill, which is automatically added to every strategy actor, execution actor, and invariant reconciliation actor's skill set.

# Built-in tool definition (registered automatically)
tool:
  name: local/record-decision
  description: >-
    Record a decision made during plan reasoning. Call this tool whenever you
    identify a choice point — a question where multiple valid approaches exist
    and a specific option must be selected to proceed.
  source: builtin

  input_schema:
    type: object
    properties:
      decision_type:
        type: string
        enum:
          - invariant_enforced
          - strategy_choice
          - resource_selection
          - subplan_spawn
          - subplan_parallel_spawn
          - implementation_choice
          - tool_invocation
          - error_recovery
          - validation_response
      question:
        type: string
        description: What question or ambiguity is being resolved
      chosen_option:
        type: string
        description: The selected approach or answer
      alternatives_considered:
        type: array
        items: { type: string }
        description: Other options that were evaluated and rejected
      confidence_score:
        type: number
        minimum: 0.0
        maximum: 1.0
        description: How confident the actor is in this choice (0.0-1.0)
      rationale:
        type: string
        description: Why this option was chosen over the alternatives
      parent_decision_id:
        type: string
        description: >-
          ULID of the parent decision in the tree structure. If omitted,
          the system infers the parent from the current decision context.
    required:
      - decision_type
      - question
      - chosen_option
      - rationale

  output_schema:
    type: object
    properties:
      decision_id:
        type: string
        description: The ULID of the created decision record

  capability:
    read_only: false
    writes: true
    checkpointable: false
    side_effects: [persist_decision]

Context Snapshot Auto-Capture

When record_decision is called, the system automatically captures the context snapshot before persisting the decision. The actor does not need to provide these fields — they are extracted from the runtime environment:

  • hot_context_hash: Cryptographic hash of the actor's current context window contents.
  • hot_context_ref: Reference to the full stored context snapshot in the warm/cold tier.
  • relevant_resources: Resource references extracted from the actor's current context (files, symbols, UKO entities that were loaded for this reasoning step).
  • actor_state_ref: A LangGraph checkpoint of the actor's complete state at the moment of the tool call. This enables replay from this exact point during correction.

The actor_state_ref is the critical field for rollback: it captures the actor's full conversation history, intermediate reasoning, and graph state, allowing the system to restore the actor to the exact state it was in when the decision was made.

Strategize-Phase Recording Loop

During the Strategize phase, the strategy actor follows a structured reasoning loop:

# Pseudocode: Strategy actor's decision recording loop

def strategize(plan, context):
    # System has already created the prompt_definition root decision
    # Invariant Reconciliation Actor has already recorded invariant_enforced decisions

    # Actor receives: plan description, enforced invariants, project context

    while unresolved_ambiguities_remain(plan, context):
        # 1. Identify a choice point
        choice_point = analyze_context_for_ambiguity(context)

        # 2. Evaluate options
        options = generate_options(choice_point, context)
        best_option = evaluate_options(options, context, invariants)

        # 3. Record the decision via tool call
        decision_id = record_decision(
            decision_type = choice_point.type,  # strategy_choice, subplan_spawn, etc.
            question = choice_point.question,
            chosen_option = best_option.description,
            alternatives_considered = [o.description for o in options if o != best_option],
            confidence_score = best_option.confidence,
            rationale = best_option.rationale
        )

        # 4. Decision becomes part of the actor's context for subsequent reasoning
        context.add_decision(decision_id)

    # Result: a complete decision tree for this plan's strategy

The actor's system prompt explicitly instructs it to call record_decision for every strategic choice. The prompt includes guidance on what constitutes a decision (ambiguities not resolved by the plan description) versus what is simply executing instructions (no decision needed).

Execute-Phase Recording Pattern

During the Execute phase, the execution actor operates under constraints established by Strategize-phase decisions:

  1. The execution actor receives the Strategize-phase decision tree as read-only context.
  2. When the actor encounters a choice point not already resolved by the strategy (implementation detail, error handling, tool selection), it calls record_decision.
  3. Execute-phase decisions must be consistent with Strategize-phase constraints. If the actor determines it cannot proceed within those constraints, it signals for phase reversion (Execute → Strategize) rather than recording a contradictory decision.
  4. Critical: In the Execute phase, record_decision is treated as a write operation for checkpoint purposes. The system automatically creates a coordinated checkpoint group across all modified sandbox resources when this tool is called (see ADR-035 for the checkpoint group mechanism).

Automation Profile Interaction

When record_decision is called, the system checks the actor's confidence_score against the applicable automation profile threshold before persisting:

  1. Actor calls record_decision with confidence_score = 0.72.
  2. System checks the profile: if edit_code = 0.8 (during Strategize) or execute_command = 0.8 (during Execute), the confidence is below the threshold.
  3. System pauses execution and presents the decision to the user for approval or override.
  4. User either approves (decision is persisted as-is), selects an alternative (decision is persisted with the user's choice), or provides custom guidance (decision is persisted with the user's input).
  5. If the user overrides the actor's choice, the decision's chosen_option reflects the user's selection, and the original actor choice is recorded in alternatives_considered.

If confidence meets or exceeds the threshold, the decision is persisted automatically without user interaction.

Invariant Reconciliation Actor Recording

The Invariant Reconciliation Actor is a specialized actor (configurable at plan, action, project, or global level) that runs at Strategize entry. It uses record_decision with decision_type = invariant_enforced to record each invariant it determines should apply:

  1. The actor collects invariants from all four scopes (global, project, action, plan).
  2. It identifies conflicts between scopes.
  3. It resolves conflicts using the precedence chain (plan > action > project > global).
  4. For each enforced invariant, it calls record_decision:
    • decision_type: invariant_enforced
    • question: "Should this invariant apply to this plan?"
    • chosen_option: The invariant text
    • alternatives_considered: Any conflicting invariants from lower-precedence scopes that were overridden
    • rationale: Why this invariant applies (scope, precedence, conflict resolution reasoning)

This makes invariant enforcement an explicit, auditable, correctable part of the decision tree — the same as any other decision.

Constraints

  • Every actor-created or reconciliation-actor-created decision must go through the record_decision tool. Direct database insertion of decision records by actors is prohibited.
  • Only the plan lifecycle engine may create prompt_definition and user_intervention decisions. Actors must not call record_decision with these types.
  • The local/decision-tools skill containing record_decision is automatically included in every actor's skill set for plan-related invocations. Actors cannot opt out of decision recording.
  • Context snapshot capture is mandatory and automatic. A decision record without a valid context_snapshot is invalid and must not be persisted.
  • Execute-phase record_decision calls must trigger checkpoint creation (coordinated with ADR-035). Strategize-phase calls do not trigger checkpoints (Strategize is read-only).

Consequences

Positive

  • Explicit tool-based recording ensures every decision is typed, structured, and auditable.
  • Context snapshot auto-capture guarantees that every decision can be replayed from its exact context, enabling reliable correction.
  • The same record_decision tool is used by all actor roles, providing a uniform recording interface.
  • Automation profile integration at the recording point provides a natural gate for human-in-the-loop review.
  • Invariant enforcement is recorded through the same mechanism as all other decisions, making it visible, auditable, and correctable.

Negative

  • Tool-call overhead: each decision requires a round-trip tool call, adding latency to the reasoning loop.
  • System prompt engineering: actors must be reliably instructed to call record_decision at every choice point. Under-recording (missing decisions) or over-recording (trivial choices) reduces the value of the decision tree.
  • Context snapshot storage: capturing and storing LangGraph checkpoints at every decision point consumes significant storage, especially for actors with large context windows.

Risks

  • LLM actors may fail to call record_decision consistently, leading to gaps in the decision tree. Mitigation: post-phase validation checks that the decision tree covers expected decision types.
  • Over-recording could create excessively large decision trees for simple plans. Mitigation: actor system prompts should include guidance on decision granularity.
  • LangGraph checkpoint capture may introduce latency spikes for large actor states. Mitigation: async checkpoint capture with write-behind.

Alternatives Considered

Structured output extraction (parse decisions from LLM text output) — The actor outputs decisions as structured blocks in its response, and the system parses them. Less explicit than tool calls, harder to validate, and loses the natural checkpoint/pause point that a tool call provides. Rejected because tool-based recording aligns with the existing tool-based architecture and provides clear interception points for automation profile gating and context snapshot capture.

System-level interception (automatically detect decisions from reasoning) — The system analyzes the actor's reasoning output and automatically identifies and records decisions without explicit actor participation. Rejected because it requires unreliable heuristic analysis of LLM output, produces inconsistent decision boundaries, and cannot capture the actor's own confidence assessment or rationale.

Compliance

  • Recording completeness tests: BDD scenarios verify that Strategize produces the expected set of decision types (at minimum: prompt_definition, invariant_enforced if invariants exist, and at least one strategy_choice).
  • Tool schema tests: Unit tests verify that record_decision validates input against the JSON Schema and rejects invalid decision types.
  • Context snapshot tests: Integration tests verify that actor_state_ref is captured at every record_decision call and that the captured state can be restored to resume actor execution.
  • Automation profile gating tests: Tests verify that decisions with confidence below the profile threshold pause for user input, and decisions meeting the threshold proceed automatically.
  • Invariant recording tests: Tests verify that the Invariant Reconciliation Actor creates one invariant_enforced decision per enforced invariant via record_decision.
  • Execute checkpoint tests: Tests verify that record_decision calls during Execute trigger checkpoint group creation (coordinated with ADR-035 compliance tests).