---
adr_number: 18
title: Semantic Error Prevention
status_history:
- - '2026-02-16'
  - Proposed
  - Jeffrey Phillips Freeman
- - '2026-02-16'
  - Accepted
  - Jeffrey Phillips Freeman
tier: 3
authors:
- Jeffrey Phillips Freeman
superseded_by: null
related_adrs:
- number: 6
  title: Plan Lifecycle
  relationship: Error prevention layers operate at different lifecycle phases
- number: 7
  title: Decision Tree and Correction
  relationship: Decision-time validation catches errors during tree construction
- number: 13
  title: Validation Abstraction
  relationship: Validation tools implement the execution-time semantic guards
- number: 16
  title: Invariant System
  relationship: Invariant enforcement is one of the four error prevention layers
- number: 17
  title: Automation Profiles
  relationship: Low-confidence decisions are flagged based on automation profile thresholds
acceptance:
  votes_for:
  - voter: Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com>
    comment: Multiple prevention layers operating at different lifecycle points catch errors progressively earlier
  votes_against: []
  abstentions: []
---
## Context

LLM-driven systems can produce errors that are syntactically valid but semantically wrong — choosing an incompatible library version, applying a pattern that contradicts project conventions, or making a decision that violates an upstream constraint. Traditional testing catches many errors after the fact, but the cost of fixing errors increases dramatically the later they are caught. The system needs multiple layers of error prevention that operate at different points in the lifecycle — from decision time through execution to predictive analysis.

## Decision Drivers

- LLM-driven systems produce errors that are syntactically valid but semantically wrong (incompatible versions, convention violations)
- The cost of fixing errors increases dramatically the later they are caught in the lifecycle
- Different error types are detectable at different lifecycle points (decision-time, execution-time, post-execution)
- Error prevention must respect automation profile thresholds to ensure appropriate human involvement
- Historical correction data should improve future error detection, not be discarded after each plan

## Decision

CleverAgents implements **four layers of semantic error prevention** that operate progressively through the plan lifecycle: decision-time validation during Strategize, execution-time semantic guards, invariant enforcement across all phases, and predictive error prevention using historical data and pattern matching.

## Design

### Layer 1: Decision-Time Validation During Strategize

During the Strategize phase, the system validates decisions as they are being made:

- **Schema validation**: Strategy decisions are validated against the expected schemas (e.g., resource selections must reference valid resource IDs, tool invocations must match tool input schemas).
- **Constraint checking**: Each decision is checked against enforced invariants (`invariant_enforced` decisions) before being accepted into the tree.
- **Consistency checking**: New decisions are checked for consistency with existing decisions in the tree (e.g., a resource selection must not contradict a prior strategy choice).
- **Confidence scoring**: The strategy actor provides confidence scores for decisions. Low-confidence decisions are flagged for human review based on the automation profile thresholds.

### Layer 2: Execution-Time Semantic Guards

During the Execute phase, the system applies guards before and after tool invocations:

- **Pre-execution guards**: Before a tool is invoked, the system verifies that the invocation is consistent with the strategy (the tool is within the execution plan, the resource is within scope, the operation type matches the strategy's intent).
- **Post-execution guards**: After a tool completes, the system verifies that the output is within expected bounds (file sizes within limits, change scope within the declared `write_scope`, no unexpected side effects).
- **Tool capability enforcement**: The system enforces tool capability metadata — `read_only` actions can only invoke `read_only` tools, `human_approval_required` tools trigger approval gates, write scope restrictions are enforced at runtime.

### Layer 3: Invariant Enforcement

Invariants (ADR-016) provide persistent, cross-cutting constraints:

- **Four scopes**: Global, project, action, and plan invariants are reconciled at Strategize entry.
- **Decision tree integration**: Enforced invariants are recorded as decision nodes that constrain downstream decisions.
- **Execution-time checking**: During execution, tool invocations are checked against applicable invariants. Violations trigger either automatic correction or escalation.
- **Apply-time checking**: During Apply, the final changeset is checked against invariants. Violations produce the `constrained` terminal state.

### Layer 4: Predictive Error Prevention

The system uses historical data to predict and prevent errors:

- **Pattern matching**: Historical correction data (from `correction_attempts`) is analyzed to identify recurring error patterns. When a similar decision is about to be made, the system can warn the actor or the user.
- **Confidence calibration**: Historical confidence scores are compared against actual outcomes to improve future confidence estimates.
- **Common failure modes**: The system learns from past validation failures, strategy revisions, and plan failures to flag high-risk decisions proactively.

### Escalation Flow

When any layer detects a potential error:

1. **Low severity**: Warning logged, execution continues.
2. **Medium severity**: Actor is informed and given the opportunity to self-correct.
3. **High severity**: Execution pauses, user is prompted for guidance.
4. **Critical**: Phase reverts or plan fails, depending on the automation profile settings.

The severity classification depends on the confidence score, the automation profile thresholds, and the specific layer that detected the issue.

## Constraints

- Layer 1 (decision-time validation) must run synchronously during Strategize. Decisions cannot be accepted without validation.
- Layer 2 (execution-time guards) must not significantly increase execution latency. Guards should be lightweight checks, not full re-analyses.
- Layer 3 (invariant enforcement) follows the invariant system's precedence rules (plan > action > project > global).
- Layer 4 (predictive prevention) is advisory only — it generates warnings and confidence adjustments, never blocks decisions autonomously.
- All four layers must be active simultaneously. Disabling a layer requires explicit configuration.

## Consequences

### Positive
- Multi-layered prevention catches errors at the earliest possible point, reducing correction cost.
- Invariant enforcement provides persistent, declarative constraints that survive across plan iterations.
- Predictive prevention improves over time as historical data accumulates.
- The escalation flow respects automation profiles, ensuring appropriate human involvement.

### Negative
- Four prevention layers add complexity to the execution engine.
- False positives from overly aggressive guards could slow execution and erode user trust.
- Predictive prevention requires sufficient historical data to be useful — cold-start problem.

### Risks
- The interaction between four layers may produce conflicting signals (e.g., Layer 1 approves a decision that Layer 3 rejects during execution).
- Performance impact of runtime guards on high-frequency tool invocations.
- Historical pattern matching may produce stale predictions if the codebase or tooling evolves significantly.

## Alternatives Considered

**Validation-only approach (catch errors after execution)** — Cheaper upfront but more expensive to fix. The multi-layered approach catches errors earlier in the lifecycle when they are less costly to correct.

**Single-layer invariant enforcement** — Misses decision-time and execution-time errors that are not expressible as invariants.

## Compliance

- **Layer coverage tests**: Tests verify that all four layers are active and correctly detect their respective error types.
- **Escalation tests**: Tests verify that the escalation flow correctly routes issues to the appropriate severity level based on confidence scores and automation profiles.
- **Guard performance tests**: Benchmarks verify that execution-time guards add acceptable latency (targets defined in ASV benchmarks).
- **False positive tracking**: Metrics track false positive rates for each layer, enabling tuning.
- **Integration tests**: BDD scenarios exercise multi-layer error detection including decision-time validation failure, execution-time guard intervention, invariant violation, and predictive warnings.
