cleveragents-core/docs/adr/ADR-013-validation-abstraction.md

---
adr_number: 13
title: Validation Abstraction
status_history:
- - '2026-02-16'
  - Proposed
  - Jeffrey Phillips Freeman
- - '2026-02-16'
  - Accepted
  - Jeffrey Phillips Freeman
tier: 2
authors:
- Jeffrey Phillips Freeman
superseded_by: null
related_adrs:
- number: 6
  title: Plan Lifecycle
  relationship: Validations gate phase transitions and can trigger phase reversion
- number: 11
  title: Tool System
  relationship: Validations are a specialized Tool subtype inheriting the tool lifecycle
- number: 12
  title: Skill System
  relationship: Validation tools can be composed into skills alongside regular tools
- number: 16
  title: Invariant System
  relationship: Invariant enforcement uses validation tools as its execution mechanism
acceptance:
  votes_for:
  - voter: Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com>
    comment: Modeling validation as a Tool subtype with structured pass/fail keeps the system uniform and composable
  votes_against: []
  abstentions: []
---
## Context

Plans must verify that their work meets quality, correctness, and compliance criteria before changes are applied to real project resources. This verification must be structured (machine-parseable pass/fail results, not free-text), composable (multiple validations can run in parallel), and gating (required validations block Apply if they fail). The system needs a validation mechanism that integrates with the existing tool infrastructure rather than being a separate system.

## Decision Drivers

- Verification results must be machine-parseable (structured pass/fail), not free-text LLM interpretation
- Validations must integrate with the existing tool infrastructure rather than creating a parallel system
- Required validations must gate the Apply phase — failed checks block changes from reaching real resources
- Read-only enforcement is essential for safe parallel execution and retry without side effects
- Validation attachment must support multiple scoping levels (direct, project, plan) for fine-grained control
- Existing tools should be reusable as validations via wrapping, avoiding reimplementation

## Decision

A **Validation** is a specialized subtype of Tool designed to verify work. It extends Tool via standard class inheritance, adding `mode` (required/informational), a structured JSON return format (`{ "passed": bool, "message": str, "data": object }`), and strict read-only enforcement. Validations share the tool namespace, reuse the entire tool infrastructure, and are composable into skills and actor graphs.

## Design

### Relationship to Tool

Validations inherit all Tool fields: `name`, `description`, `source`, `input_schema`, `output_schema`, `capability`, `resource_slots`, `lifecycle` hooks, `code`, `mcp_server`/`mcp_tool_name`, `timeout`, `idempotent`.

Additional validation-specific fields: `mode`, structured return format enforcement, read-only enforcement, attachment scoping, attachment ULID, `wraps` (for tool wrapping), and `transform`.

Validations share the same namespace as tools. They are a superset (validation IS-A tool). Unified management via `agents tool` and `agents validation` commands.

### Validation Mode

**Required** (`mode: required`): Must pass for execution to proceed to Apply. On failure: fix-then-revalidate loop with retry limit (default: 3), then strategy revision, then escalation to user.

**Informational** (`mode: informational`): Recorded but does not block execution. Useful for advisory checks and gradual adoption. Mode is set at registration time and cannot be overridden per-attachment.

### Structured Return Format

```json
{
  "passed": true,
  "message": "All 247 tests passed, 94% coverage",
  "data": {
    "tests_run": 247,
    "tests_passed": 247,
    "coverage_percent": 94.2
  }
}
```

`passed` (boolean) is required. `message` (string) and `data` (object) are optional. Malformed returns are treated as errors with `passed: false`.

### Read-Only Enforcement

Validations are always read-only: `writes: false`, `checkpointable: false`, `read_only: true`. Enforced at registration time and at runtime. This guarantee enables safe parallelization, speculative validation, and retry without side-effect concerns.

### Attachment Scoping

Validations are attached to resources with three scoping levels:

1. **Direct attachment**: Always active for that resource, regardless of plan or project.
2. **Project-scoped attachment**: Active only when the resource is accessed through the specified project.
3. **Plan-scoped attachment**: Active only for the specified plan.

Attachment resolution: collect all applicable attachments (direct → project → plan), union them, most-specific scope wins for identical arguments. Each attachment gets a system-assigned ULID, enabling the same validation to be attached multiple times with different arguments.

### Validation Lifecycle in Plan Execution

**Strategize**: Not executed, but the strategy actor is aware of applicable validations and plans accordingly.

**Execute**: Final step of execution. Collect all applicable validations → execute in parallel (safe due to read-only) → process results. All required pass → proceed to Apply. Any required fail → fix-then-revalidate loop.

**Apply**: Not run. The sandbox model ensures all verification happens during Execute. Apply is the point of no return.

### Required Validation Failure Handling

1. Diagnose (examine `message` and `data`)
2. Self-fix attempt by the execution actor
3. Re-validation
4. Retry limit reached (default: 3)
5. Request strategy revision (revert to Strategize)
6. Escalate to user
7. Terminal failure if still unresolved

### Tool Wrapping

Validations can wrap existing tools using `wraps` and `transform` fields. The `wraps` field references an existing tool; the `transform` function interprets its output as pass/fail. Wrapped tools inherit `input_schema`, `resource_slots`, and `timeout` from the wrapped tool. The transform is a pure data transformation that receives `tool_output` and returns the structured validation result. Multiple validations can wrap the same tool with different transforms.

### Validations in Actor Graphs and Skills

Validations can appear as tool nodes in actor graphs, with return values available to downstream nodes. Being in a skill makes a validation callable; attachment determines whether it acts as a mandatory gate.

## Constraints

- Validations must always be read-only. Registration of a validation with `writes: true` is rejected.
- The structured return format is enforced. `passed` (boolean) is required in every return value.
- Validation mode (`required` / `informational`) is set at registration and cannot be overridden per-attachment.
- Validations run during Execute, never during Apply.
- Required validation failures must follow the fix-then-revalidate → strategy revision → escalation sequence. No bypassing.
- Wrapped tool transforms must be pure data transformations with no side effects.

## Consequences

### Positive
- Extending Tool rather than creating a separate system means validations benefit from the entire tool infrastructure — lifecycle, adapters, resource bindings, skills, actor graphs.
- Read-only enforcement enables safe parallel execution of all validations.
- Attachment scoping provides fine-grained control over which validations apply in which contexts.
- The structured return format enables machine-driven fix-then-revalidate loops.
- Tool wrapping enables reusing existing tools as validations without reimplementation.

### Negative
- The IS-A relationship with Tool means validation-specific behavior (mode, gating) must be carefully distinguished from general tool behavior in the execution engine.
- Attachment scoping with three levels and system-assigned ULIDs adds management complexity.
- The fix-then-revalidate loop with strategy revision escalation is a complex state machine.

### Risks
- Required validations that are flaky (intermittent failures unrelated to code quality) could block plans and erode trust in the system.
- The retry limit is a blunt instrument — some failures need more attempts while others should fail immediately.
- Informational validations may be ignored entirely if there is no mechanism to surface their results prominently.

## Alternatives Considered

**Separate validation system (not extending Tool)** — Would duplicate the lifecycle, adapter, registry, and namespace infrastructure. The specification explicitly chose inheritance to avoid this duplication.

**Validations only in Apply phase** — Would require running validations on real (non-sandboxed) resources, risking failures after changes are partially committed. The sandbox model requires validation during Execute.

## Compliance

- **Read-only enforcement tests**: Tests verify that validation registration rejects `writes: true` and that runtime execution blocks any write attempts.
- **Structured return tests**: Tests verify that well-formed, malformed, and missing return values are handled correctly.
- **Attachment scoping tests**: Tests verify correct resolution across direct, project-scoped, and plan-scoped attachments.
- **Failure handling tests**: BDD scenarios exercise the full failure sequence: fix → revalidate → retry limit → strategy revision → escalation → terminal failure.
- **Wrapping tests**: Tests verify that wrapped tool transforms correctly interpret tool output and that inheritance rules are applied.