# ADR-004: Data Validation

**Status:** Accepted  
**Date:** 2026-02-16  
**Supersedes:** None  
**Author(s):** Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com>  
**Approver(s):** Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com>

## Context

CleverAgents processes data from multiple untrusted or semi-trusted sources: user CLI input, YAML configuration files, LLM-generated tool call arguments, MCP server responses, environment variables, and database records. Every boundary crossing — between layers, between the system and external services, between the user and the application — is an opportunity for invalid data to enter the system and cause subtle downstream failures.

The system needs a uniform validation mechanism that provides runtime type checking, schema generation (for tool calling and API contracts), serialization/deserialization, and clear error messages — without requiring developers to write manual validation logic for each data boundary.

## Decision

All domain models, configuration objects, and API schemas use **Pydantic V2** (>= 2.7.0) for runtime data validation, JSON Schema generation, and serialization. **Pydantic Settings** (>= 2.11.0) extends this to environment variable loading. Pydantic serves as the type-safe data boundary between all layers.

## Design

### Domain Models

Every domain entity (Plan, Decision, Action, Resource, Actor, Tool, Skill, Session, Invariant, AutomationProfile, Checkpoint, CorrectionAttempt) is a Pydantic `BaseModel` subclass. Fields use Python type annotations with Pydantic field validators for business rule enforcement.

### Configuration Objects

YAML entity configuration files (actors, skills, tools, actions, resource types, automation profiles, context views) are parsed into Pydantic models. Validation errors are reported with field paths and human-readable messages before any processing occurs.

### Environment and Global Configuration

`BaseSettings` subclasses from Pydantic Settings handle environment variable loading with the `CLEVERAGENTS_` prefix, `.env` file support, and nested model resolution. This implements the four-tier configuration resolution chain (CLI flag > environment variable > project-scoped config > global config file > built-in default).

### JSON Schema Generation

Pydantic models automatically generate JSON Schema definitions, which are used for:

- Tool input/output schema definitions (consumed by LLM tool-calling protocols).
- Validation of LLM-generated tool call arguments before execution.
- API request/response schema documentation.

### Serialization

Pydantic handles serialization to and from JSON, YAML (via dict intermediary), and database column types. Custom serializers are defined for domain-specific types (ULIDs, datetime formats, enum values).

## Constraints

- All data crossing a layer boundary must pass through a Pydantic model. Raw dictionaries or untyped data structures must not propagate between layers.
- Pydantic's strict mode is used where appropriate to prevent implicit type coercion (e.g., strings silently becoming integers).
- Configuration YAML files must validate completely before any side effects occur. A partially valid configuration must never be applied.
- JSON Schema generated from Pydantic models is the canonical schema for tool definitions. Manual schema definitions must not diverge from the model.
- Validation error messages must include the field path and a human-readable explanation suitable for CLI display.

## Consequences

### Positive
- Runtime type safety is enforced at every data boundary without manual validation code.
- A single modeling framework serves domain models, configuration, API schemas, and tool definitions — reducing duplication.
- JSON Schema generation eliminates manual schema maintenance for tool calling and API contracts.
- Pydantic's error reporting provides structured, actionable validation messages.

### Negative
- Pydantic V2's strict mode can reject data that "looks right" to a developer unfamiliar with the strictness rules (e.g., `"123"` rejected where `123` is expected).
- All domain models carry a Pydantic dependency, which is a framework coupling in the domain layer. This is an accepted trade-off given Pydantic's role as a validation primitive rather than an infrastructure framework.
- Complex nested models with custom validators can be difficult to debug when validation fails deep in the model hierarchy.

### Risks
- If Pydantic's validation performance becomes a bottleneck (e.g., in hot paths during context assembly or decision tree traversal), selective use of `model_construct()` (bypassing validation) may be needed — but this must be explicitly justified and documented.
- Pydantic major version upgrades (V2 → future V3) could require model migration effort.

## Alternatives Considered

**dataclasses + manual validation** — Python dataclasses provide structure but no runtime validation, no JSON Schema generation, and no serialization. Every boundary check would require hand-written validation functions, leading to inconsistency and maintenance burden.

**attrs + cattrs** — A capable alternative, but lacks the JSON Schema generation and Settings integration that Pydantic provides out of the box. The LLM ecosystem (LangChain, MCP) standardizes on Pydantic models, making it the natural choice for interoperability.

## Compliance

- **Type checking**: Pyright in strict mode verifies that Pydantic model field types are consistent across the codebase.
- **Schema drift tests**: Automated tests verify that JSON Schema generated from Pydantic tool models matches the schemas expected by MCP and LangChain tool-calling protocols.
- **Validation coverage**: Unit tests exercise validation edge cases — missing required fields, wrong types, out-of-range values, malformed nested objects — for every domain model and configuration object.
- **Code review**: New data structures that cross layer boundaries must be Pydantic models. Raw dict usage at boundaries is flagged during review.

## Related ADRs

| ADR | Title | Relationship |
|-----|-------|-------------|
| [ADR-001](ADR-001-layered-architecture.md) | Layered Architecture | Pydantic models enforce Domain Layer contracts at layer boundaries |
| [ADR-005](ADR-005-technical-stack.md) | Technical Stack | Specifies Pydantic V2 as the chosen validation library |
| [ADR-011](ADR-011-tool-system.md) | Tool System | Tool input/output schemas are Pydantic models that generate JSON Schema |
| [ADR-024](ADR-024-configuration-system.md) | Configuration System | Pydantic Settings powers environment variable loading and config validation |

## Acceptance

### Votes For

| Voter | Comment |
|-------|---------|
| Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> | Pydantic V2 strict mode gives us runtime type safety with minimal boilerplate and native JSON Schema support |

**Total: 1**

### Votes Against

| Voter | Comment |
|-------|---------|

**Total: 0**

### Abstentions

| Voter | Comment |
|-------|---------|

**Total: 0**
