cleveragents-core/docs/architecture.md

# Architecture Overview

CleverAgents is a Python-first AI agent orchestration platform.  It provides
a unified CLI (`agents`), an interactive TUI, an embedded runtime, and
service orchestration tools for building and running complex, long-running
AI-driven workflows.

This document gives a developer-oriented overview of the system.  For the
authoritative specification see [`docs/specification.md`](specification.md).
For individual design decisions see the [ADR index](adr/index.md).

---

## High-Level Component Map

```mermaid
graph TD
    subgraph "Entry Points"
        CLI["CLI<br/>(agents / cleveragents)"]
        TUI["TUI<br/>(agents tui)"]
        SERVER["Server<br/>(agents server)"]
    end

    subgraph "Application Layer"
        A2A["A2A Facade<br/>(local / HTTP)"]
        DI["DI Container<br/>(application.container)"]
        SERVICES["Application Services<br/>(plan, session, actor, skill, tool, …)"]
    end

    subgraph "Domain Layer"
        PLAN["Plan Lifecycle<br/>(Strategize → Execute → Apply)"]
        ACTOR["Actor System<br/>(config, loader, compiler)"]
        RESOURCE["Resource DAG<br/>(handlers, inheritance)"]
        DECISION["Decision Tree<br/>(record, version, rollback)"]
    end

    subgraph "Infrastructure Layer"
        DB["Database<br/>(SQLAlchemy + Alembic)"]
        SANDBOX["Sandbox<br/>(checkpoint, rollback)"]
        OBS["Observability<br/>(logging, metrics, audit)"]
        EVENTS["Event Bus<br/>(reactive streams)"]
        PLUGINS["Plugin Manager"]
    end

    subgraph "Integration Layer"
        LANGGRAPH["LangGraph Bridge"]
        MCP["MCP Adapter"]
        LSP["LSP Client"]
        ACMS["ACMS<br/>(context management)"]
        UKO["UKO Runtime<br/>(knowledge ontology)"]
    end

    CLI --> A2A
    TUI --> A2A
    SERVER --> A2A
    A2A --> DI
    DI --> SERVICES
    SERVICES --> PLAN
    SERVICES --> ACTOR
    SERVICES --> RESOURCE
    SERVICES --> DECISION
    PLAN --> LANGGRAPH
    ACTOR --> LANGGRAPH
    LANGGRAPH --> MCP
    LANGGRAPH --> LSP
    SERVICES --> ACMS
    ACMS --> UKO
    SERVICES --> DB
    SERVICES --> SANDBOX
    SERVICES --> OBS
    SERVICES --> EVENTS
    SERVICES --> PLUGINS
```

---

## Layered Architecture

CleverAgents follows a strict layered architecture
([ADR-001](adr/ADR-001-layered-architecture.md)):

| Layer | Package | Responsibility |
|-------|---------|----------------|
| **Entry Points** | `cli`, `tui` | User-facing interfaces |
| **Application** | `application` | DI wiring, service orchestration |
| **Domain** | `domain`, `actor`, `skills`, `tool`, `resource` | Business logic |
| **Infrastructure** | `infrastructure` | Database, sandbox, events, observability |
| **Integration** | `a2a`, `mcp`, `lsp`, `langgraph`, `acms` | External protocol adapters |
| **Core** | `core`, `shared`, `config` | Cross-cutting utilities |

Dependencies flow **downward only** — upper layers may import from lower
layers but never the reverse.

---

## Plan Lifecycle

The plan lifecycle is the central abstraction
([ADR-006](adr/ADR-006-plan-lifecycle.md)):

```
Action ──► Strategize ──► Execute ──► Apply
  │            │              │          │
  │        (estimate)    (validate)  (diff review)
  │                           │
  └───────────────────────────┘
           (correction / rollback)
```

| Phase | Description |
|-------|-------------|
| **Action** | User intent captured as a plan request |
| **Strategize** | LLM generates a structured plan with operations |
| **Execute** | Operations executed against resources via tools |
| **Apply** | Validated changes committed; diff reviewed and approved |

Phase transitions are gated by:
- **Validation pipeline** — rule-based checks with severity levels
- **Definition-of-Done** — explicit criteria evaluated before Apply
- **Advisory locking** — prevents concurrent modification of the same plan
- **Invariant reconciliation** — `InvariantReconciliationActor` runs automatically
  at each phase transition (`start_strategize`, `execute_plan`, `apply_plan`);
  failures raise `ReconciliationBlockedError` and emit `INVARIANT_VIOLATED` events,
  blocking the transition until invariants are satisfied

---

## Actor System

Actors are the execution units ([ADR-010](adr/ADR-010-actor-and-agent-architecture.md),
[ADR-031](adr/ADR-031-actor-abstraction-definition.md)):

- Defined in YAML with a LangGraph node graph
- Each node binds an LLM, a set of tools, and optional LSP/MCP sources
- Compiled by `ActorCompiler` into a runnable `CompiledActor`
- Registered in `ActorRegistry` and resolved by name at runtime

```yaml
# examples/actors/graph_workflow.yaml
name: openai/gpt-4o
entry_node: planner
nodes:
  planner:
    model: gpt-4o
    tool_sources: [builtin, mcp://bash-tools]
  executor:
    model: gpt-4o-mini
    tool_sources: [builtin]
edges:
  - from: planner
    to: executor
    condition: needs_execution
```

---

## Tool System

Tools are the atomic capabilities available to actors
([ADR-011](adr/ADR-011-tool-system.md)):

- Registered in `ToolRegistry` with a `ToolSpec` (JSON Schema) and executor
- Four-stage lifecycle: **activate → validate → execute → deactivate**
- Sources: `builtin`, `mcp`, `lsp`, `skill`
- Safety profiles control sandbox requirements, human approval gates, and cost limits
- Read-only enforcement blocks write-capable tools on read-only plans

---

## Skill System

Skills are composable capability bundles
([ADR-012](adr/ADR-012-skill-system.md),
[ADR-028](adr/ADR-028-agent-skills-standard.md)):

- Loaded from YAML or AgentSkills.io-compatible directories
- Three-tier progressive disclosure: **discover → activate → deactivate**
- Each skill exposes one or more tools to the actor
- `SkillRegistry` tracks all registered skills; `SkillRefreshResult` reports changes

---

## Resource System

Resources are managed external entities
([ADR-008](adr/ADR-008-resource-system.md)):

- Organized as a DAG (Directed Acyclic Graph) with dependency tracking
- Type hierarchy with multiple inheritance and field merging
  ([ADR-042](adr/ADR-042-resource-type-inheritance.md))
- Built-in types: `file`, `directory`, `sqlite`, `postgresql`, `mysql`,
  `duckdb`, `container.docker`, `container.podman`, `lsp.*`, `cloud.*`
- Each type has a handler implementing CRUD, checkpoint, and rollback

---

## Context Management (ACMS)

The Advanced Context Management System manages what context is sent to the
LLM at each phase ([ADR-014](adr/ADR-014-context-management-acms.md)):

- **Tier 1** — Always-present system context
- **Tier 2** — Phase-specific context (plan, project, actor)
- **Tier 3** — Dynamic retrieval (semantic search, UKO inference)

The **UKO Runtime** (Universal Knowledge Ontology) provides graph-based
knowledge inference and persistence for Tier 3 context strategies.

Key UKO capabilities:

- **Typed triples with provenance** — every triple carries `sourceResource`,
  `validFrom`, and `isCurrent` metadata
- **Temporal versioning** — a revision chain tracks ontology state across
  indexing runs, enabling point-in-time queries
- **Implicit inference** — `UKOInferenceEngine` produces `uko:implicitSiblingOf`,
  `uko:implicitContains`, and `uko:implicitDependsOn` triples with confidence 0.7
- **Graph persistence** — `UKOGraphPersistence` serialises/restores state via
  JSON-file or in-memory backends across application restarts

**Context hydration** bridges the resource registry and the ACMS tier service.
`context_tier_hydrator.hydrate_tiers_for_plan()` reads files from all
resources linked to the active project and stores them as `TieredFragment`
objects in `ContextTierService` before context assembly begins.  Without
hydration, the LLM receives zero file context.  See
[ACMS Context Hydration](modules/context-hydration.md) for details.

---

## Invariant Reconciliation

The **Invariant Reconciliation Actor** (`builtin/invariant-reconciliation`)
is automatically invoked at the start of the Strategize, Execute, and Apply
phase transitions by `PlanLifecycleService`.

**Reconciliation algorithm** (spec §19440–19600):

1. Collect invariants from four scopes: global, project, action, plan.
2. Group by normalised text (case-insensitive, stripped).
3. Detect conflicts between invariants at different scopes.
4. Resolve using specificity: `plan > action > project > global`.
   Exception: `non_overridable` global invariants always win.
5. Record an `invariant_enforced` decision for each active invariant.
6. Return a reconciled `InvariantSet`.

**Failure behaviour:**

- Reconciliation failures block the phase transition with
  `ReconciliationBlockedError` and emit an `INVARIANT_VIOLATED` event.
- Post-correction reconciliation runs via `CORRECTION_APPLIED` event
  subscription (best-effort; does not block correction completion).

**DI registration:** `InvariantService` is registered as a Singleton
provider in the DI container.

```python
from cleveragents.actor.reconciliation import InvariantReconciliationActor

actor = InvariantReconciliationActor(
    invariant_service=container.invariant_service(),
    decision_service=container.decision_service(),
)
result = actor.run(plan_id="...", project_name="...", action_name="...")
# result.reconciled_set — effective InvariantSet
# result.conflicts     — list[ConflictRecord] with resolution details
```

---

## Git Worktree Sandbox Apply

When `plan apply` commits changes to a git-checkout resource, the changes
are isolated in a dedicated git branch and worktree rather than written
directly to the working tree.

**Flow:**

1. `PlanLifecycleService` detects a git-checkout resource and creates a
   `GitWorktreeSandbox`.
2. A new branch `cleveragents/plan-<plan_id>` is created from the current HEAD.
3. A temporary worktree is created at a system temp directory.
4. The actor writes changes inside the worktree.
5. On approval: changes are staged, committed in the worktree, then merged
   back to the original branch via `git merge`.
6. On rejection: the worktree is discarded and the original branch is
   untouched.

Non-git projects fall back to the original flat file copy strategy.

See [Git Worktree Sandbox](modules/git-worktree-sandbox.md) for the full
API reference.

---

## A2A Protocol Boundary

All CLI, TUI, and server interactions cross the A2A boundary
([ADR-026](adr/ADR-026-agent-client-protocol.md),
[ADR-047](adr/ADR-047-acp-standard-adoption.md)):

- **Local mode** — `A2aLocalFacade` maps operations to direct Python calls
- **Server mode** — `A2aHttpTransport` (stub; full implementation pending)
- Version negotiation via `A2aVersionNegotiator`

---

## MCP Integration

External tool servers are integrated via the Model Context Protocol
([ADR-029](adr/ADR-029-model-context-protocol.md)):

- `MCPToolAdapter` manages server lifecycle and tool discovery
- `McpClient` provides lazy start and auto-stop
- `SandboxPathRewriter` prevents path traversal
- `MCPRefreshHook` keeps skill registrations in sync with server changes

---

## LSP Integration

Language Server Protocol servers provide code intelligence tools
([ADR-027](adr/ADR-027-language-server-protocol.md)):

- `LSPClient` manages server lifecycle and capability negotiation
- LSP tools are registered in `ToolRegistry` with `source="lsp"`
- Per-node LSP bindings in actor YAML control which server each node uses

---

## Dependency Injection

All application services are wired via the DI container
([ADR-003](adr/ADR-003-dependency-injection.md)):

```python
from cleveragents.application.container import Container

container = Container()
plan_service = container.plan_service()
session_service = container.session_service()
```

The container is the single source of truth for service instantiation.
Never construct services directly in application code.

---

## Observability

([ADR-025](adr/ADR-025-observability-and-logging.md))

- Structured logging via `cleveragents.config.logging`
- Metrics processing via `cleveragents.config.metrics_processor`
- Audit logging via `infrastructure.observability`
- Token/cost tracking per plan and per day with budget enforcement

---

## Key Design Decisions

| Decision | ADR | Summary |
|----------|-----|---------|
| Layered architecture | [ADR-001](adr/ADR-001-layered-architecture.md) | Strict layer boundaries, no upward imports |
| Namespace system | [ADR-002](adr/ADR-002-namespace-system.md) | `provider/name` qualified identifiers |
| Dependency injection | [ADR-003](adr/ADR-003-dependency-injection.md) | Single DI container, no service locator |
| Plan lifecycle | [ADR-006](adr/ADR-006-plan-lifecycle.md) | Four-phase lifecycle with gating |
| LangGraph integration | [ADR-022](adr/ADR-022-langchain-langgraph-integration.md) | LangGraph as graph execution engine |
| Sandboxing | [ADR-015](adr/ADR-015-sandbox-and-checkpoint.md) | Checkpoint/rollback for all resource writes |
| Template security | [ADR-032](adr/ADR-032-jinja2-yaml-template-preprocessing.md) | Sandboxed Jinja2 with allowlist |
| TUI framework | [ADR-044](adr/ADR-044-tui-architecture-and-framework.md) | Textual-based full-screen TUI with first-run experience, persona system, session export/import |
| Server architecture | [ADR-048](adr/ADR-048-server-application-architecture.md) | ASGI server with Kubernetes deployment |