cleveragents-core/docs/architecture.md

Architecture Overview

CleverAgents is a Python-first AI agent orchestration platform. It provides a unified CLI (agents), an interactive TUI, an embedded runtime, and service orchestration tools for building and running complex, long-running AI-driven workflows.

This document gives a developer-oriented overview of the system. For the authoritative specification see docs/specification.md. For individual design decisions see the ADR index.

---

High-Level Component Map

graph TD
    subgraph "Entry Points"
        CLI["CLI<br/>(agents / cleveragents)"]
        TUI["TUI<br/>(agents tui)"]
        SERVER["Server<br/>(agents server)"]
    end

    subgraph "Application Layer"
        A2A["A2A Facade<br/>(local / HTTP)"]
        DI["DI Container<br/>(application.container)"]
        SERVICES["Application Services<br/>(plan, session, actor, skill, tool, …)"]
    end

    subgraph "Domain Layer"
        PLAN["Plan Lifecycle<br/>(Strategize → Execute → Apply)"]
        ACTOR["Actor System<br/>(config, loader, compiler)"]
        RESOURCE["Resource DAG<br/>(handlers, inheritance)"]
        DECISION["Decision Tree<br/>(record, version, rollback)"]
    end

    subgraph "Infrastructure Layer"
        DB["Database<br/>(SQLAlchemy + Alembic)"]
        SANDBOX["Sandbox<br/>(checkpoint, rollback)"]
        OBS["Observability<br/>(logging, metrics, audit)"]
        EVENTS["Event Bus<br/>(reactive streams)"]
        PLUGINS["Plugin Manager"]
    end

    subgraph "Integration Layer"
        LANGGRAPH["LangGraph Bridge"]
        MCP["MCP Adapter"]
        LSP["LSP Client"]
        ACMS["ACMS<br/>(context management)"]
        UKO["UKO Runtime<br/>(knowledge ontology)"]
    end

    CLI --> A2A
    TUI --> A2A
    SERVER --> A2A
    A2A --> DI
    DI --> SERVICES
    SERVICES --> PLAN
    SERVICES --> ACTOR
    SERVICES --> RESOURCE
    SERVICES --> DECISION
    PLAN --> LANGGRAPH
    ACTOR --> LANGGRAPH
    LANGGRAPH --> MCP
    LANGGRAPH --> LSP
    SERVICES --> ACMS
    ACMS --> UKO
    SERVICES --> DB
    SERVICES --> SANDBOX
    SERVICES --> OBS
    SERVICES --> EVENTS
    SERVICES --> PLUGINS

---

Layered Architecture

CleverAgents follows a strict layered architecture (ADR-001):

Layer	Package	Responsibility
Entry Points	cli, tui	User-facing interfaces
Application	application	DI wiring, service orchestration
Domain	domain, actor, skills, tool, resource	Business logic
Infrastructure	infrastructure	Database, sandbox, events, observability
Integration	a2a, mcp, lsp, langgraph, acms	External protocol adapters
Core	core, shared, config	Cross-cutting utilities

Dependencies flow downward only — upper layers may import from lower layers but never the reverse.

---

Plan Lifecycle

The plan lifecycle is the central abstraction (ADR-006):

Action ──► Strategize ──► Execute ──► Apply
  │            │              │          │
  │        (estimate)    (validate)  (diff review)
  │                           │
  └───────────────────────────┘
           (correction / rollback)

Phase	Description
Action	User intent captured as a plan request
Strategize	LLM generates a structured plan with operations
Execute	Operations executed against resources via tools
Apply	Validated changes committed; diff reviewed and approved

Phase transitions are gated by:

• Validation pipeline — rule-based checks with severity levels
• Definition-of-Done — explicit criteria evaluated before Apply
• Advisory locking — prevents concurrent modification of the same plan
• Invariant reconciliation — InvariantReconciliationActor runs automatically at each phase transition (start_strategize, execute_plan, apply_plan); failures raise ReconciliationBlockedError and emit INVARIANT_VIOLATED events, blocking the transition until invariants are satisfied

---

Actor System

Actors are the execution units (ADR-010, ADR-031):

• Defined in YAML with a LangGraph node graph
• Each node binds an LLM, a set of tools, and optional LSP/MCP sources
• Compiled by ActorCompiler into a runnable CompiledActor
• Registered in ActorRegistry and resolved by name at runtime

# examples/actors/graph_workflow.yaml
name: openai/gpt-4o
entry_node: planner
nodes:
  planner:
    model: gpt-4o
    tool_sources: [builtin, mcp://bash-tools]
  executor:
    model: gpt-4o-mini
    tool_sources: [builtin]
edges:
  - from: planner
    to: executor
    condition: needs_execution

---

Tool System

Tools are the atomic capabilities available to actors (ADR-011):

• Registered in ToolRegistry with a ToolSpec (JSON Schema) and executor
• Four-stage lifecycle: activate → validate → execute → deactivate
• Sources: builtin, mcp, lsp, skill
• Safety profiles control sandbox requirements, human approval gates, and cost limits
• Read-only enforcement blocks write-capable tools on read-only plans

---

Skill System

Skills are composable capability bundles (ADR-012, ADR-028):

• Loaded from YAML or AgentSkills.io-compatible directories
• Three-tier progressive disclosure: discover → activate → deactivate
• Each skill exposes one or more tools to the actor
• SkillRegistry tracks all registered skills; SkillRefreshResult reports changes

---

Resource System

Resources are managed external entities (ADR-008):

• Organized as a DAG (Directed Acyclic Graph) with dependency tracking
• Type hierarchy with multiple inheritance and field merging (ADR-042)
• Built-in types: file, directory, sqlite, postgresql, mysql, duckdb, container.docker, container.podman, lsp.*, cloud.*
• Each type has a handler implementing CRUD, checkpoint, and rollback

---

Context Management (ACMS)

The Advanced Context Management System manages what context is sent to the LLM at each phase (ADR-014):

• Tier 1 — Always-present system context
• Tier 2 — Phase-specific context (plan, project, actor)
• Tier 3 — Dynamic retrieval (semantic search, UKO inference)

The UKO Runtime (Universal Knowledge Ontology) provides graph-based knowledge inference and persistence for Tier 3 context strategies.

Key UKO capabilities:

• Typed triples with provenance — every triple carries sourceResource, validFrom, and isCurrent metadata
• Temporal versioning — a revision chain tracks ontology state across indexing runs, enabling point-in-time queries
• Implicit inference — UKOInferenceEngine produces uko:implicitSiblingOf, uko:implicitContains, and uko:implicitDependsOn triples with confidence 0.7
• Graph persistence — UKOGraphPersistence serialises/restores state via JSON-file or in-memory backends across application restarts

Context hydration bridges the resource registry and the ACMS tier service. context_tier_hydrator.hydrate_tiers_for_plan() reads files from all resources linked to the active project and stores them as TieredFragment objects in ContextTierService before context assembly begins. Without hydration, the LLM receives zero file context. See ACMS Context Hydration for details.

---

Invariant Reconciliation

The Invariant Reconciliation Actor (builtin/invariant-reconciliation) is automatically invoked at the start of the Strategize, Execute, and Apply phase transitions by PlanLifecycleService.

Reconciliation algorithm (spec §19440–19600):

1. Collect invariants from four scopes: global, project, action, plan.
2. Group by normalised text (case-insensitive, stripped).
3. Detect conflicts between invariants at different scopes.
4. Resolve using specificity: plan > action > project > global. Exception: non_overridable global invariants always win.
5. Record an invariant_enforced decision for each active invariant.
6. Return a reconciled InvariantSet.

Failure behaviour:

• Reconciliation failures block the phase transition with ReconciliationBlockedError and emit an INVARIANT_VIOLATED event.
• Post-correction reconciliation runs via CORRECTION_APPLIED event subscription (best-effort; does not block correction completion).

DI registration: InvariantService is registered as a Singleton provider in the DI container.

from cleveragents.actor.reconciliation import InvariantReconciliationActor

actor = InvariantReconciliationActor(
    invariant_service=container.invariant_service(),
    decision_service=container.decision_service(),
)
result = actor.run(plan_id="...", project_name="...", action_name="...")
# result.reconciled_set — effective InvariantSet
# result.conflicts     — list[ConflictRecord] with resolution details

---

Git Worktree Sandbox Apply

When plan apply commits changes to a git-checkout resource, the changes are isolated in a dedicated git branch and worktree rather than written directly to the working tree.

Flow:

1. PlanLifecycleService detects a git-checkout resource and creates a GitWorktreeSandbox.
2. A new branch cleveragents/plan-<plan_id> is created from the current HEAD.
3. A temporary worktree is created at a system temp directory.
4. The actor writes changes inside the worktree.
5. On approval: changes are staged, committed in the worktree, then merged back to the original branch via git merge.
6. On rejection: the worktree is discarded and the original branch is untouched.

Non-git projects fall back to the original flat file copy strategy.

See Git Worktree Sandbox for the full API reference.

---

A2A Protocol Boundary

All CLI, TUI, and server interactions cross the A2A boundary (ADR-026, ADR-047):

• Local mode — A2aLocalFacade maps operations to direct Python calls
• Server mode — A2aHttpTransport (stub; full implementation pending)
• Version negotiation via A2aVersionNegotiator

---

MCP Integration

External tool servers are integrated via the Model Context Protocol (ADR-029):

• MCPToolAdapter manages server lifecycle and tool discovery
• McpClient provides lazy start and auto-stop
• SandboxPathRewriter prevents path traversal
• MCPRefreshHook keeps skill registrations in sync with server changes

---

LSP Integration

Language Server Protocol servers provide code intelligence tools (ADR-027):

• LSPClient manages server lifecycle and capability negotiation
• LSP tools are registered in ToolRegistry with source="lsp"
• Per-node LSP bindings in actor YAML control which server each node uses

---

Dependency Injection

All application services are wired via the DI container (ADR-003):

from cleveragents.application.container import Container

container = Container()
plan_service = container.plan_service()
session_service = container.session_service()

The container is the single source of truth for service instantiation. Never construct services directly in application code.

---

Observability

(ADR-025)

• Structured logging via cleveragents.config.logging
• Metrics processing via cleveragents.config.metrics_processor
• Audit logging via infrastructure.observability
• Token/cost tracking per plan and per day with budget enforcement

---

Key Design Decisions

Decision	ADR	Summary
Layered architecture	ADR-001	Strict layer boundaries, no upward imports
Namespace system	ADR-002	provider/name qualified identifiers
Dependency injection	ADR-003	Single DI container, no service locator
Plan lifecycle	ADR-006	Four-phase lifecycle with gating
LangGraph integration	ADR-022	LangGraph as graph execution engine
Sandboxing	ADR-015	Checkpoint/rollback for all resource writes
Template security	ADR-032	Sandboxed Jinja2 with allowlist
TUI framework	ADR-044	Textual-based full-screen TUI with first-run experience, persona system, session export/import
Server architecture	ADR-048	ASGI server with Kubernetes deployment