docs: add documentation for completed milestones v3.0.0 and v3.1.0 [AUTO-DOCS-1] #9738

Closed
HAL9000 wants to merge 1 commits from docs/auto-docs-1-milestone-docs-v3.0.0-v3.1.0 into master
5 changed files with 708 additions and 0 deletions
+98
View File
@@ -291,3 +291,101 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
renders permission requests directly in the conversation stream for single-file
operations. Users can allow/reject with single-key shortcuts (`a`/`A`/`r`/`R`),
navigate with arrow keys, confirm with `Enter`, or press `v` to open the full
## [3.1.0] — 2025-Q4
### Added
- **Actor Compiler** — GRAPH-type actor YAML files now compile into live
LangGraph `StateGraph` structures via `ActorCompiler`. Compilation validates
node references, detects intra-graph and cross-actor subgraph cycles, maps
node types to LangGraph `NodeType` values, and returns a `CompiledActor`
bundle with nodes, edges, entry point, and `CompilationMetadata`.
- **`agents actor add --config actor.yaml`** — New CLI command loads and
registers custom actors from YAML files. The loader parses, validates via
Pydantic, enforces graph topology rules, and persists to the actor registry.
Supports `--update`, `--unsafe`, and `--set-default` flags.
- **Actor YAML `version: "3"` schema** — Actor YAML files now declare
`version: "3"` and support `type: llm | tool | graph`. All fields are
validated via Pydantic with precise error messages including dotted field
paths.
- **Tool Call Router** — `ToolCallRouter` normalizes OpenAI, Anthropic, and
LangChain tool call formats into `NormalizedToolCallResult` objects.
Supports single routing, batch routing, and streaming execution. Generates
stable deterministic tool call IDs from `plan_id + sequence`.
- **MCP Adapter** — `MCPToolAdapter` connects to external MCP (Model Context
Protocol) tool servers, discovers tools, validates inputs against JSON
Schema, and bulk-registers tools into the `ToolRegistry`. Supports `stdio`,
`sse`, and `streamable-http` transports. Thread-safe via `threading.RLock`.
- **Validation Runner** — Executes resource-attached validations before the
apply gate. Classifies validations as required (blocking) or informational
(non-blocking). `ApplyValidationSummary.all_required_passed` returns `False`
when zero validations were run (empty-run guard).
- **Multi-file ChangeSet generation** — The execution actor now correctly
produces `ChangeSet` records with `added_files`, `modified_files`, and
`deleted_files` for multi-file generation scenarios.
- **Skill registry and tool lifecycle CLI** — `agents skill add/list/show/remove`
commands manage named skill collections. Skills provide tool sets to actors
during plan execution.
---
## [3.0.0] — 2025-Q3
### Added
- **`agents action create --config action.yaml`** — Register reusable action
templates from YAML files. Actions are persisted to SQLite and include
`name`, `description`, `strategy_actor`, `execution_actor`,
`definition_of_done`, optional `arguments`, `invariants`, `inputs_schema`,
and `automation_profile`.
- **`agents resource add git-checkout`** — Register git repositories as
resources in the resource registry. The `git-checkout` resource type is the
primary resource type for the git worktree sandbox.
- **`agents project create` and `agents project link-resource`** — Create
projects and link resources to them. Projects group resources together and
provide context for plan execution.
- **`agents plan use`** — Create plan records from action templates and
projects. Plans are persisted to SQLite with ULID identifiers and start in
the `strategize/queued` state.
- **`agents plan execute <plan_id>`** — Run the Strategize and Execute phases
of the plan lifecycle. Invokes the actor-based LLM path and writes generated
file changes to the git worktree sandbox.
- **`agents plan diff <plan_id>`** — Show pending sandbox changes as a unified
diff for review before applying.
- **`agents plan apply <plan_id>`** — Merge sandbox changes into the real git
repository via `git merge` from the isolated worktree branch. Non-git
projects fall back to flat file copy.
- **Git worktree sandbox** — Isolated working directories for plan execution.
Each plan gets its own branch (`cleveragents/plan-<plan_id>`) and a
temporary worktree. On apply, the sandbox branch is merged back into the
original branch and the worktree is cleaned up.
- **Pydantic v2 domain models with `frozen=True`** — All domain models
(`ActionRecord`, `ResourceRecord`, `ProjectRecord`, `PlanRecord`,
`ChangeSet`, etc.) use Pydantic v2 with `frozen=True` for immutability
guarantees.
- **SQLite persistence with Alembic migrations** — All records are persisted
to a local SQLite database. Schema migrations are managed by Alembic and
run automatically on first use.
- **Plan state machine** — Plans follow a strict state machine:
`strategize/queued``strategize/in_progress``strategize/complete`
`execute/queued``execute/in_progress``execute/complete`
`apply/queued``apply/in_progress``applied`. Terminal states:
`applied`, `cancelled`, `errored`.
+38
View File
@@ -170,3 +170,41 @@ Set `CLEVERAGENTS_DEFAULT_PROVIDER` to pin the global provider (for example `exp
- Built-in actors (`<provider>/<model>`) are immutable, custom actors must be named `local/<id>`, and the default actor cannot be removed. Use `--unsafe` when adding/updating configs marked unsafe; runtime only warns when invoking unsafe actors.
- `CLEVERAGENTS_TESTING_USE_MOCK_AI=true` forces the in-repo mock provider so Behave/Robot suites never hit external APIs.
- The full capability matrix (streaming, tool calls, JSON mode, etc.) is documented in `docs/reference/providers.md`.
## Milestone History
### v3.0.0 — M1: Minimal Local Source-Code Workflow
The first minimally usable local-mode flow. Register an action from YAML,
link a git repository resource to a project, and run a plan end-to-end.
**Key capabilities:**
- `agents action create --config action.yaml` — register action templates
- `agents resource add git-checkout` — register git repository resources
- `agents project create` / `agents project link-resource` — project management
- `agents plan use` — create plan records with state machine transitions
- `agents plan execute <plan_id>` — invoke actor-based LLM path
- `agents plan diff <plan_id>` — review pending sandbox changes
- `agents plan apply <plan_id>` — merge sandbox changes with git commit
- Git worktree sandbox for isolated working directories
- Pydantic v2 domain models with `frozen=True`
- SQLite persistence with Alembic migrations
See [`docs/modules/milestone-v3.0.0-local-workflow.md`](docs/modules/milestone-v3.0.0-local-workflow.md) for the full guide.
### v3.1.0 — M2: Actor Compiler + Full LLM Integration
Actor YAML files compile into live LangGraph graphs. Custom actors are fully
operational with a normalized tool router, validation runner, and MCP adapter.
**Key capabilities:**
- Actor YAML files with `version: "3"`, `type: llm|tool|graph` — parse and validate via Pydantic
- GRAPH-type actors compile into LangGraph `StateGraph` structures
- `agents actor add --config actor.yaml` — load and register custom actors
- Skill registry and tool lifecycle via CLI (`agents skill add/list/show/remove`)
- MCP adapter discovers and connects to external tool servers
- Tool call router normalizes OpenAI / Anthropic / LangChain formats
- Validation runner executes required and informational validations
- Multi-file generation produces correct `ChangeSet`
See [`docs/modules/milestone-v3.1.0-actor-compiler.md`](docs/modules/milestone-v3.1.0-actor-compiler.md) for the full guide.
@@ -0,0 +1,265 @@
# v3.0.0 — M1: Minimal Local Source-Code Workflow
**Milestone:** v3.0.0 — M1 (CLOSED)
**Released:** 2025-Q3
This milestone delivered the first minimally usable local-mode flow for
CleverAgents. A user can register an action from YAML, link a git repository
resource to a project, and run a plan end-to-end — from creation through
execution, diff review, and final apply — using a sandboxed workspace with
tool-based change capture.
---
## Overview
The M1 milestone established the foundational plan lifecycle:
```
agents action create → agents resource add → agents project create
│ │
└──────────────────────────────────────────────►│
agents plan use
agents plan execute
agents plan diff
agents plan apply
```
All state is persisted to SQLite via Alembic-managed migrations. Domain
models use Pydantic v2 with `frozen=True` for immutability guarantees.
---
## CLI Commands Introduced
### `agents action create`
Register a reusable action template from a YAML configuration file.
```bash
agents action create --config action.yaml
```
**Minimal `action.yaml`:**
```yaml
name: local/my-action
description: Describe what this action does
strategy_actor: openai/gpt-4
execution_actor: openai/gpt-4
definition_of_done: |
The task is complete when all acceptance criteria are met.
```
Actions are persisted to SQLite and can be listed with `agents action list`.
See [Action CLI Reference](../reference/action_cli.md) for the full field
listing and options.
---
### `agents resource add git-checkout`
Register a git repository as a resource in the resource registry.
```bash
agents resource add git-checkout local/my-repo --path /home/user/my-repo
```
The `git-checkout` resource type represents a local git repository. It is
the primary resource type used by the git worktree sandbox during plan
execution.
See [Resource CLI Reference](../reference/resource_cli.md) for all resource
commands.
---
### `agents project create` and `agents project link-resource`
Create a project and link one or more resources to it.
```bash
agents project create local/my-project --description "My project"
agents project link-resource local/my-project local/my-repo
```
Projects group resources together and provide the context for plan execution.
Multiple resources can be linked to a single project.
---
### `agents plan use`
Create a plan record from an action template and one or more projects.
```bash
agents plan use local/my-action local/my-project
```
This creates a plan in the `strategize/queued` state and persists it to
SQLite. The plan is assigned a ULID identifier.
See [Plan CLI Reference](../reference/plan_cli.md) for full options including
`--arg`, `--automation-profile`, and `--invariant` flags.
---
### `agents plan execute <plan_id>`
Run the Strategize and Execute phases of the plan lifecycle. This invokes
the actor-based LLM path and writes generated file changes to the sandbox.
```bash
agents plan execute 01HXYZ1234567890ABCDEFGH
```
The executor:
1. Runs the **Strategize** phase — the strategy actor produces a decision
tree of tasks.
2. Runs the **Execute** phase — the execution actor generates file changes
and writes them to the git worktree sandbox.
When the automation profile permits auto-apply (e.g. `ci` or `full-auto`),
`plan execute` drives the plan all the way to `applied` in a single call.
---
### `agents plan diff <plan_id>`
Show pending changes in the sandbox as a unified diff.
```bash
agents plan diff 01HXYZ1234567890ABCDEFGH
```
Displays the ChangeSet as a unified diff so you can review what the LLM
generated before committing it to the real repository.
---
### `agents plan apply <plan_id>`
Merge sandbox changes into the real git repository with a commit.
```bash
agents plan apply 01HXYZ1234567890ABCDEFGH
```
For git-checkout resources, this uses the **git worktree sandbox** strategy:
changes are merged via `git merge` from the isolated worktree branch into
the original branch. Non-git projects fall back to flat file copy.
Pass `--yes` / `-y` to skip the confirmation prompt in CI pipelines.
---
## Git Worktree Sandbox
The git worktree sandbox provides isolated working directories for plan
execution. Each plan gets its own branch (`cleveragents/plan-<plan_id>`)
and a temporary worktree directory.
**Lifecycle:**
```
PENDING → CREATED → ACTIVE → COMMITTED → CLEANED_UP
└→ ROLLED_BACK → CLEANED_UP
```
On `plan apply`, the sandbox branch is merged back into the original branch
and the worktree is cleaned up. On failure or cancellation, the worktree is
discarded without touching the original branch.
See [Git Worktree Sandbox](git-worktree-sandbox.md) for the full API
reference.
---
## Domain Models
All domain models introduced in M1 use **Pydantic v2** with `frozen=True`:
| Model | Description |
|-------|-------------|
| `ActionRecord` | Persisted action template |
| `ActionArgument` | Typed argument definition for an action |
| `ResourceRecord` | Registered resource instance |
| `ProjectRecord` | Project grouping resources |
| `ProjectResourceLink` | Many-to-many link between projects and resources |
| `PlanRecord` | Plan instance with state machine |
| `ChangeSet` | Set of file changes produced by execution |
Frozen models ensure that domain objects are never mutated after creation,
preventing accidental state corruption in multi-phase workflows.
---
## SQLite Persistence and Alembic Migrations
All records are persisted to a local SQLite database (default:
`~/.config/cleveragents/cleveragents.db`). Schema migrations are managed
by **Alembic** and run automatically on first use.
The database path can be overridden via the `CLEVERAGENTS_DB_PATH`
environment variable or the `database.path` config key.
---
## State Machine
Plans follow a strict state machine with the following phases and states:
| Phase | States |
|-------|--------|
| `strategize` | `queued``in_progress``complete` / `errored` |
| `execute` | `queued``in_progress``complete` / `errored` |
| `apply` | `queued``in_progress``applied` / `errored` |
Terminal states: `applied`, `cancelled`, `errored`.
---
## End-to-End Example
```bash
# 1. Register an action
agents action create --config action.yaml
# 2. Register a git repository resource
agents resource add git-checkout local/my-repo --path /home/user/my-repo
# 3. Create a project and link the resource
agents project create local/my-project
agents project link-resource local/my-project local/my-repo
# 4. Create a plan
agents plan use local/my-action local/my-project
# 5. Execute the plan (Strategize + Execute phases)
agents plan execute 01HXYZ1234567890ABCDEFGH
# 6. Review the generated changes
agents plan diff 01HXYZ1234567890ABCDEFGH
# 7. Apply the changes to the real repository
agents plan apply 01HXYZ1234567890ABCDEFGH
```
---
## Related
- [Plan CLI Reference](../reference/plan_cli.md)
- [Action CLI Reference](../reference/action_cli.md)
- [Resource CLI Reference](../reference/resource_cli.md)
- [Git Worktree Sandbox](git-worktree-sandbox.md)
- [ADR-006 Plan Lifecycle](../adr/ADR-006-plan-lifecycle.md)
- [ADR-008 Resource System](../adr/ADR-008-resource-system.md)
- [ADR-009 Project Model](../adr/ADR-009-project-model.md)
- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md)
- [ADR-019 Storage & Persistence](../adr/ADR-019-storage-and-persistence.md)
@@ -0,0 +1,305 @@
# v3.1.0 — M2: Actor Compiler + Full LLM Integration
**Milestone:** v3.1.0 — M2 (CLOSED)
**Released:** 2025-Q4
This milestone made Actor YAML files fully operational. Custom actors compile
into live LangGraph graphs, the tool router normalizes calls across providers,
the validation runner enforces resource-attached validations, and the MCP
adapter connects to external tool servers.
---
## Overview
M2 built on the M1 plan lifecycle by introducing a full actor compilation
pipeline and LLM integration layer:
```
actor.yaml (version: "3")
ActorLoader (parse + validate via Pydantic)
ActorCompiler (GRAPH type → LangGraph StateGraph)
LangGraph Runtime
├── ToolCallRouter (OpenAI / Anthropic / LangChain normalization)
├── ValidationRunner (required + informational validations)
└── MCPToolAdapter (external tool servers)
```
---
## Actor YAML Schema (version: "3")
Actor YAML files use `version: "3"` and support three actor types:
| Type | Description |
|------|-------------|
| `llm` | Single LLM invocation node |
| `tool` | Tool execution node |
| `graph` | Multi-node LangGraph workflow |
**Minimal LLM actor:**
```yaml
version: "3"
name: local/my-llm-actor
type: llm
description: A simple LLM actor
model: gpt-4
```
**Minimal GRAPH actor:**
```yaml
version: "3"
name: local/my-pipeline
type: graph
description: A multi-node pipeline
model: gpt-4
route:
nodes:
- id: planner
type: agent
name: Planner
description: Plans the work
- id: executor
type: agent
name: Executor
description: Executes the plan
edges:
- from_node: planner
to_node: executor
entry_node: planner
exit_nodes:
- executor
```
See [Actor Configuration Reference](../reference/actor_config.md) and
[Actor YAML Schema](../reference/actors_schema.md) for the full field listing.
---
## `agents actor add --config actor.yaml`
Load and register a custom actor from a YAML file.
```bash
agents actor add local/my-actor --config ./actors/my-actor.yaml
```
The loader:
1. Parses the YAML file.
2. Validates the schema via Pydantic (all fields, types, and constraints).
3. For GRAPH actors, validates the route topology (no cycles, reachable nodes,
valid entry/exit points).
4. Persists the actor to the registry.
Use `--update` to replace an existing actor, `--unsafe` to register actors
marked as unsafe, and `--set-default` to make the actor the default.
See [Actor CLI Reference](../reference/actor_cli.md) for all options.
---
## Actor Compiler: GRAPH → LangGraph
GRAPH-type actors compile into LangGraph `StateGraph` structures via the
`ActorCompiler`. The compilation pipeline:
1. **Input validation** — Accepts `ActorConfigSchema` with `type=GRAPH`.
2. **Reference validation** — All node IDs in edges, entry, and exit points
are checked against the declared node set.
3. **Cycle detection** — Intra-graph and cross-actor subgraph cycles are
detected and rejected.
4. **Node mapping** — Each `NodeDefinition` maps to a LangGraph `NodeConfig`.
5. **Edge mapping** — Each `EdgeDefinition` maps to a LangGraph `Edge`.
6. **LSP binding extraction** — Per-node LSP bindings are extracted into
`LspBinding` objects.
7. **Metadata assembly** — Returns a `CompiledActor` with nodes, edges, and
`CompilationMetadata`.
**Node type mapping:**
| Actor Node Type | LangGraph NodeType |
|---|---|
| `agent` | `AGENT` |
| `tool` | `TOOL` |
| `conditional` | `CONDITIONAL` |
| `subgraph` | `SUBGRAPH` |
See [Actor Compiler Reference](../reference/actor_compiler.md) for the full
API reference.
---
## Skill Registry and Tool Lifecycle
Skills provide named collections of tools that actors can use. The skill
registry manages the full lifecycle:
```bash
# Add a skill
agents skill add local/my-skill --config skill.yaml
# List registered skills
agents skill list
# Show skill details
agents skill show local/my-skill
# Remove a skill
agents skill remove local/my-skill
```
Tools within skills are registered in the `ToolRegistry` and made available
to actors during plan execution. See [Skill CLI Reference](../reference/skill_cli.md)
for all commands.
---
## MCP Adapter
The `MCPToolAdapter` connects to external MCP (Model Context Protocol) tool
servers and registers their tools in the `ToolRegistry`.
**Server configuration:**
```yaml
name: local/my-mcp-server
transport: stdio
command: npx
args: ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/dir"]
```
**Lifecycle:**
```python
adapter = MCPToolAdapter(config)
adapter.connect()
tools = adapter.discover_tools()
adapter.register_tools(registry, namespace="mcp")
# ... use tools in plan execution ...
adapter.disconnect()
```
Supported transports: `stdio` (child process), `sse`, `streamable-http`.
See [MCP Tool Adapter Reference](../reference/mcp_adapter.md) for the full
API reference.
---
## Tool Call Router
The `ToolCallRouter` normalizes tool calls across LLM providers. It accepts
OpenAI, Anthropic, and LangChain tool call formats and produces normalized
`NormalizedToolCallResult` objects.
**Provider format mapping:**
| Provider | Arguments key | Arguments type |
|----------|---------------|----------------|
| OpenAI | `arguments` | JSON string |
| Anthropic | `input` | dict |
| LangChain | `args` | dict |
The router automatically detects the provider format and normalizes the call
before dispatching to the `ToolRunner`.
See [Tool Call Router Reference](../reference/tool_router.md) for the full
API reference.
---
## Validation Runner
The validation runner executes resource-attached validations before the
apply gate. Validations are classified as:
- **Required** — must pass for `plan apply` to proceed.
- **Informational** — run and reported but do not block apply.
The `ApplyValidationSummary.all_required_passed` property returns `False`
when zero validations were run (empty-run guard), ensuring apply is always
blocked unless at least one validation was executed.
---
## Multi-File Generation and ChangeSet
The execution actor generates a `ChangeSet` containing multiple file
operations. M2 ensures that multi-file generation produces a correct
`ChangeSet` with:
- `added_files` — new files created by the actor.
- `modified_files` — existing files changed by the actor.
- `deleted_files` — files removed by the actor.
The ChangeSet is written to the git worktree sandbox and can be reviewed
with `agents plan diff` before applying.
---
## End-to-End Example with Custom Actor
```bash
# 1. Write an actor YAML file
cat > my-actor.yaml << 'EOF'
version: "3"
name: local/code-reviewer
type: graph
description: Multi-step code review pipeline
model: gpt-4
route:
nodes:
- id: analyzer
type: agent
name: Code Analyzer
description: Analyzes code quality
- id: reporter
type: agent
name: Report Writer
description: Writes the review report
edges:
- from_node: analyzer
to_node: reporter
entry_node: analyzer
exit_nodes:
- reporter
EOF
# 2. Register the actor
agents actor add local/code-reviewer --config my-actor.yaml
# 3. Create an action using the custom actor
agents action create --config action.yaml # references local/code-reviewer
# 4. Run a plan using the custom actor
agents plan use local/review-action local/my-project
agents plan execute 01HXYZ1234567890ABCDEFGH
agents plan diff 01HXYZ1234567890ABCDEFGH
agents plan apply 01HXYZ1234567890ABCDEFGH
```
---
## Related
- [Actor CLI Reference](../reference/actor_cli.md)
- [Actor Configuration Reference](../reference/actor_config.md)
- [Actor YAML Schema](../reference/actors_schema.md)
- [Actor Compiler Reference](../reference/actor_compiler.md)
- [MCP Tool Adapter Reference](../reference/mcp_adapter.md)
- [Tool Call Router Reference](../reference/tool_router.md)
- [Skill CLI Reference](../reference/skill_cli.md)
- [ADR-010 Actor & Agent Architecture](../adr/ADR-010-actor-and-agent-architecture.md)
- [ADR-011 Tool System](../adr/ADR-011-tool-system.md)
- [ADR-022 LangChain/LangGraph Integration](../adr/ADR-022-langchain-langgraph-integration.md)
- [ADR-029 Model Context Protocol (MCP)](../adr/ADR-029-model-context-protocol.md)
- [ADR-031 Actor Abstraction Definition](../adr/ADR-031-actor-abstraction-definition.md)
+2
View File
@@ -30,6 +30,8 @@ nav:
- Invariant Reconciliation: modules/invariant-reconciliation.md
- ACMS Context Hydration: modules/context-hydration.md
- Git Worktree Sandbox: modules/git-worktree-sandbox.md
- M1: Minimal Local Workflow: modules/milestone-v3.0.0-local-workflow.md
- M2: Actor Compiler + LLM Integration: modules/milestone-v3.1.0-actor-compiler.md
- Development:
- Agent System Specification: development/agent-system-specification.md
- CI/CD Pipeline: development/ci-cd.md