diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 60ee6fb1d..31ad773dd 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -21,3 +21,4 @@ Below are some of the specific details of various contributions. * HAL 9000 has contributed the plugin entry point security hardening fix (#7476): enforced entry point allowlist validation before importing plugin modules to prevent malicious plugin loading. * This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc. * HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system. +* HAL 9000 has contributed initial documentation for v3.0.0 (Minimal Local Source-Code Workflow) and v3.1.0 (Actor Compiler + Full LLM Integration) milestones, including CHANGELOG entries, CLI reference, architecture overview, and configuration YAML schema reference. [AUTO-DOCS-1] diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 000000000..d86d97c76 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,101 @@ +# CleverAgents Architecture + +CleverAgents uses a layered architecture for autonomous code modification. This document +provides a high-level overview. For detailed design decisions, see `docs/adr/` (48 ADRs) +and `docs/architecture.md`. + +## Core Layers + +### Domain Layer + +- **Pydantic v2 domain models** with `frozen=True` for immutability. All models inherit + from `cleveragents.domain.DomainBaseModel` which provides shared `model_config`. +- **SQLite persistence** via SQLAlchemy ORM and Alembic migrations + (`alembic/versions/`). Default database: `~/.cleveragents/cleveragents.db`. +- **Key entities**: `Action`, `Resource`, `Project`, `Plan`, `Actor`, `Decision`, + `ChangeSet`, `Session`, `Invariant`, `Skill`, `Tool`. +- **Repository pattern**: Each entity has a corresponding repository class (e.g., + `cleveragents.domain.ActionRepository`) providing CRUD operations. + +### Actor Layer + +- **Actor YAML v3 schema** (`version: "3"`, `type: llm|tool|graph`). Defined in + `cleveragents.actor.config.ActorConfigSchema`. +- **LangGraph StateGraph compilation**: `cleveragents.actor.compiler.ActorCompiler` + translates YAML actor definitions into executable LangGraph `StateGraph` instances. +- **Subgraph resolution**, cycle detection, entry/exit validation. +- **Actor types**: + - `llm` — LLM-backed actor with model, system prompt, and tool bindings. + - `tool` — Pure tool executor without LLM. + - `graph` — Composite actor composed of sub-actors connected by edges. +- **Actor registry** (`cleveragents.actor.registry.ActorRegistry`): Persists and + resolves actors by namespace/name. + +### Execution Layer + +- **Actor-based LLM execution path**: `cleveragents.plan.executor.PlanExecutor` + orchestrates the Strategize → Execute → Apply lifecycle. +- **Git worktree sandbox** (`cleveragents.sandbox.GitWorktreeSandbox`): Isolated + working directory using `git worktree add` so LLM-generated changes do not affect + the original repository until `plan apply` is called. +- **ChangeSet** (`cleveragents.changeset.ChangeSet`): Built from tool invocations + (not parsed from LLM output) to ensure structural correctness. +- **MCP adapter** (`cleveragents.mcp.MCPToolAdapter`): External tool server + connectivity via the Model Context Protocol. + +### ACMS Layer (Adaptive Context Management System) + +- **UKO** (Universal Knowledge Ontology): Four-layer ontology hierarchy for + structured knowledge representation. Defined in `docs/ontology/uko.ttl`. +- **CRP** (Context Retrieval Pipeline): Six context strategies for assembling + relevant context for LLM execution. Documented in `docs/reference/crp.md`. +- **Context tiers**: Hot/warm/cold tier service (`cleveragents.acms.ContextTierService`) + with LRU eviction, staleness enforcement, and actor-scoped views. +- **Skeleton compressor** (`cleveragents.acms.SkeletonCompressor`): Reduces large + codebases to structural skeletons for context budget management. + +### Plan Lifecycle + +The plan lifecycle follows a state machine with four primary phases: + +1. **Use** (`agents plan use`) — Create plan record, associate with action, initialize + state machine. Defined in `cleveragents.cli.plan.use_plan`. +2. **Execute** (`agents plan execute `) — Invoke actor-based LLM path. + Strategize phase builds decision tree; Execute phase produces ChangeSet via tool + invocations. Defined in `cleveragents.cli.plan.execute_plan`. +3. **Diff** (`agents plan diff `) — Show pending changes in sandbox without + modifying the target repository. Defined in `cleveragents.cli.plan.diff_plan`. +4. **Apply** (`agents plan apply `) — Merge sandbox branch into target + repository via `git merge` with a structured commit. Defined in + `cleveragents.cli.plan.apply_plan`. + +### Validation Layer + +- **Validation runner** (`cleveragents.validation.ValidationRunner`): Executes + required and informational validations. Required validations block Apply if they fail. +- **Fix-then-revalidate loop**: Failed required validations trigger a correction cycle + before re-running the validation. +- **Invariant reconciliation** (`cleveragents.invariant.InvariantReconciliationActor`): + Runs automatically at every plan phase transition; failures block the transition and + emit `INVARIANT_VIOLATED` events. + +## Key Design Decisions + +- **Changes in sandbox do not affect original until Apply**: The git worktree sandbox + provides complete isolation. See ADR-015 (`docs/adr/ADR-015-sandbox-and-checkpoint.md`). +- **ChangeSet built from tool invocations**: Structural correctness is guaranteed by + building ChangeSets from tool call results rather than parsing LLM text output. + See ADR-007 (`docs/adr/ADR-007-decision-tree-and-correction.md`). +- **Pydantic v2 with `frozen=True`**: Domain models are immutable to prevent accidental + mutation. See ADR-004 (`docs/adr/ADR-004-data-validation.md`). +- **Layered architecture**: CLI → Service → Domain → Persistence separation of concerns. + See ADR-001 (`docs/adr/ADR-001-layered-architecture.md`). +- **LangGraph for actor compilation**: StateGraph provides deterministic execution flow + with cycle detection. See ADR-022 (`docs/adr/ADR-022-langchain-langgraph-integration.md`). + +## Further Reading + +- Architecture Decision Records: `docs/adr/` (48 ADRs covering all major design choices) +- Full architecture specification: `docs/architecture.md` +- Module guides: `docs/modules/` +- API reference: `docs/api/` diff --git a/docs/cli/README.md b/docs/cli/README.md new file mode 100644 index 000000000..87491e21c --- /dev/null +++ b/docs/cli/README.md @@ -0,0 +1,123 @@ +# CleverAgents CLI Reference + +The `agents` CLI provides commands for managing actions, resources, projects, plans, +actors, skills, invariants, context, sessions, and more. + +## Commands + +### Action Commands + +- `agents action create --config ` — Create an action from a YAML configuration + file and persist it to SQLite. Defined in `cleveragents.cli.action.create_action`. + +### Resource Commands + +- `agents resource add git-checkout` — Register a git-checkout resource linking a + local repository path to a project. Defined in `cleveragents.cli.resource.add_resource`. +- `agents resource add container-instance [--container-id ]` — Register a + container-instance resource. +- `agents resource list` — List all registered resources. +- `agents resource stop` — Stop a running resource (supports `container-instance` and + `devcontainer-instance` types). + +### Project Commands + +- `agents project create` — Create a new project record with namespace/name identity. + Defined in `cleveragents.cli.project.create_project`. +- `agents project link-resource` — Link a registered resource to a project. Defined in + `cleveragents.cli.project.link_resource`. + +### Plan Commands + +- `agents plan use` — Create a new plan record with state machine transitions and + associate it with an action. Defined in `cleveragents.cli.plan.use_plan`. +- `agents plan execute ` — Execute a plan via the actor-based LLM path, + producing a ChangeSet from tool invocations. Defined in + `cleveragents.cli.plan.execute_plan`. +- `agents plan diff ` — Show pending changes in the git worktree sandbox + before applying. Defined in `cleveragents.cli.plan.diff_plan`. +- `agents plan apply ` — Apply sandbox changes to the target repository with + a git commit. Defined in `cleveragents.cli.plan.apply_plan`. +- `agents plan list` — List all plans with status and timestamps. +- `agents plan tree ` — Display the decision tree for a plan (full 26-char + ULIDs for correction workflows). +- `agents plan correct ` — Correct a decision in the plan decision tree. +- `agents plan rollback ` — Roll back a plan to a previous checkpoint. +- `agents plan errors ` — Show error details for a failed plan. + +### Actor Commands + +- `agents actor add --config ` — Load and register an actor from a YAML + configuration file. Defined in `cleveragents.cli.actor.add_actor`. +- `agents actor list` — List all registered actors. +- `agents actor remove ` — Remove an actor from the registry. +- `agents actor set-default ` — Set the default actor for CLI commands. +- `agents actor run ` — Run an actor directly with a prompt. + +### Skill Commands + +- `agents skill list` — List all registered skills. +- `agents skill refresh` — Refresh skill tool bindings from MCP servers. + +### Invariant Commands + +- `agents invariant add` — Add an invariant to a project or plan. +- `agents invariant list` — List all invariants. + +### Context Commands + +- `agents context list` — List context fragments in the active tier service. +- `agents context add` — Add a context fragment. +- `agents context show` — Show details of a context fragment. +- `agents context clear` — Clear all context fragments. + +### Session Commands + +- `agents session create --actor ` — Create a new conversation session. +- `agents session list` — List all sessions. +- `agents session show --session-id ` — Show session details. +- `agents session export --session-id --output ` — Export a session to JSON. +- `agents session import --input ` — Import a session from JSON. +- `agents session delete --session-id ` — Delete a session. + +### Validation Commands + +- `agents validation add` — Attach a validation to an action. +- `agents validation list` — List validations for an action. + +### Tool Commands + +- `agents tool add --config ` — Register a tool from YAML configuration. +- `agents tool list` — List all registered tools. + +### Config Commands + +- `agents config get ` — Get a configuration value. +- `agents config set ` — Set a configuration value. + +### Diagnostics Commands + +- `agents diagnostics` — Check provider credentials and actor/provider selection. + +### TUI Command + +- `agents tui` — Launch the interactive Textual terminal UI. + +### Server Commands + +- `agents server connect --url --token ` — Connect to a remote + CleverAgents server. +- `agents server status` — Check server connection status. + +## Global Options + +- `--format ` — Output format (default: `rich`). +- `--version` — Show version and git commit. +- `--help` — Show help for any command. + +## See Also + +- Detailed CLI references: `docs/reference/action_cli.md`, `docs/reference/actor_cli.md`, + `docs/reference/plan_cli.md`, `docs/reference/resource_cli.md`, + `docs/reference/session_cli.md`, `docs/reference/skill_cli.md` +- Showcase examples: `docs/showcase/cli-tools/` diff --git a/docs/configuration/README.md b/docs/configuration/README.md new file mode 100644 index 000000000..61d418b5f --- /dev/null +++ b/docs/configuration/README.md @@ -0,0 +1,178 @@ +# Configuration YAML Reference + +CleverAgents uses YAML configuration files for defining actions, actors, skills, tools, +and validation rules. This document covers the primary configuration schemas. + +## Action Configuration (`action.yaml`) + +Actions define the behavior, actor, and invariants for plan execution. + +```yaml +# action.yaml example +name: my-action # Required: namespace/name or plain name +description: "Refactor the auth module" +actor: local/my-actor # Required: actor name to use for execution +automation_profile: supervised # Optional: manual|supervised|review|ci|trusted +arguments: + - key: target_file + value: src/auth.py +invariants: + - name: tests-pass + description: "All tests must pass after changes" + required: true +``` + +Schema defined in `docs/schema/action.schema.yaml`. + +## Actor Configuration (`actor.yaml`) + +Actors define the LLM/tool/graph behavior for plan execution. All actors use +`version: "3"` and specify a `type`. + +### LLM Actor + +```yaml +# actor.yaml — LLM actor example +version: "3" +type: llm +name: local/my-llm-actor +model: gpt-4o # Provider/model identifier +system_prompt: | + You are a helpful coding assistant. Analyze the codebase and make targeted changes. +tools: [] # Optional: list of tool names to bind +``` + +### Tool Actor + +```yaml +# actor.yaml — Tool actor example +version: "3" +type: tool +name: local/my-tool-actor +tool: my-namespace/my-tool # Required: tool to execute +``` + +### Graph Actor + +```yaml +# actor.yaml — Graph actor example +version: "3" +type: graph +name: local/my-graph-actor +entry: strategize # Required: entry node name +nodes: + strategize: + type: llm + model: gpt-4o + system_prompt: "Analyze the task and create a plan." + execute: + type: llm + model: gpt-4o + system_prompt: "Execute the plan step by step." +edges: + - from: strategize + to: execute + - from: execute + to: __end__ +``` + +Schema defined in `docs/reference/actors_schema.md` and `docs/reference/actor_config.md`. + +## Skill Configuration (`skill.yaml`) + +Skills define reusable tool bundles that can be loaded into actors. + +```yaml +# skill.yaml example +version: "1" +name: my-namespace/my-skill +description: "File editing and search tools" +tools: + - name: read_file + description: "Read a file from the filesystem" + - name: write_file + description: "Write content to a file" + - name: search_files + description: "Search for patterns in files" +``` + +Schema defined in `docs/schema/skill.schema.yaml`. + +## Tool Configuration (`tool.yaml`) + +Tools define individual callable functions available to actors. + +```yaml +# tool.yaml example +tool: + name: my-namespace/my-tool + description: "Execute a shell command safely" + parameters: + - name: command + type: string + description: "The command to execute" + required: true + execution_env: sandbox # Optional: sandbox|local|container +``` + +Schema defined in `docs/schema/tool.schema.yaml`. + +## Validation Configuration (`validation.yaml`) + +Validations define checks that must pass before a plan can be applied. + +```yaml +# validation.yaml example +name: my-namespace/run-tests +description: "Run the test suite" +type: required # required|informational +command: "pytest tests/ -x" +timeout: 300 +``` + +Schema defined in `docs/schema/validation.schema.yaml`. + +## Automation Profile Configuration + +Automation profiles control how much human oversight is required during plan execution. + +| Profile | Description | +|---------|-------------| +| `manual` | All steps require explicit human approval | +| `supervised` | Pauses at confidence thresholds for human review | +| `review` | Runs autonomously but requires review before apply | +| `ci` | Fully automated for CI/CD pipelines | +| `trusted` | Maximum autonomy, minimal interruptions | + +Profiles are configured via `agents config set actor.default.automation_profile ` +or per-action in `action.yaml`. + +## Global Settings + +CleverAgents settings are stored in `~/.config/cleveragents/settings.yaml` and can be +managed via `agents config`: + +```bash +agents config set actor.default.strategy local/my-strategy-actor +agents config set actor.default.estimation local/my-estimation-actor +agents config set server_url https://my-server.example.com +agents config set format rich +``` + +Key settings defined in `cleveragents.config.Settings`: + +| Key | Description | +|-----|-------------| +| `actor.default.strategy` | Default strategy actor for plan execution | +| `actor.default.estimation` | Default estimation actor | +| `actor.default.automation_profile` | Default automation profile | +| `server_url` | Remote CleverAgents server URL | +| `server_token` | Authentication token for server mode | +| `format` | Default output format (`plain`, `json`, `yaml`, `rich`, `color`) | + +## See Also + +- Actor schema reference: `docs/reference/actors_schema.md` +- Actor configuration guide: `docs/reference/actor_config.md` +- Automation profiles: `docs/reference/automation_profiles.md` +- Config resolution: `docs/reference/config_resolution.md`