diff --git a/docs/specification.md b/docs/specification.md index f0803345d..3c71f6d15 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -309,6 +309,11 @@ The following standards are integrated into the architecture: agents resource type list [<REGEX>] agents resource type show <NAME> +agents provider add --config|-c <FILE> [--update] +agents provider remove [--yes|-y] <NAME> +agents provider list [--all] [--namespace|-n <NS>] +agents provider show <NAME> + agents resource add [(--description|-d) <DESC>] [--update] <TYPE> <NAME> [type-specific-flags...] agents resource remove [--yes|-y] <NAME> agents resource list [--all] [(--type|-t) <TYPE>] @@ -345,6 +350,7 @@ The following standards are integrated into the architecture: agents plan prompt <PLAN_ID> <GUIDANCE> agents plan rollback [--yes|-y] <PLAN_ID> <CHECKPOINT_ID> agents plan errors <PLAN_ID> +agents plan guard <PLAN_ID> agents action create --config|-c <CFG_FILE> agents action list [(--namespace|-n) <NS>] [(--state|-s) <STATE>] [<REGEX>] @@ -365,6 +371,9 @@ The following standards are integrated into the architecture: agents invariant list [--global] [(--project|-p) PROJECT] [--plan PLAN_ID] [--action ACTION] [--effective] [<REGEX>] agents invariant remove [--yes|-y] <INVARIANT_ID> + +agents tui [--session <SESSION_ID>] [--persona <PERSONA>] +agents tui web [--port <PORT>] [--host <HOST>] ### Command Reference @@ -10558,6 +10567,72 @@ Show detailed information about a resource type, including its full schema. - "Resource type details loaded" ``` +#### agents provider + +**Purpose** +Manage LLM provider registrations. Providers encapsulate LLM SDK integrations and are referenced by actors via the `provider/model` notation. Commands in this group operate on provider definitions stored in YAML configuration files or discovered via entry points. + +##### agents provider add + +
agents provider add --config|-c <FILE> [--update]
+ +**Purpose** +Register or update a provider definition. The provider becomes available immediately for actor references (e.g., `local/my-provider/my-model`). + +**Arguments** + +- `--config/-c FILE`: Path to the provider YAML file (required). The file must declare the provider's `name`, implementation module/class, and any required environment variables. +- `--update`: If the provider already exists, replace the registration in place. Without this flag, attempting to register an existing provider fails. + +**Examples** + +=== "Rich" + + 

+    $ agents provider add --config providers/my-custom-provider.yaml
+
+    ╭─ Provider Registered ─────────────────────────────╮
+    │ Name: local/my-llm-provider                    │
+    │ Module: my_package.providers.my_llm         │
+    │ Class: MyLLMProvider                        │
+    │ Models: my-model-v1, my-model-v2            │
+    ╰────────────────────────────────────────────────────╯
+
+    ✓ OK Provider local/my-llm-provider registered
+
+ +=== "Plain" + + ``` + $ agents provider add --config providers/my-custom-provider.yaml + + Provider Registered + Name: local/my-llm-provider + Module: my_package.providers.my_llm + Class: MyLLMProvider + Models: my-model-v1, my-model-v2 + + [OK] Provider local/my-llm-provider registered + ``` + +##### agents provider list + +
agents provider list [--all] [(--namespace|-n) <NS>]
+ +List registered providers. By default only user-defined providers are shown; pass `--all` to include built-ins. `--namespace` filters by namespace prefix. + +##### agents provider show + +
agents provider show <NAME>
+ +Display the full provider definition, including environment requirements and supported models. + +##### agents provider remove + +
agents provider remove [--yes|-y] <NAME>
+ +Remove a provider registration. Built-in providers cannot be removed. + #### agents resource !!! info "Purpose" @@ -16210,6 +16285,121 @@ Provide additional guidance to a plan, typically when it is errored or awaiting - "Rollback complete" ``` +##### agents plan guard + +
agents plan guard <PLAN_ID>
+ +**Purpose** +Show the active guards for a plan — the denylist entries, budget caps, and tool call limits that are currently enforced. Guards are derived from the plan's resolved automation profile and any plan-level overrides. + +**Arguments** + +- ``: Plan ID. + +**Guard Evaluation Order** + +Guards are evaluated in this order (first match wins for denylist; all limits apply independently for budget/tool caps): + +1. **Denylist** — Tool names or patterns that are explicitly forbidden. Checked first as a fast-reject before any budget or limit evaluation. +2. **Budget caps** — `max_cost_per_plan` (USD), `max_tokens_per_plan`, `max_wall_clock_seconds`. Checked before each tool invocation. +3. **Tool call limits** — `max_tool_calls_per_actor`, `max_retries_per_tool`. Checked per-actor per-invocation. + +**Examples** + +=== "Rich" + + 

+    $ agents plan guard 01HXM8C2ZK4Q7C2B3F2R4VYV6J
+
+    ╭─ Active Guards — Plan 01HXM8C2ZK4Q7C2B3F2R4VYV6J ──────────────────────────────────────╮
+    │ Profile: trusted (resolved from action default)                                        │
+    ╰──────────────────────────────────────────────────────────────────────────────────────────╯
+
+    ╭─ Denylist ─────────────────────────────────────────────────────────────────────────────╮
+    │ Pattern                    Source                                                      │
+    │ ──────────────────────────  ──────────────────────────────────────────────────────     │
+    │ local/shell-exec           automation profile: trusted                                 │
+    │ local/network-*            automation profile: trusted                                 │
+    ╰────────────────────────────────────────────────────────────────────────────────────────╯
+
+    ╭─ Budget Caps ──────────────────────────────────────────────────────────────────────────╮
+    │ Limit                      Value          Current       Status                         │
+    │ ──────────────────────────  ─────────────  ─────────────  ──────────────────────     │
+    │ max_cost_per_plan          $5.00          $1.23          OK                          │
+    │ max_tokens_per_plan        500,000        187,432        OK                          │
+    │ max_wall_clock_seconds     3,600          412            OK                          │
+    ╰────────────────────────────────────────────────────────────────────────────────────────╯
+
+    ╭─ Tool Call Limits ─────────────────────────────────────────────────────────────────────╮
+    │ Limit                      Value          Current       Status                         │
+    │ ──────────────────────────  ─────────────  ─────────────  ──────────────────────     │
+    │ max_tool_calls_per_actor   200            47             OK                          │
+    │ max_retries_per_tool       3              (per-tool)     OK                          │
+    ╰────────────────────────────────────────────────────────────────────────────────────────╯
+
+    ✓ OK 2 denylist entries, 3 budget caps, 2 tool call limits active
+
+ +=== "Plain" + + ``` + $ agents plan guard 01HXM8C2ZK4Q7C2B3F2R4VYV6J + + Active Guards — Plan 01HXM8C2ZK4Q7C2B3F2R4VYV6J + Profile: trusted (resolved from action default) + + Denylist + Pattern Source + ------------------------- ------------------------------------------------ + local/shell-exec automation profile: trusted + local/network-* automation profile: trusted + + Budget Caps + Limit Value Current Status + ------------------------- ------------- ------------- -------------------- + max_cost_per_plan $5.00 $1.23 OK + max_tokens_per_plan 500,000 187,432 OK + max_wall_clock_seconds 3,600 412 OK + + Tool Call Limits + Limit Value Current Status + ------------------------- ------------- ------------- -------------------- + max_tool_calls_per_actor 200 47 OK + max_retries_per_tool 3 (per-tool) OK + + [OK] 2 denylist entries, 3 budget caps, 2 tool call limits active + ``` + +=== "JSON" + + ```json + { + "command": "plan guard", + "status": "ok", + "exit_code": 0, + "data": { + "plan_id": "01HXM8C2ZK4Q7C2B3F2R4VYV6J", + "resolved_profile": "trusted", + "profile_source": "action default", + "denylist": [ + { "pattern": "local/shell-exec", "source": "automation profile: trusted" }, + { "pattern": "local/network-*", "source": "automation profile: trusted" } + ], + "budget_caps": [ + { "limit": "max_cost_per_plan", "value": 5.00, "current": 1.23, "unit": "USD", "status": "ok" }, + { "limit": "max_tokens_per_plan", "value": 500000, "current": 187432, "unit": "tokens", "status": "ok" }, + { "limit": "max_wall_clock_seconds", "value": 3600, "current": 412, "unit": "seconds", "status": "ok" } + ], + "tool_call_limits": [ + { "limit": "max_tool_calls_per_actor", "value": 200, "current": 47, "status": "ok" }, + { "limit": "max_retries_per_tool", "value": 3, "current": null, "status": "ok" } + ] + }, + "timing": { "started": "2025-06-15T10:38:00Z", "duration_ms": 45 }, + "messages": ["2 denylist entries, 3 budget caps, 2 tool call limits active"] + } + ``` + #### agents action !!! info "Purpose" @@ -18276,6 +18466,95 @@ List invariants at a given scope. Use `--effective` with `--plan` to show the fi - "Invariant removed" ``` +#### agents tui + +!!! info "Purpose" + Launch the ==Text User Interface== (TUI) — a Textual-based interactive terminal application for real-time plan monitoring, multi-session management, and rich actor conversation. The TUI is the second Presentation-layer surface alongside the CLI. + +!!! adr "Architecture Decision" + TUI architecture, framework selection (Textual >= 1.0), screen hierarchy, TuiMaterializer integration, and A2A event subscription model are defined in [ADR-044: TUI Architecture and Framework](adr/ADR-044-tui-architecture-and-framework.md). + +##### agents tui + +
agents tui [--session <SESSION_ID>] [--persona <PERSONA>]
+ +**Purpose** +Launch the Textual-based TUI. Opens directly to the MainScreen (no launcher). If `--session` is provided, resumes that session; otherwise opens the most recent session or creates a new one. + +**Arguments** + +- `--session SESSION_ID`: Resume a specific session by ID. +- `--persona PERSONA`: Activate a named persona on launch (overrides session default). + +**Behavior** + +- Opens `~/.local/state/cleveragents/tui.db` for session persistence. +- Subscribes to A2A events via `A2aLocalFacade` for real-time plan updates. +- All CLI operations are available via `TuiMaterializer`; no TUI-exclusive features. +- Exit with `ctrl+q` or `ctrl+c`. + +**Examples** + +=== "Rich" + + 

+    $ agents tui
+
+    [TUI launches — terminal switches to Textual application]
+
+ +=== "Plain" + + ``` + $ agents tui + + [TUI launches — terminal switches to Textual application] + ``` + +##### agents tui web + +
agents tui web [--port <PORT>] [--host <HOST>]
+ +**Purpose** +Launch the TUI in Textual Web mode, serving the same Textual widget tree over HTTP so it can be accessed from a browser. Uses Textual's built-in web server. + +**Arguments** + +- `--port PORT`: Port to listen on (default: `8000`). +- `--host HOST`: Host to bind to (default: `127.0.0.1`). + +**Behavior** + +- Starts a Textual Web server; the TUI is accessible at `http://:/`. +- The same widget tree, session persistence, and A2A bindings as the terminal TUI. +- Useful for remote access, IDE plugin integration, and headless environments. +- Exit with `ctrl+c` in the terminal. + +**Examples** + +=== "Rich" + + 

+    $ agents tui web --port 8080
+
+    ╭─ TUI Web Mode ─────────────────────────────────────────────────────────────────────────╮
+    │ URL: http://127.0.0.1:8080/                                                            │
+    │ Status: Listening                                                                       │
+    │ Press ctrl+c to stop                                                                    │
+    ╰────────────────────────────────────────────────────────────────────────────────────────╯
+
+ +=== "Plain" + + ``` + $ agents tui web --port 8080 + + TUI Web Mode + URL: http://127.0.0.1:8080/ + Status: Listening + Press ctrl+c to stop + ``` + ## Core Concepts ### Plan @@ -18353,6 +18632,8 @@ A plan's ==phase== indicates "what step of the lifecycle it is in." Separately, | :---- | :-------: | :---------- | | `queued` | No | Waiting for compute/worker | | `processing` | No | Currently running | + | `awaiting_approval` | No | Paused pending human approval due to guard budget caps or automation profile thresholds | + | `blocked` | No | Guard violation or safety constraint prevented progress; requires user intervention or correction | | `errored` | Yes | Failed; includes error metadata | | `complete` | Yes | Finished successfully | | `cancelled` | Yes | User/system cancelled; safe terminal | @@ -18363,11 +18644,15 @@ A plan's ==phase== indicates "what step of the lifecycle it is in." Separately, | :---- | :-------: | :---------- | | `queued` | No | Waiting for compute/worker | | `processing` | No | Currently running — diff review, conflict resolution, validation | + | `awaiting_merge_resolution` | No | Merge conflicts detected; waiting for user to resolve sandbox conflicts | | `errored` | Yes | Failed; includes error metadata | | `applied` | Yes | ==Changes successfully committed== to real resources | | `constrained` | Yes | Cannot complete within current strategy's constraints; may trigger reversion to Strategize | | `cancelled` | Yes | User/system cancelled; safe terminal | +!!! note "Guard-driven states" + Plans enter `awaiting_approval` when budget-related guards pause execution and resume once the user approves additional budget. Plans enter `blocked` when a denylist or retry guard prevents a tool invocation; the user must cancel, correct, or adjust the automation profile before the plan can continue. During Apply, merge conflicts transition the plan to `awaiting_merge_resolution`; completing conflict resolution and re-running `agents plan apply` returns the plan to `processing`. + #### Plan Identity and Traceability Every plan should have: @@ -18537,6 +18822,103 @@ The way child plan results are merged depends on the resource type: * **Other resources**: Pluggable merge strategies based on resource type * **Non-mergeable resources**: May require sequential execution only +The merge is performed by the `MergeWorkflow` component after all child plans in a `subplan_parallel_spawn` group have completed. See [Merge Strategies](#merge-strategies) for the full specification. + +#### Merge Strategies {#merge-strategies} + +When child plans complete, the parent plan merges their results using the `MergeWorkflow`. The merge strategy is configured per `SubplanConfig` via the `merge_strategy` field (default: `git_three_way`). + +##### Available Strategies + +| Strategy | `merge_strategy` value | Applicable Resource Types | Description | +|---|---|---|---| +| **Git Three-Way Merge** | `git_three_way` | `git-checkout`, `fs-mount` (text files) | Standard git three-way merge using the common ancestor (the parent sandbox state before child plans ran) as the base. Non-conflicting changes are merged automatically; conflicting hunks are surfaced to the user. | +| **Sequential Apply** | `sequential` | All resource types | Child plan results are applied one at a time in spawn order. No conflict detection — later results overwrite earlier ones for the same file/record. Use when ordering matters and conflicts are impossible by design. | +| **Last Write Wins** | `last_write_wins` | `fs-mount`, databases | The last child plan to complete for a given resource path/key wins. Useful for idempotent operations where all child plans produce equivalent results. | +| **Union** | `union` | `fs-mount` (append-only files), databases (insert-only) | All child plan results are combined without deduplication. Suitable for log files, event streams, or insert-only database tables. | +| **Custom** | `custom:` | Any | A fully-qualified Python callable (`module.path:function_name`) that implements the `MergeStrategy` protocol. Receives the parent sandbox state, list of child sandbox states, and `SubplanConfig`; returns the merged state. | + +##### Git Three-Way Merge Algorithm + +The `git_three_way` strategy is the default and most commonly used. The algorithm: + +1. **Identify the base**: The parent sandbox state at the moment the `subplan_parallel_spawn` decision was recorded. This is the common ancestor for all child plans. +2. **Collect diffs**: For each child plan, compute `diff(base, child_result)` — the set of changes the child plan made relative to the base. +3. **Detect conflicts**: A conflict occurs when two child plans modify the same file region (hunk) in incompatible ways. Specifically: + - Same file, overlapping line ranges, different content → **conflict** + - Same file, non-overlapping line ranges → **auto-merge** (both changes applied) + - Different files → **auto-merge** (independent changes) +4. **Apply non-conflicting changes**: All non-conflicting hunks are applied to the base to produce the merged result. +5. **Surface conflicts**: Conflicting hunks are written to the merged result using standard conflict markers: + ``` + <<<<<<< child_plan_01HXM8C2 (Convert auth module) + def authenticate(user, password): + return bcrypt.check(password, user.hash) + ======= + def authenticate(user: User, password: str) -> bool: + return argon2.verify(password, user.hash) + >>>>>>> child_plan_01HXM9A1 (Add type annotations) + ``` +6. **Report conflicts**: `agents plan status ` shows the number of conflicts. The plan enters `awaiting_merge_resolution` state. The user resolves conflicts manually (or via a correction) and re-triggers apply. + +##### MergeWorkflow Component + +The `MergeWorkflow` is an Application-layer component responsible for orchestrating the merge: + +```python +class MergeWorkflow: + """Orchestrates result merging for completed parallel subplan groups.""" + + def merge( + self, + parent_plan_id: str, + subplan_group_decision_id: str, + strategy: MergeStrategy, + config: SubplanConfig, + ) -> MergeResult: + """ + Merge results from all child plans in a parallel spawn group. + + Returns MergeResult with: + - merged_sandbox_state: The combined result + - conflicts: List[MergeConflict] — empty if no conflicts + - auto_merged_count: Number of hunks merged automatically + - conflict_count: Number of unresolved conflicts + """ + ... +``` + +**`MergeResult` data model:** + +| Field | Type | Description | +|---|---|---| +| `merged_sandbox_state` | `SandboxState` | The merged result (may contain conflict markers if `conflict_count > 0`) | +| `conflicts` | `list[MergeConflict]` | Unresolved conflicts requiring user intervention | +| `auto_merged_count` | `int` | Number of hunks merged automatically without conflict | +| `conflict_count` | `int` | Number of unresolved conflicts | +| `strategy_used` | `str` | The `merge_strategy` value that was applied | + +**`MergeConflict` data model:** + +| Field | Type | Description | +|---|---|---| +| `resource_path` | `str` | File path or resource key where the conflict occurred | +| `hunk_start` | `int` | Line number (1-indexed) where the conflict begins | +| `hunk_end` | `int` | Line number where the conflict ends | +| `ours` | `str` | Content from one child plan | +| `theirs` | `str` | Content from the other child plan | +| `base` | `str` | Content from the common ancestor (base) | +| `child_plan_ids` | `list[str]` | The two child plan IDs whose changes conflict | + +##### Conflict Resolution UX + +When a merge produces conflicts, the plan enters `awaiting_merge_resolution` state: + +- `agents plan status ` shows `State: awaiting_merge_resolution` and `Conflicts: N`. +- `agents plan diff ` shows the merged result with conflict markers highlighted. +- The user resolves conflicts by editing the files in the sandbox directly, then runs `agents plan apply ` to re-attempt apply with the resolved content. +- Alternatively, `agents plan correct --mode=revert` can be used to revert the conflicting child plan and re-run it with different guidance. + #### The Plan "Decision Tree" and Visualization !!! adr "Architecture Decision" @@ -28518,10 +28900,12 @@ A `SafetyProfile` may also be attached directly to an `Action` (via the `safety_ | `require_human_approval` | boolean | `false` | Require human approval before each action step | When `true`, every action step pauses for explicit human approval before execution. | | `allowed_skill_categories` | list[string] | `[]` (all) | Skill categories permitted for execution | When non-empty, only skills in the listed categories may be used. Empty list means all categories are allowed. | | `max_cost_per_plan` | float \| null | `null` | Maximum cost in USD per plan execution | When set, the plan is paused or terminated if the cost limit is reached. `null` means no limit. Must be <= `max_total_cost` when both are set. | +| `max_tokens_per_plan` | int \| null | `null` | Maximum total LLM tokens per plan execution | When set, the plan pauses in `awaiting_approval` once the cumulative token usage would exceed the limit. | +| `max_wall_clock_seconds` | int \| null | `null` | Maximum wall-clock time allowed for the plan | When set, exceeding the limit pauses the plan in `awaiting_approval` until the user approves extending the budget. | | `max_total_cost` | float \| null | `null` | Maximum total cost in USD across all plans | When set, execution is paused or terminated if the aggregate cost limit is reached. `null` means no limit. | -| `max_retries_per_step` | integer | `3` | Maximum retry attempts per action step | Limits the number of retries for a single step before escalating to the user. Range: 0–100. | +| `max_retries_per_step` | integer | `3` | Maximum retry attempts per tool invocation | Limits retries for a single tool before escalating to the user. Range: 0–100. | -**Relationship to Automation Guards**: The `SafetyProfile.max_total_cost` field sets a **plan-level** budget cap (broad scope), while `AutomationGuard.max_total_cost` sets a **per-invocation** budget cap (narrow scope). These operate at different granularities and both may be active simultaneously — the tighter constraint takes precedence at any given point. +**Relationship to Automation Guards**: The `SafetyProfile` budget fields (`max_cost_per_plan`, `max_tokens_per_plan`, `max_wall_clock_seconds`) set **plan-level** caps. The `AutomationGuard` sub-model can further constrain individual tool invocations (per-invocation cost caps, tool allowlists/denylists, retry limits). Both layers may be active simultaneously — the tightest constraint in effect takes precedence at any moment. #### Automation Guard Sub-Model @@ -28964,6 +29348,258 @@ agents plan diff --correction <correction_a This keeps history reproducible and prevents accidental destructive edits. +### Guard Enforcement {#guard-enforcement} + +Guards are the runtime enforcement layer that prevents plans from violating safety and resource constraints. They are derived from the plan's resolved automation profile and any plan-level overrides. Guards are evaluated by the `GuardEnforcer` component in the Application layer before and during plan execution. + +#### Guard Evaluation Order + +Guards are evaluated in strict priority order. The first guard that triggers causes the plan to be blocked or paused: + +1. **Denylist** (fast reject) — Tool names or glob patterns that are explicitly forbidden. Checked synchronously before any tool invocation. If a tool matches a denylist pattern, the invocation is rejected immediately with a `GuardViolationError` and the plan enters `blocked` state. +2. **Budget caps** — Checked before each tool invocation and after each LLM call: + - `max_cost_per_plan` (USD): Estimated cost of the current invocation would exceed the cap → plan paused, user prompted to approve continuation or cancel. + - `max_tokens_per_plan` (tokens): Total tokens consumed would exceed the cap → same pause behavior. + - `max_wall_clock_seconds` (seconds): Elapsed time since plan start would exceed the cap → plan paused. +3. **Tool call limits** — Checked per-actor per-invocation: + - `max_tool_calls_per_step`: Total tool calls by this actor in this plan would exceed the limit → actor invocation rejected. + - `max_retries_per_step`: Retry count for this specific tool in this plan would exceed the limit → tool invocation rejected, plan enters `blocked` state. + +#### GuardEnforcer Component + +```python +class GuardEnforcer: + """Evaluates guards before and during plan execution.""" + + def check_tool_invocation( + self, + plan_id: str, + actor_id: str, + tool_name: str, + estimated_cost_usd: float | None, + estimated_tokens: int | None, + ) -> GuardCheckResult: + """ + Check all guards before a tool invocation. + + Returns GuardCheckResult with: + - allowed: bool + - violated_guard: GuardType | None + - violation_message: str | None + - requires_user_approval: bool (True for budget cap pauses) + """ + ... +``` + +#### Guard Sources + +Guards are resolved from multiple sources with the following precedence (highest to lowest): + +| Source | Precedence | Example | +|---|---|---| +| Plan-level override | Highest | `agents plan use --automation-profile ci ` | +| Action-level default | Medium | `automation_profile: trusted` in action YAML | +| Global default | Lowest | `automation.default_profile: manual` in `~/.config/cleveragents/config.yaml` | + +The resolved profile's `guard` sub-model defines the active denylist, budget caps, and tool call limits. See `agents plan guard ` to inspect the active guards for a running plan. + +#### Guard Violation Behavior + +| Guard Type | Violation Behavior | Plan State After | +|---|---|---| +| Denylist | Immediate rejection; tool call not executed | `blocked` | +| Budget cap (cost/tokens) | Plan paused; user prompted to approve or cancel | `awaiting_approval` | +| Budget cap (wall clock) | Plan paused; user prompted to approve or cancel | `awaiting_approval` | +| Tool call limit | Tool invocation rejected; plan blocked | `blocked` | +| Retry limit | Tool invocation rejected; plan blocked | `blocked` | + +When a plan is in `blocked` state due to a guard violation, the user can: +- `agents plan cancel ` — abandon the plan +- `agents plan correct --mode=append --guidance "..."` — add guidance and retry +- Modify the automation profile and re-run (for budget cap violations) + +--- + +### Event Queue {#event-queue} + +The Event Queue is the internal publish/subscribe system that decouples plan lifecycle producers (the Application layer) from consumers (the TUI, CLI streaming, external integrations). It is implemented as an in-process async queue in local mode and as a persistent message queue in server mode. + +#### Event Types + +| Event Type | Published When | Key Fields | +|---|---|---| +| `plan.created` | A new plan is created via `agents plan use` | `plan_id`, `action`, `project_ids` | +| `plan.phase_changed` | Plan transitions between phases (strategize → execute → apply) | `plan_id`, `from_phase`, `to_phase` | +| `plan.state_changed` | Plan processing state changes (queued → processing → completed) | `plan_id`, `from_state`, `to_state` | +| `plan.decision_recorded` | A new decision is added to the decision tree | `plan_id`, `decision_id`, `decision_type` | +| `plan.tool_called` | A tool is invoked by an actor | `plan_id`, `actor_id`, `tool_name`, `invocation_id` | +| `plan.tool_completed` | A tool invocation completes | `plan_id`, `invocation_id`, `success`, `duration_ms` | +| `plan.guard_violated` | A guard check fails | `plan_id`, `guard_type`, `violation_message` | +| `plan.correction_started` | A correction attempt begins | `plan_id`, `correction_id`, `mode` | +| `plan.correction_completed` | A correction attempt finishes | `plan_id`, `correction_id`, `success` | +| `plan.subplan_spawned` | A child plan is spawned | `parent_plan_id`, `child_plan_id`, `decision_id` | +| `plan.merge_completed` | Subplan merge finishes | `plan_id`, `conflict_count`, `auto_merged_count` | +| `session.created` | A new session is created | `session_id` | +| `session.deleted` | A session is deleted | `session_id` | + +#### Event Schema + +All events share a common envelope: + +```python +@dataclass +class PlanEvent: + event_id: str # ULID + event_type: str # e.g., "plan.phase_changed" + occurred_at: datetime # UTC timestamp + plan_id: str | None # Plan ULID (None for session events) + session_id: str | None # Session ID (None for plan-only events) + payload: dict[str, Any] # Event-specific fields (see table above) +``` + +#### Publish/Subscribe API + +The `A2aEventQueue` provides the publish/subscribe interface: + +```python +from dataclasses import dataclass +from collections.abc import AsyncIterator + +@dataclass +class SubscriptionHandle: + """Active subscription returned by `subscribe`.""" + + id: str + iterator: AsyncIterator[PlanEvent] + + +class A2aEventQueue: + """In-process async event queue for plan lifecycle events.""" + + async def publish(self, event: PlanEvent) -> None: + """Publish an event to all registered subscribers.""" + ... + + def subscribe( + self, + event_types: list[str] | None = None, # None = subscribe to all + plan_id: str | None = None, # None = all plans + ) -> SubscriptionHandle: + """ + Subscribe to events. Returns a SubscriptionHandle containing the + subscription ID and an async iterator that yields matching events. + Callers must consume events promptly to avoid backpressure. + """ + ... + + async def unsubscribe(self, subscription: SubscriptionHandle | str) -> None: + """Cancel a subscription by handle or subscription ID.""" + ... +``` + +`SubscriptionHandle` allows callers to manage long-lived subscriptions safely. The handle's `iterator` yields events, while the `id` is used when calling `unsubscribe()` or when storing subscriptions for later cancellation. + +#### Local vs. Server Mode + +| Mode | Implementation | Persistence | +|---|---|---| +| **Local** | In-process `asyncio.Queue` per subscriber | Not persisted; events lost on process exit | +| **Server** | Persistent message queue (Redis Streams or PostgreSQL LISTEN/NOTIFY) | Events persisted; subscribers can replay from a cursor | + +In local mode, the TUI subscribes to the `A2aEventQueue` on startup and receives all plan lifecycle events in real time. The CLI streaming output (`agents plan status --follow`) also subscribes to the queue. + +--- + +### Autonomy Acceptance {#autonomy-acceptance} + +The Autonomy Acceptance criteria define what it means for the system to successfully complete a realistic, large-scale task autonomously. This section specifies the measurable acceptance criteria for v3.5.0 Deliverable 10. + +#### Definition of "Autonomous Completion" + +A task is considered **autonomously completed** when: + +1. The task is initiated with a single `agents plan use ` command. +2. No human intervention is required during Strategize or Execute phases (no `awaiting_approval` pauses, no manual corrections). +3. The plan reaches the Apply phase with a non-empty, correct changeset. +4. The changeset passes all required validations (if any are attached). +5. The user reviews and approves the apply (the apply step itself is always human-gated unless `automation_profile: full-auto`). + +#### Reference Task: Python Module Port + +The canonical reference task for autonomy acceptance testing is **porting a Python module from one framework to another**. This task is chosen because it: + +- Requires multi-level hierarchical decomposition (root → subsystem → module → file) +- Involves parallel execution (multiple files ported concurrently) +- Requires decision correction capability (if early decisions are wrong) +- Has clear, verifiable success criteria (tests pass, type checker passes) +- Is representative of real-world software engineering tasks + +**Reference task specification:** + +```yaml +# actions/port-python-module.yaml +action: + name: local/port-python-module + description: "Port a Python module from one framework to another" + args: + - name: source_framework + type: string + description: "Framework to port from (e.g., 'flask')" + - name: target_framework + type: string + description: "Framework to port to (e.g., 'fastapi')" + - name: module_path + type: string + description: "Path to the module to port (e.g., 'src/api/')" + definition_of_done: | + - All tests pass (pytest exit 0) + - Type checker passes (pyright --strict exit 0) + - No import errors at runtime + - API surface preserved (same endpoints, same response schemas) + strategy_actor: local/strategy-actor + execution_actor: local/code-executor +``` + +#### Acceptance Criteria + +| # | Criterion | Measurement | +|---|-----------|-------------| +| 1 | Task initiated with single command | `agents plan use local/port-python-module local/my-project --arg source_framework=flask --arg target_framework=fastapi --arg module_path=src/api/` | +| 2 | Hierarchical decomposition creates ≥ 4 levels | `agents plan tree ` shows depth ≥ 4 | +| 3 | Parallel execution used for independent files | `agents plan tree ` shows `subplan_parallel_spawn` decisions | +| 4 | No human intervention required | Plan reaches Apply phase without entering `awaiting_approval` state | +| 5 | All required validations pass | `agents plan status ` shows `validations: all_passed` | +| 6 | Changeset is non-empty and correct | `agents plan diff ` shows changes; ported module tests pass | +| 7 | Completes within wall-clock budget | Plan completes within `max_wall_clock_seconds` (default: 3600) | +| 8 | Coverage ≥ 97% for autonomy-related code paths | `nox -s coverage_report` passes | + +#### Autonomy Acceptance Test + +The Robot Framework E2E test for autonomy acceptance: + +```robotframework +*** Test Cases *** +Autonomous Python Module Port Completes Without Intervention + [Documentation] Verify that a realistic porting task completes autonomously + ... with 4+ subplan levels and no human intervention required. + [Tags] autonomy e2e slow + + # Setup: create a Flask project with a simple API module + ${project}= Create Test Project flask-to-fastapi-test + Add Resource To Project ${project} local/flask-api-repo + + # Run the porting task + ${plan_id}= Run Plan local/port-python-module ${project} + ... source_framework=flask target_framework=fastapi module_path=src/api/ + + # Verify autonomous completion + Wait For Plan Phase ${plan_id} apply timeout=3600s + Plan Should Not Have Entered State ${plan_id} awaiting_approval + Plan Tree Depth Should Be At Least ${plan_id} 4 + Plan Should Have Parallel Spawn Decisions ${plan_id} + Plan Validations Should All Pass ${plan_id} +``` + ### Human-in-the-Loop Collaboration !!! adr "Architecture Decision" @@ -29219,6 +29855,87 @@ Hidden ──shift+tab──► Visible ──shift+tab──► Fullscreen ``` +### TUI Materializer {#tui-materializer} + +The `TuiMaterializer` is the bridge between the Output Rendering Framework (ADR-021) and the Textual widget tree. It implements the `MaterializationStrategy` protocol, enabling all CLI command producers to render in the TUI without modification — the same code that writes to a terminal writes to the TUI. + +#### Architecture + +``` +CLI Command Producer + │ + ▼ + ElementHandle (ADR-021) + │ events: append, update, complete, error + ▼ + TuiMaterializer ◄── implements MaterializationStrategy + │ + ▼ + Textual Widget Tree + (ConversationBlock, ToolCallWidget, TableWidget, ...) +``` + +The `TuiMaterializer` subscribes to `ElementHandle` events and routes them to the appropriate Textual widget. Each `ElementHandle` type maps to a specific widget class: + +| `ElementHandle` Type | Textual Widget | Behavior | +|---|---|---| +| `TextElement` | `ConversationBlock` | Appends text; supports streaming (character-by-character append) | +| `TableElement` | `DataTable` | Rows appended incrementally; columns auto-sized | +| `TreeElement` | `Tree` | Nodes added as decisions are recorded | +| `ProgressElement` | `ProgressBar` | Progress updated on each `update` event | +| `ErrorElement` | `ConversationBlock` (error style) | Rendered with error styling (red, bold) | +| `ToolCallElement` | `ToolCallWidget` | Shows tool name, status (running/success/error), duration | + +#### TuiMaterializer Interface + +```python +class TuiMaterializer: + """ + Maps ElementHandle events to Textual widget operations. + Implements the MaterializationStrategy protocol. + """ + + def __init__(self, conversation_view: ConversationView) -> None: + """ + Args: + conversation_view: The Textual widget that owns the conversation stream. + TuiMaterializer posts widget updates to this view's + message queue (thread-safe via call_from_thread). + """ + ... + + def materialize(self, handle: ElementHandle) -> None: + """ + Called by the Output Rendering Framework when a new element is created. + Creates the appropriate Textual widget and registers event listeners. + """ + ... + + def on_element_event(self, event: ElementEvent) -> None: + """ + Called when an ElementHandle emits an event (append, update, complete, error). + Routes the event to the appropriate widget update method. + Thread-safe: uses call_from_thread to post updates to the Textual event loop. + """ + ... +``` + +#### A2A Integration + +The TUI communicates with the Application layer exclusively through A2A: + +1. **Command invocation**: User input in the TUI is translated to A2A method calls (`tasks/send`, `plans/use`, etc.) via `A2aLocalFacade`. +2. **Event subscription**: The TUI subscribes to the `A2aEventQueue` on startup. Plan lifecycle events (phase changes, decision recordings, tool calls) are received and routed to the appropriate TUI widgets. +3. **Output rendering**: Command producers write to `ElementHandle` objects; the `TuiMaterializer` maps these to Textual widget updates. + +This architecture ensures **TUI/CLI parity**: every operation available in the CLI is available in the TUI via the same Application-layer code paths. No TUI-exclusive features exist. + +#### Thread Safety + +Textual's event loop runs on the main thread. The `TuiMaterializer` receives events from background threads (A2A event queue consumers, command producers). All widget updates must be posted to the Textual event loop via `widget.call_from_thread(update_fn)` or `app.call_from_thread(update_fn)`. The `TuiMaterializer` is responsible for this thread-safety boundary. + +--- + ### Persona System !!! adr "Architecture Decision" @@ -46581,6 +47298,94 @@ The sandbox layer supports custom isolation strategies for specialized resource Custom strategies are mapped to resource types via the resource type configuration's `sandbox_strategy` field, or globally via `sandbox.strategy` config. +#### Provider Registry {#provider-registry} + +The `ProviderRegistry` manages LLM provider implementations. It is the central registry for all LLM backends, enabling actors to reference providers by name without coupling to specific SDK implementations. + +##### Built-in Providers + +| Provider Name | Package | Models | +|---|---|---| +| `openai` | `langchain-openai` | `gpt-4o`, `gpt-4o-mini`, `o1`, `o3-mini`, ... | +| `anthropic` | `langchain-anthropic` | `claude-3-5-sonnet`, `claude-3-5-haiku`, `claude-4-sonnet`, ... | +| `google` | `langchain-google-genai` | `gemini-2.0-flash`, `gemini-2.0-pro`, ... | +| `ollama` | `langchain-ollama` | Any locally-running Ollama model | +| `azure-openai` | `langchain-openai` | Azure-hosted OpenAI models | + +##### Registering a Custom Provider + +Custom providers implement the `AIProviderInterface` protocol and are registered via YAML configuration: + +```yaml +# File: providers/my-custom-provider.yaml +cleveragents: + version: "3.0" + +provider: + name: local/my-llm-provider + description: "Custom LLM provider for internal models" + module: my_package.providers.my_llm + class: MyLLMProvider + # Optional: environment variables required by this provider + required_env: + - MY_LLM_API_KEY + - MY_LLM_BASE_URL +``` + +Register with: `agents provider add --config providers/my-custom-provider.yaml` + +##### Actor YAML with Custom Provider + +Once registered, actors reference custom providers by name: + +```yaml +# File: actors/my-actor.yaml +actor: + name: local/my-actor + type: agent + config: + actor: local/my-llm-provider/my-model-v2 + temperature: 0.3 + skills: + - local/file-ops +``` + +##### Auto-Discovery + +The `ProviderRegistry` auto-discovers installed `langchain-*` packages on startup. Any package that exports a class implementing `AIProviderInterface` and registers it via the `cleveragents.providers` entry point is automatically available: + +```toml +# pyproject.toml for a custom provider package +[project.entry-points."cleveragents.providers"] +my-llm = "my_package.providers:MyLLMProvider" +``` + +##### ProviderRegistry API + +```python +class ProviderRegistry: + """Registry for LLM provider implementations.""" + + def register(self, name: str, provider: AIProviderInterface) -> None: + """Register a provider by name. Raises ValueError if name already registered.""" + ... + + def get(self, name: str) -> AIProviderInterface: + """Get a provider by name. Raises KeyError if not found.""" + ... + + def list_providers(self) -> list[ProviderInfo]: + """List all registered providers with their capabilities.""" + ... + + def resolve_model(self, actor_ref: str) -> tuple[AIProviderInterface, str]: + """ + Parse 'provider/model' actor reference and return (provider, model_name). + E.g., 'anthropic/claude-4-sonnet' → (AnthropicProvider(), 'claude-4-sonnet') + """ + ... +``` + #### Plugin Security Contract The plugin loading system enforces a security boundary: module imports are restricted to a configurable prefix allowlist to prevent arbitrary code execution from untrusted configuration. However, **protocol validation must never instantiate the plugin class** to perform type checking. Instantiation during validation runs `__init__` side effects (network connections, file I/O, subprocess spawning) before the plugin is approved for use. @@ -46815,7 +47620,7 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a **Goal**: Decisions are recorded during Strategize and Execute phases and persisted to the database. Users can view the decision tree, inspect individual decisions, manage invariants, and correct decisions with selective subtree recomputation. -**Spec Coverage**: [Decision Tree and Correction](#decision-tree-and-correction), [Invariant System](#invariant-system), [Validation Abstraction](#validation-abstraction) +**Spec Coverage**: [Decision Tree and Visualization](#the-plan-decision-tree-and-visualization), [Correcting Plans](#correcting-plans-core-feature), [Semantic Error Prevention — Invariants](#semantic-error-prevention), [Validation Abstraction](#validation) #### Deliverables @@ -46895,7 +47700,7 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a **Goal**: The Advanced Context Management System v1 is operational. Projects with 10,000+ files can be indexed and queried. The context assembly pipeline produces scoped, budget-constrained context views for actors. Hot/warm/cold storage tiers manage context lifecycle. -**Spec Coverage**: [ACMS Architecture](#acms-advanced-context-management-system), [Context Assembly Pipeline](#context-assembly-pipeline), [UKO Ontology](#uko-universal-knowledge-ontology), [Hot/Warm/Cold Tiers](#context-storage-tiers) +**Spec Coverage**: [ACMS Architecture](#acms-advanced-context-management-system), [Context Assembly Pipeline](#context-assembly-pipeline), [UKO Ontology](#uko-universal-knowledge-ontology), [Hot/Warm/Cold Tiers](#temporal-data-model-and-storage-tiers) #### Deliverables @@ -46976,7 +47781,7 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a **Goal**: Advanced concepts not needed for basic MVP. Extends beyond core MVP but does not require TUI (v3.7.0) or Server (v3.8.0). Includes advanced context strategies, additional LLM backends, additional resource types, A2A module rename, container tool execution, pluggable scope chain extensions, cost/session budgets, and E2E workflow specification tests. -**Spec Coverage**: [LSP Integration](#lsp-integration), [Resource Type Inheritance](#resource-type-inheritance), [Devcontainer Integration](#devcontainer-integration), [Container Resource Types](#container-resource-types), [Advanced Context Strategies](#advanced-context-strategies), [Provider Registry](#provider-registry) +**Spec Coverage**: [LSP Integration](#lsp-integration), [Resource Type Inheritance](#resource-type-inheritance), [Devcontainer Integration](#devcontainer-integration), [Container Resource Types](#container-resource-types), [Advanced Context Strategies](#advanced-context-strategies), [Provider Registry](#provider-registry), [Testing Strategy — E2E](#testing-strategy-e2e) #### Deliverables @@ -47066,6 +47871,104 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a --- +### Testing Strategy — E2E {#testing-strategy-e2e} + +End-to-end (E2E) tests verify complete user workflows from CLI invocation through to observable output. They are implemented using **Robot Framework** and run via `nox -s integration_tests`. + +#### E2E Test Structure + +E2E tests live in `robot/` and are organized by workflow domain: + +``` +robot/ + plan_lifecycle/ + plan_use_and_execute.robot # Basic plan lifecycle + plan_correction_revert.robot # Correction with revert mode + plan_correction_append.robot # Correction with append mode + plan_subplans.robot # Subplan spawning and merging + plan_autonomy.robot # Autonomy acceptance test + acms/ + context_assembly.robot # ACMS pipeline end-to-end + context_scaling.robot # 10k+ file project indexing + invariants/ + invariant_enforcement.robot # Invariant add/list/enforce + tui/ + tui_launch.robot # TUI launch and basic interaction + tui_session_persistence.robot # Session persistence across restarts + server/ + a2a_local_facade.robot # A2A local facade operations +``` + +#### Required E2E Coverage (v3.6.0) + +The following workflows must be covered by Robot Framework E2E tests by v3.6.0: + +| Workflow | Test File | Key Assertions | +|---|---|---| +| **Plan lifecycle** | `plan_lifecycle/plan_use_and_execute.robot` | Plan created, strategize completes, execute completes, apply succeeds | +| **Plan correction (revert)** | `plan_lifecycle/plan_correction_revert.robot` | Correction creates new attempt; affected subtree recomputed; unaffected decisions preserved | +| **Plan correction (append)** | `plan_lifecycle/plan_correction_append.robot` | Append correction recorded; no subtree recomputation; plan continues | +| **Subplan spawning** | `plan_lifecycle/plan_subplans.robot` | Child plans spawned; parallel execution works; results merged | +| **Invariant enforcement** | `invariants/invariant_enforcement.robot` | Invariant added; enforced during strategize; violation blocked | +| **ACMS context assembly** | `acms/context_assembly.robot` | Context assembled for plan; budget constraints respected | +| **ACMS scaling** | `acms/context_scaling.robot` | 10k+ file project indexes without timeout | +| **Autonomy acceptance** | `plan_lifecycle/plan_autonomy.robot` | See §Autonomy Acceptance for full criteria | + +#### Robot Framework Keywords + +Common keywords are defined in `robot/resources/` and shared across test files: + +```robotframework +*** Settings *** +Library robot/lib/CleverAgentsLibrary.py + +*** Keywords *** +Create Test Project + [Arguments] ${name} + ${result}= Run CLI agents project create ${name} + Should Be Equal ${result.rc} ${0} + RETURN ${name} + +Run Plan + [Arguments] ${action} ${project} &{args} + ${arg_flags}= Build Arg Flags &{args} + ${result}= Run CLI agents plan use ${action} ${project} ${arg_flags} + Should Be Equal ${result.rc} ${0} + ${plan_id}= Extract Plan ID ${result.stdout} + RETURN ${plan_id} + +Wait For Plan Phase + [Arguments] ${plan_id} ${phase} ${timeout}=300s + Wait Until Keyword Succeeds ${timeout} 5s + ... Plan Should Be In Phase ${plan_id} ${phase} + +Plan Should Be In Phase + [Arguments] ${plan_id} ${expected_phase} + ${result}= Run CLI agents plan status ${plan_id} --format json + ${data}= Parse JSON ${result.stdout} + Should Be Equal ${data['data']['phase']} ${expected_phase} +``` + +#### E2E Test Execution + +```bash +# Run all E2E tests +nox -s integration_tests + +# Run a specific suite +nox -s integration_tests -- robot/plan_lifecycle/ + +# Run with verbose output +nox -s integration_tests -- --loglevel DEBUG robot/ + +# Run only tests tagged 'autonomy' +nox -s integration_tests -- --include autonomy robot/ +``` + +E2E tests require a running CleverAgents environment with at least one configured actor. The test environment is set up by `nox -s integration_tests` using a dedicated test configuration that uses mock LLM providers (no real API calls) for deterministic test execution. + +--- + ### Cross-Milestone Quality Gates These quality gates apply to **every milestone** and must pass before a milestone is considered complete: