From 44652c216a7578c1b4bfad20337b9864413b6a1d Mon Sep 17 00:00:00 2001 From: HAL 9000 Date: Fri, 10 Apr 2026 19:50:00 +0000 Subject: [PATCH] docs: add sandbox, correction, and invariant module guides Document lifecycle details, security warnings, and CLI usage for the new module guides so implementors have a single reference. ISSUES CLOSED: #4848 --- docs/modules/correction-attempts.md | 455 +++++++++++++++++++++++ docs/modules/invariant-reconciliation.md | 383 ++++++++++++++----- docs/modules/sandbox.md | 381 +++++++++++++++++++ mkdocs.yml | 2 + 4 files changed, 1136 insertions(+), 85 deletions(-) create mode 100644 docs/modules/correction-attempts.md create mode 100644 docs/modules/sandbox.md diff --git a/docs/modules/correction-attempts.md b/docs/modules/correction-attempts.md new file mode 100644 index 000000000..170ad9864 --- /dev/null +++ b/docs/modules/correction-attempts.md @@ -0,0 +1,455 @@ +# Correction Attempts Module + +**Package:** `cleveragents.domain.models.core.correction` +**Introduced:** M4 (v3.3.0); `CorrectionAttemptRecord` and `correction_attempts` table added in M7 (v3.6.0, issue #920) + +The correction module provides the domain models and lifecycle management for +plan decision tree corrections. When a plan's execution produces an +undesirable result, a correction allows the user to revert to a prior decision +point or append new guidance without restarting the entire plan. + +For the design rationale see +[ADR-007](../adr/ADR-007-decision-tree-and-correction.md) (Decision Tree & Correction), +[ADR-035](../adr/ADR-035-decision-tree-rollback-and-replay.md) (Rollback & Replay). + +--- + +## Purpose + +During plan execution, the actor makes a series of decisions (tool calls, +resource writes, sub-plan spawns). These decisions form a tree. If the user +or an automated check determines that a decision was wrong, a correction +provides two recovery paths: + +| Mode | Description | +|------|-------------| +| **Revert** | Invalidate the subtree rooted at the target decision; restore checkpoints; re-execute from that point with new guidance | +| **Append** | Add a new child plan at the target decision without invalidating existing decisions | + +--- + +## Core Domain Models + +### `CorrectionRequest` + +```python +from cleveragents.domain.models.core.correction import CorrectionRequest, CorrectionMode + +request = CorrectionRequest( + plan_id="01ARZ3...", + target_decision_id="01BCD4...", + mode=CorrectionMode.REVERT, + guidance="The file path was wrong — use src/api/ not src/app/", +) +``` + +| Field | Type | Description | +|-------|------|-------------| +| `correction_id` | `str` (ULID) | Unique identifier (auto-generated) | +| `plan_id` | `str` | Plan that owns the targeted decision tree | +| `target_decision_id` | `str` | Decision node to apply correction at | +| `mode` | `CorrectionMode` | `revert` or `append` | +| `guidance` | `str` (max 10,000 chars) | Human-supplied guidance | +| `dry_run` | `bool` | When `True`, only compute impact without executing | +| `status` | `CorrectionStatus` | Current lifecycle status | +| `created_at` | `datetime` (UTC) | When the request was created | + +!!! warning "Security: validate untrusted `guidance` input" + The `guidance` field is injected directly into the LLM actor during + re-execution. When accepting guidance from untrusted integrations, validate + and sanitise the input to reduce prompt-injection risk. The 10,000-character + limit is enforced at the model layer but does not prevent malicious + instructions. + +--- + +### `CorrectionMode` + +```python +from cleveragents.domain.models.core.correction import CorrectionMode + +CorrectionMode.REVERT # "revert" — invalidate subtree and re-execute +CorrectionMode.APPEND # "append" — add new child plan at target +``` + +--- + +### `CorrectionStatus` + +Lifecycle states for a correction request: + +``` +PENDING → ANALYZING → EXECUTING → APPLIED + └──► FAILED + └──► CANCELLED + └──► REJECTED +``` + +| Status | Meaning | +|--------|---------| +| `PENDING` | Request created, not yet processed | +| `ANALYZING` | Impact analysis in progress | +| `EXECUTING` | Correction is being applied | +| `APPLIED` | Correction successfully applied | +| `FAILED` | Correction failed | +| `CANCELLED` | Correction was cancelled | +| `REJECTED` | Correction rejected (e.g. applied child plans block revert) | + +--- + +### `CorrectionImpact` + +Result of the impact analysis — computed before executing a correction. + +```python +from cleveragents.domain.models.core.correction import CorrectionImpact + +impact = CorrectionImpact( + affected_decisions=["01BCD4...", "01BCD5..."], + excluded_decisions=["01BCD6..."], + affected_files=["src/api/handler.py"], + affected_child_plans=["01EFG7..."], + risk_level="medium", + rollback_tier="full", + rollback_tier_depth=2, +) +``` + +| Field | Type | Description | +|-------|------|-------------| +| `affected_decisions` | `list[str]` | Decision IDs in the affected subtree (BFS order) | +| `excluded_decisions` | `list[str]` | Decision IDs NOT in the affected subtree (preserved) | +| `affected_files` | `list[str]` | Files touched by affected decisions | +| `affected_child_plans` | `list[str]` | Child plan IDs spawned by affected decisions | +| `estimated_cost` | `float \| None` | Estimated recompute cost | +| `risk_level` | `str` | `low`, `medium`, or `high` | +| `rollback_tier` | `str` | `full`, `phase`, or `append_only` | +| `rollback_tier_depth` | `int` | Depth of the targeted decision from the tree root | +| `artifacts_to_archive` | `list[str]` | Artifact paths to archive on revert | + +--- + +### `CorrectionResult` + +Outcome of executing a correction. For revert corrections, includes fields +for checkpoint restoration and re-execution signalling. + +```python +from cleveragents.domain.models.core.correction import CorrectionResult, CorrectionStatus + +result = CorrectionResult( + correction_id="01ARZ3...", + status=CorrectionStatus.APPLIED, + reverted_decisions=["01BCD4...", "01BCD5..."], + checkpoint_restored=True, + actor_state_ref="lc:checkpoint:01XYZ...", + user_intervention_decision_id="01HIJ8...", + phase_transition_target="strategize", +) +``` + +**Revert re-execution fields** (spec § Correction Flow, Revert Mode): + +| Field | Type | Description | +|-------|------|-------------| +| `checkpoint_restored` | `bool` | Whether a sandbox checkpoint was restored | +| `actor_state_ref` | `str` | LangGraph actor checkpoint reference for reasoning rollback | +| `user_intervention_decision_id` | `str` | ULID of the user_intervention decision created to inject guidance | +| `phase_transition_target` | `str` | Target plan phase after revert (e.g. `"strategize"`) | + +--- + +### `CorrectionDryRunReport` + +Detailed report showing what a correction *would* do, without modifying state. + +```python +from cleveragents.domain.models.core.correction import CorrectionDryRunReport + +report = service.generate_dry_run_report(request) +# Access impact data via the nested impact object: +print(report.impact.affected_decisions) +print(report.impact.rollback_tier_depth) +print(report.impact.affected_child_plans) +``` + +> **Note:** `excluded_decisions`, `rollback_tier_depth`, and +> `child_plans_to_rollback` were previously duplicated at the top level. +> They were removed in M7 (issue #1087) to eliminate divergence risk. +> Access them via `report.impact.*` instead. + +--- + +## Correction Attempt Records + +The `correction_attempts` table (added in M7, issue #920) provides a +persistent audit trail of every correction execution attempt. + +### `CorrectionAttemptRecord` + +```python +from cleveragents.domain.models.core.correction import ( + CorrectionAttemptRecord, + CorrectionAttemptState, + CorrectionMode, +) + +record = CorrectionAttemptRecord( + plan_id="01ARZ3...", + original_decision_id="01BCD4...", + mode=CorrectionMode.REVERT, + guidance="Use src/api/ not src/app/", +) +# record.state == CorrectionAttemptState.PENDING +# record.correction_attempt_id — auto-generated ULID +``` + +| Field | Type | Description | +|-------|------|-------------| +| `correction_attempt_id` | `str` (ULID) | Unique identifier (auto-generated) | +| `plan_id` | `str` | Plan that owns the targeted decision tree | +| `original_decision_id` | `str` | Decision node being corrected | +| `new_decision_id` | `str \| None` | New decision created by the correction (if any) | +| `mode` | `CorrectionMode` | `revert` or `append` | +| `guidance` | `str` (max 10,000 chars) | Human-supplied guidance (required, non-empty) | +| `archived_artifacts_path` | `str \| None` | Path to archived artifacts (revert mode) | +| `state` | `CorrectionAttemptState` | Current lifecycle state | +| `created_at` | `datetime` (UTC) | When the record was created | +| `completed_at` | `datetime \| None` | When the attempt completed | + +> **Note:** `archived_artifacts_path` is always system-generated within the +> configured artifacts directory. The path is validated to stay inside that +> directory before storage, and no direct user input is accepted for this +> field. + +**Validation rules:** +- `plan_id`, `original_decision_id`, and `guidance` must be non-empty (whitespace stripped) +- `new_decision_id` must be non-empty when set +- `created_at` and `completed_at` are normalised to UTC (naive datetimes are coerced) + +--- + +### `CorrectionAttemptState` + +Spec-defined lifecycle for correction attempt records: + +``` +PENDING → EXECUTING → COMPLETE + └──► FAILED +``` + +| State | Meaning | +|-------|---------| +| `PENDING` | Attempt created, not yet started | +| `EXECUTING` | Correction is actively running | +| `COMPLETE` | Correction completed successfully (terminal) | +| `FAILED` | Correction failed (terminal) | + +```python +from cleveragents.domain.models.core.correction import ( + CorrectionAttemptState, + CORRECTION_ATTEMPT_VALID_TRANSITIONS, + CORRECTION_ATTEMPT_TERMINAL_STATES, + validate_correction_state_transition, +) + +# Check valid transitions +CORRECTION_ATTEMPT_VALID_TRANSITIONS[CorrectionAttemptState.PENDING] +# → frozenset({CorrectionAttemptState.EXECUTING}) + +# Check terminal states +CorrectionAttemptState.COMPLETE in CORRECTION_ATTEMPT_TERMINAL_STATES +# → True + +# Validate a transition (raises ValueError on invalid) +validate_correction_state_transition( + CorrectionAttemptState.PENDING, + CorrectionAttemptState.EXECUTING, +) +``` + +--- + +## Cross-Plan Cascade + +When a correction targets a decision that spawned child plans, the cascade +mechanism determines what to do with each child plan. + +### `ChildPlanState` + +Observed state of a child plan for cascade classification: + +| State | Meaning | +|-------|---------| +| `NOT_STARTED` | Child plan exists but has not begun executing | +| `IN_PROGRESS` | Child plan is currently executing | +| `COMPLETED_UNAPPLIED` | Child plan finished but changes not yet applied | +| `APPLIED` | Child plan's changes have been applied (blocks revert) | + +### `CascadeAction` + +```python +from cleveragents.domain.models.core.correction import CascadeAction, ChildPlanState + +action = CascadeAction( + child_plan_id="01EFG7...", + child_plan_state=ChildPlanState.IN_PROGRESS, + action="cancel_and_rollback", + sandbox_rolled_back=True, +) +``` + +| `action` value | Meaning | +|----------------|---------| +| `cancel` | Cancel the child plan (no sandbox rollback needed) | +| `cancel_and_rollback` | Cancel and roll back the child plan's sandbox | +| `reject` | Cannot cascade — child plan is already applied | + +### `CorrectionRejection` + +Returned when a correction cannot proceed because applied child plans block it: + +```python +from cleveragents.domain.models.core.correction import CorrectionRejection + +rejection = CorrectionRejection( + correction_id="01ARZ3...", + reason="Child plan 01EFG7... has already been applied and cannot be reverted", + affected_applied_child_plan_ids=["01EFG7..."], +) +``` + +### `CascadeResult` + +Outcome of the full cascade operation: + +```python +from cleveragents.domain.models.core.correction import CascadeResult + +result = CascadeResult( + correction_id="01ARZ3...", + cascade_actions=[...], + rejected=False, + all_cancelled_plan_ids=["01EFG7..."], + all_rolled_back_plan_ids=["01EFG7..."], +) +``` + +--- + +## Using the Correction Service + +The `CorrectionService` (in `cleveragents.application.services`) orchestrates +the full correction workflow. Obtain it via the DI container: + +```python +service = container.correction_service() + +# Dry run — compute impact without executing +report = service.generate_dry_run_report(request) +print(f"Risk: {report.impact.risk_level}") +print(f"Affected decisions: {report.impact.affected_decisions}") + +# Execute a revert correction +result = service.execute_revert(request) +if result.status == CorrectionStatus.APPLIED: + print(f"Reverted {len(result.reverted_decisions)} decisions") + print(f"Phase transition target: {result.phase_transition_target}") + +# Execute an append correction +result = service.execute_append(request) +if result.spawned_child_plan_id: + print(f"Spawned child plan: {result.spawned_child_plan_id}") +``` + +--- + +## CLI Usage + +```bash +# Revert to a prior decision point +agents plan correct --plan --decision \ + --mode revert \ + --guidance "Use src/api/ not src/app/" + +# Dry run first to see impact +agents plan correct --plan --decision \ + --mode revert --dry-run + +# Append new guidance without reverting +agents plan correct --plan --decision \ + --mode append \ + --guidance "Also add error handling for the 404 case" +``` + +--- + +## Database Schema + +The `correction_attempts` table stores `CorrectionAttemptRecord` instances: + +```sql +CREATE TABLE correction_attempts ( + correction_attempt_id TEXT PRIMARY KEY, + plan_id TEXT NOT NULL REFERENCES v3_plans(plan_id) ON DELETE CASCADE, + original_decision_id TEXT NOT NULL REFERENCES decisions(decision_id) ON DELETE RESTRICT, + new_decision_id TEXT REFERENCES decisions(decision_id) ON DELETE RESTRICT, + mode TEXT NOT NULL, + guidance TEXT NOT NULL, + archived_artifacts_path TEXT, + state TEXT NOT NULL DEFAULT 'pending', + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%f', 'now')), + completed_at TEXT +); +``` + +**FK semantics:** +- `plan_id` → `CASCADE` (correction attempts are deleted with the plan) +- `original_decision_id` → `RESTRICT` (preserves correction audit trail when decisions are cleaned up) +- `new_decision_id` → `RESTRICT` (consistent with `original_decision_id`) + +--- + +## Testing + +The correction module is covered by 53 BDD scenarios in +`features/correction_attempts.feature` and 5 Robot Framework integration +tests in `robot/correction_attempts.robot`. + +```python +from cleveragents.domain.models.core.correction import ( + CorrectionAttemptRecord, + CorrectionAttemptState, + CorrectionMode, + validate_correction_state_transition, +) + +# Create a record +record = CorrectionAttemptRecord( + plan_id="plan-001", + original_decision_id="dec-001", + mode=CorrectionMode.REVERT, + guidance="Fix the path", +) +assert record.state == CorrectionAttemptState.PENDING + +# Validate transition +validate_correction_state_transition( + CorrectionAttemptState.PENDING, + CorrectionAttemptState.EXECUTING, +) # OK + +validate_correction_state_transition( + CorrectionAttemptState.COMPLETE, + CorrectionAttemptState.EXECUTING, +) # raises ValueError — terminal state +``` + +--- + +## Related Documentation + +- [ADR-007 Decision Tree & Correction](../adr/ADR-007-decision-tree-and-correction.md) +- [ADR-035 Decision Tree Rollback & Replay](../adr/ADR-035-decision-tree-rollback-and-replay.md) +- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) +- [Sandbox Module](sandbox.md) — checkpoint/rollback used during revert diff --git a/docs/modules/invariant-reconciliation.md b/docs/modules/invariant-reconciliation.md index bb6271165..185e529f5 100644 --- a/docs/modules/invariant-reconciliation.md +++ b/docs/modules/invariant-reconciliation.md @@ -1,34 +1,68 @@ # Invariant Reconciliation Module -**Package:** `cleveragents.actor.reconciliation` -**Introduced:** v3.8.0 +**Package:** `cleveragents.actor.reconciliation`, `cleveragents.application.services.invariant_service` +**Introduced:** v3.8.0 (the actor was originally prototyped in M3/v3.2.0; automatic +phase transition invocation shipped in M8/v3.7.0, issue #1941) -The Invariant Reconciliation Actor is a built-in actor that automatically runs at -every plan phase transition to ensure all active invariants are consistent and -correctly prioritised before any strategy or execution work begins. +The invariant reconciliation module enforces user-defined constraints +(*invariants*) at every plan phase transition. Invariants are declarative rules +that must hold throughout plan execution — for example, "all output files must +be UTF-8 encoded" or "never modify files outside the project directory". The +`InvariantReconciliationActor` merges invariants from every scope, resolves +conflicts deterministically, and blocks plan execution whenever a violation is +detected. -For the invariant data model and scope hierarchy, see -[`docs/reference/invariants.md`](../reference/invariants.md). -For the `InvariantService` API, see -[`docs/api/core.md`](../api/core.md). +For further background, see: +- [`docs/reference/invariants.md`](../reference/invariants.md) — data model and scope hierarchy +- [ADR-016 — Invariant System](../adr/ADR-016-invariant-system.md) — design rationale +- [Core API Reference — Invariant Service](../api/core.md#invariant-service) — API surface +- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) --- ## Purpose -Invariants are natural-language constraints on plan execution that can be defined -at four scopes: global, project, action, and plan. When multiple invariants apply -to the same plan, conflicts can arise — for example, a project-level invariant may -contradict a plan-level one. The reconciliation actor resolves these conflicts -deterministically using a precedence chain and records each decision in the audit -trail. +Invariants provide a safety net that prevents plans from violating +organisation-defined constraints. The reconciliation actor runs at the start of +each Strategize, Execute, and Apply phase to produce the effective invariant +set. If any invariant is violated, the phase transition is **blocked** with a +`ReconciliationBlockedError`, an `INVARIANT_VIOLATED` event is emitted, and the +plan remains in its current phase until the violation is resolved. + +--- + +## Invariant Scopes + +Invariants can be defined at four scopes with strict precedence: + +``` +plan > action > project > global +``` + +Higher-specificity scopes override lower-specificity ones for the same +normalised text. The sole exception is the `non_overridable` flag on global +invariants. + +!!! warning "Security: `non_overridable` global invariants cannot be bypassed" + A `global`-scoped invariant with `non_overridable=True` **always wins** over + plan-, action-, or project-scoped invariants. Use this to enforce + organisation-wide guardrails such as "Never write outside the project + directory" or "Never access credentials files". Plan-level actors cannot + override these protections. + +| Scope | Description | +|-------|-------------| +| `global` | Applies to all plans across every project | +| `project` | Applies to all plans within a project | +| `action` | Applies to all plans that use a specific action | +| `plan` | Applies only to a single plan instance | --- ## Automatic Invocation -`PlanLifecycleService` automatically invokes the reconciliation actor at the start -of three phase transitions: +`PlanLifecycleService` automatically invokes the reconciliation actor at the +beginning of these phase transitions: | Transition | Method | |------------|--------| @@ -36,44 +70,74 @@ of three phase transitions: | Execute start | `execute_plan()` | | Apply start | `apply_plan()` | -If reconciliation fails, the phase transition is **blocked** with -`ReconciliationBlockedError` and an `INVARIANT_VIOLATED` event is emitted on the -event bus. The transition cannot proceed until the invariants are corrected. - -Post-correction reconciliation runs automatically via a `CORRECTION_APPLIED` event -subscription (best-effort; does not block correction completion). +If reconciliation fails during any transition, the phase change is blocked and +`INVARIANT_VIOLATED` is emitted. After a correction is applied, the actor runs +again via a `CORRECTION_APPLIED` event subscription (best-effort; it does not +delay correction completion). --- -## Reconciliation Algorithm +## Core Classes -The actor follows a six-step algorithm: +### `InvariantService` -1. **Load** — fetch all invariants in scope for the plan (global → project → action → plan) -2. **Deduplicate** — remove exact-text duplicates, keeping the highest-scope copy -3. **Detect conflicts** — identify pairs where one invariant's constraint contradicts another's -4. **Resolve** — apply the precedence chain: `plan > action > project > global`; when scopes are equal, the invariant with the lower `position` index wins -5. **Record** — write each resolution decision to the audit trail with rationale -6. **Emit** — publish `INVARIANT_RECONCILED` event on the event bus +```python +from cleveragents.application.services.invariant_service import InvariantService +from cleveragents.domain.models.core.invariant import InvariantScope ---- +# Preferred: resolve via the dependency injection container +service = container.invariant_service() -## Key Classes +# Testing: construct directly if you supply the required collaborators +service = InvariantService(event_bus=event_bus) +``` + +**Key methods** + +| Method | Returns | Description | +|--------|---------|-------------| +| `add_invariant(text, scope, source_name, *, non_overridable=False)` | `Invariant` | Validates and persists a new invariant | +| `list_invariants(scope, source_name, effective=False)` | `list[Invariant]` | Lists stored invariants; `effective=True` yields the merged precedence chain | +| `remove_invariant(invariant_id)` | `Invariant` | Soft-deletes an invariant (sets `active=False`) | +| `get_effective_invariants(plan_id, project_name)` | `list[Invariant]` | Returns merged invariants using plan > action > project > global | +| `enforce_invariants(plan_id, invariants, actor_response, violated_invariant_ids)` | `list[InvariantEnforcementRecord]` | Creates enforcement records and emits events | + +**Events emitted** + +| Event | When | +|-------|------| +| `INVARIANT_VIOLATED` | Each violated invariant ID supplied to `enforce_invariants()` | +| `INVARIANT_ENFORCED` | Each record created during enforcement | +| `INVARIANT_RECONCILED` | Once per enforcement batch summarising the reconciliation result | ### `InvariantReconciliationActor` ```python from cleveragents.actor.reconciliation import InvariantReconciliationActor -actor = InvariantReconciliationActor(invariant_service, event_bus, audit_service) -result = await actor.reconcile(plan_id) +actor = InvariantReconciliationActor( + invariant_service=container.invariant_service(), + decision_service=container.decision_service(), +) +result = actor.run( + plan_id="01ARZ3…", + project_name="local/my-app", + action_name="builtin/plan-execute", +) ``` -| Parameter | Type | Description | -|-----------|------|-------------| -| `invariant_service` | `InvariantService` | Provides invariant CRUD and scope queries | -| `event_bus` | `EventBus` | Receives `INVARIANT_RECONCILED` / `INVARIANT_VIOLATED` events | -| `audit_service` | `AuditService` | Records reconciliation decisions | +The actor is registered as `builtin/invariant-reconciliation` in the actor +registry and is invoked automatically by `PlanLifecycleService`. A coroutine +variant `reconcile()` is also available when the full event/decision pipeline is +needed. + +**`run()` return value** + +| Field | Type | Description | +|-------|------|-------------| +| `reconciled_set` | `InvariantSet` | Effective invariant set after precedence resolution | +| `conflicts` | `list[ConflictRecord]` | Conflicts detected and their resolutions | +| `violated_ids` | `list[str]` | Identifiers of invariants that were violated | ### `ReconciliationResult` @@ -95,41 +159,176 @@ class ReconciliationResult: class ReconciliationDecision: winner: Invariant loser: Invariant - rationale: str # e.g. "plan scope > project scope" + rationale: str # e.g. "plan scope > project scope" timestamp: datetime ``` --- -## Error Handling +## Reconciliation Algorithm -| Exception | When raised | -|-----------|-------------| -| `ReconciliationBlockedError` | Unresolvable conflict detected; phase transition blocked | -| `InvariantLoadError` | Database error loading invariants | -| `ReconciliationTimeoutError` | Reconciliation exceeded configured timeout | +The reconciliation algorithm (see **Invariant Reconciliation Algorithm** in the +specification) performs: -When `ReconciliationBlockedError` is raised: +1. **Collect** — load invariants from all scopes (global → project → action → plan). +2. **Normalise** — canonicalise text to group duplicates (case and whitespace agnostic). +3. **Detect** — flag conflicts between invariants that constrain the same behaviour. +4. **Resolve** — apply precedence (`plan > action > project > global`), honouring + `non_overridable` global invariants. +5. **Record** — emit `invariant_enforced` decisions and audit entries. +6. **Emit** — publish `INVARIANT_RECONCILED` and any `INVARIANT_VIOLATED` events. -1. The phase transition method re-raises the exception to the caller -2. `PlanLifecycleService` emits `INVARIANT_VIOLATED` on the event bus -3. The plan remains in its current phase (e.g., `PENDING` before Strategize) -4. The operator must correct the conflicting invariants and retry the transition +```python +invariants = service.get_effective_invariants(plan_id, project_name) + +records = service.enforce_invariants( + plan_id=plan_id, + invariants=invariants, + actor_response=actor_response_text, + violated_invariant_ids=["inv-001", "inv-002"], +) +# → emits INVARIANT_ENFORCED / INVARIANT_RECONCILED +``` --- -## DI Registration +## Failure Behaviour -`InvariantService` is registered as a **Singleton** in the DI container: +When reconciliation detects a violation: + +1. `ReconciliationBlockedError` is raised and the phase transition aborts. +2. `INVARIANT_VIOLATED` is emitted for each violated invariant. +3. The plan remains in its previous phase until the offending invariants are + corrected and reconciliation succeeds. + +```python +from cleveragents.domain.exceptions import ReconciliationBlockedError + +try: + lifecycle_service.start_execute(plan_id) +except ReconciliationBlockedError as exc: + logger.error("Phase transition blocked", violated=exc.violated_invariant_ids) +``` + +Additional operational failures (`InvariantLoadError`, `ReconciliationTimeoutError`) are +propagated to callers so that the lifecycle service can surface actionable +errors to operators. + +--- + +## Managing Invariants via CLI + +```bash +# Add a global invariant +agents invariant add --scope global \ + "All output files must be UTF-8 encoded" + +# Add a project-scoped invariant +agents invariant add --scope project --project local/my-app \ + "Never modify files outside src/" + +# Add a plan-scoped invariant (non-overridable) +agents invariant add --scope plan --plan 01ARZ3... --non-overridable \ + "Do not delete any existing tests" + +# List effective invariants for a plan +agents invariant list --plan 01ARZ3... --effective + +# Remove an invariant (soft-delete) +agents invariant remove inv-001 +``` + +--- + +## Domain Model + +### `Invariant` + +```python +from cleveragents.domain.models.core.invariant import Invariant, InvariantScope + +invariant = Invariant( + text="All output files must be UTF-8 encoded", + scope=InvariantScope.PROJECT, + source_name="local/my-app", + non_overridable=False, +) +``` + +| Field | Type | Description | +|-------|------|-------------| +| `invariant_id` | `str` (ULID) | Unique identifier | +| `text` | `str` (≤ 2,000 chars) | Constraint description | +| `scope` | `InvariantScope` | `global`, `project`, `action`, or `plan` | +| `source_name` | `str` | Project name, action name, or plan ID (scope dependent) | +| `non_overridable` | `bool` | Prevents higher-specificity overrides when `True` | +| `active` | `bool` | `False` after soft-delete | +| `created_at` | `datetime` (UTC) | Creation timestamp | + +### `InvariantScope` + +```python +InvariantScope.GLOBAL +InvariantScope.PROJECT +InvariantScope.ACTION +InvariantScope.PLAN +``` + +### `InvariantSet` + +```python +inv_set = InvariantSet(invariants=[inv1, inv2, inv3]) +effective = inv_set.get_effective() # list[Invariant] after precedence resolution +``` + +### `ConflictRecord` + +```python +record = ConflictRecord( + text="Never modify files outside src/", + winner_scope=InvariantScope.PLAN, + loser_scope=InvariantScope.PROJECT, + resolution="plan-scope invariant takes precedence", +) +``` + +--- + +## Merge Precedence Helper + +```python +from cleveragents.domain.models.core.invariant import merge_invariants + +merged = merge_invariants( + global_invariants=[...], + project_invariants=[...], + action_invariants=[...], + plan_invariants=[...], +) +``` + +Rules applied by `merge_invariants()`: + +1. De-duplicate invariants using normalised text. +2. Apply precedence `plan > action > project > global`. +3. Honour `non_overridable=True` global invariants regardless of scope. +4. Record conflicts as `ConflictRecord` instances for auditability. + +--- + +## Dependency Injection + +`InvariantService` is registered as a singleton in the application container and +the actor is exposed as a factory-bound dependency: ```python -# src/cleveragents/application/container.py (excerpt) class Container(containers.DeclarativeContainer): invariant_service = providers.Singleton( InvariantService, uow_factory=uow_factory, event_bus=event_bus, ) + invariant_reconciliation_actor = providers.Factory( InvariantReconciliationActor, invariant_service=invariant_service, @@ -138,65 +337,79 @@ class Container(containers.DeclarativeContainer): ) ``` +Always obtain services through the container to share caches and maintain +cross-cutting instrumentation. + --- ## Usage Examples -### Triggering Reconciliation Manually +### Trigger reconciliation manually ```python -from cleveragents.application.container import Container - -container = Container() actor = container.invariant_reconciliation_actor() result = await actor.reconcile(plan_id="plan-01JXYZ") if not result.success: - print(f"Reconciliation failed: {result.conflicts_detected} unresolvable conflicts") + logger.error( + "Reconciliation failed", + conflicts=result.conflicts_detected, + ) else: - print(f"Reconciled {result.conflicts_resolved} conflicts") for decision in result.decisions: - print(f" {decision.winner.text!r} won over {decision.loser.text!r}: {decision.rationale}") + logger.info( + "%r beat %r: %s", + decision.winner.text, + decision.loser.text, + decision.rationale, + ) ``` -### Listening for Reconciliation Events +### Listen for reconciliation events ```python -from cleveragents.domain.events import INVARIANT_RECONCILED, INVARIANT_VIOLATED - @event_bus.subscribe(INVARIANT_RECONCILED) async def on_reconciled(event): - logger.info("Invariants reconciled", plan_id=event.plan_id, decisions=event.decision_count) + metrics.record("invariant.decisions", event.decision_count) @event_bus.subscribe(INVARIANT_VIOLATED) async def on_violated(event): - logger.error("Invariant violation blocked phase transition", plan_id=event.plan_id) - # Notify operator, create alert, etc. + alerts.raise_("plan-blocked", plan_id=event.plan_id) ``` -### Defining Invariants at Different Scopes +--- -```bash -# Global invariant (applies to all plans) -agents invariant create --scope global --text "Never delete production data without a backup" +## Testing -# Project-level invariant -agents invariant create --scope project --project my-project \ - --text "All database migrations must be reversible" +The invariant system is covered by: +- Behave scenarios in `features/invariant_enforcement.feature` +- Robot Framework suites in `robot/invariants.robot` -# Plan-level invariant (highest precedence) -agents invariant create --scope plan --plan plan-01JXYZ \ - --text "This plan may drop the staging schema" +```python +service = InvariantService(event_bus=FakeEventBus()) + +# Add invariants +service.add_invariant( + text="All output files must be UTF-8 encoded", + scope=InvariantScope.PROJECT, + source_name="local/my-app", +) + +# Retrieve effective set +effective = service.get_effective_invariants( + plan_id="01ARZ3…", + project_name="local/my-app", +) +assert any(i.text.endswith("UTF-8 encoded") for i in effective) ``` -In this example, if the plan-level invariant conflicts with the project-level one, -the plan-level invariant wins (higher precedence). - --- ## Related Documentation -- [ADR-016 Invariant System](../adr/ADR-016-invariant-system.md) — design rationale -- [ADR-007 Decision Tree & Correction](../adr/ADR-007-decision-tree-and-correction.md) — correction flow -- [Core API Reference](../api/core.md) — `InvariantService` API -- [Shell Safety Module](shell-safety.md) — another built-in safety actor +- [ADR-016 Invariant System](../adr/ADR-016-invariant-system.md) +- [ADR-007 Decision Tree & Correction](../adr/ADR-007-decision-tree-and-correction.md) +- [Architecture — Invariant Reconciliation](../architecture.md#invariant-reconciliation) +- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) +- [Core API Reference — Invariant Service](../api/core.md#invariant-service) +- [Shell Safety Module](shell-safety.md) diff --git a/docs/modules/sandbox.md b/docs/modules/sandbox.md new file mode 100644 index 000000000..3a000852b --- /dev/null +++ b/docs/modules/sandbox.md @@ -0,0 +1,381 @@ +# Sandbox & Checkpoint Module + +**Package:** `cleveragents.infrastructure.sandbox` +**Introduced:** M4 (v3.3.0); atomic commit added in M6 (v3.5.0, issue #925) + +The sandbox module provides isolated write environments for plan execution. +Every resource write during a plan's Execute phase is routed through a sandbox +so that changes can be reviewed, rolled back, or atomically committed during +the Apply phase. + +For the design rationale see +[ADR-015](../adr/ADR-015-sandbox-and-checkpoint.md) (Sandbox & Checkpoint) and +[ADR-038](../adr/ADR-038-cross-mechanism-sandbox-coordination.md) +(Cross-Mechanism Sandbox Coordination). + +--- + +## Purpose + +When a plan executes, actors write to resources (files, databases, git +repositories). Without sandboxing, every write would immediately modify the +original resource — making rollback impossible and preventing the diff-review +step in the Apply phase. + +The sandbox module solves this by: + +1. **Intercepting writes** — all resource paths are translated to sandbox + copies before the actor touches them. +2. **Enabling rollback** — if execution fails or the user rejects the diff, + the sandbox is discarded and the original resource is untouched. +3. **Atomic commit** — when the user approves the Apply phase, all sandboxes + for the plan are committed in an all-or-nothing batch. + +--- + +## Sandbox Strategies + +| Strategy | Class | Use case | +|----------|-------|---------| +| `git_worktree` | `GitWorktreeSandbox` | Git repositories — changes tracked as a worktree branch | +| `copy_on_write` | `CopyOnWriteSandbox` | Filesystem directories — full directory copy | +| `overlay` | `OverlaySandbox` | Linux OverlayFS — efficient layered filesystem (falls back to copy-on-write) | +| `transaction_rollback` | `TransactionSandbox` | Databases — wraps writes in a transaction | +| `none` | `NoSandbox` | Resources that cannot be sandboxed (writes are immediate) | + +The strategy is selected per-resource via the `sandbox_strategy` field on the +resource definition. The `SandboxFactory` instantiates the correct class. + +--- + +## Lifecycle + +``` +PENDING ──► CREATED ──► ACTIVE ──► COMMITTED ──► CLEANED_UP + │ │ + │ └──► ROLLED_BACK ──► ACTIVE (re-use) + │ └──► CLEANED_UP + └──► ERRORED ──► CLEANED_UP +``` + +| Status | Meaning | +|--------|---------| +| `PENDING` | Sandbox object created but not yet initialised | +| `CREATED` | Sandbox environment exists; not yet activated for writes | +| `ACTIVE` | Sandbox is ready; actor writes are routed here | +| `COMMITTED` | Changes applied to the original resource | +| `ROLLED_BACK` | Changes discarded; sandbox may be re-activated | +| `CLEANED_UP` | All artefacts removed; terminal state | +| `ERRORED` | An unrecoverable error occurred | + +--- + +## Core Classes + +### `Sandbox` (Protocol) + +```python +from cleveragents.infrastructure.sandbox.protocol import Sandbox +``` + +Runtime-checkable protocol that all sandbox implementations satisfy. + +```python +# Typical usage in plan execution +sandbox = manager.get_or_create_sandbox( + plan_id="01ARZ3...", + resource_id="res-001", + original_path="/path/to/repo", + sandbox_strategy="git_worktree", +) + +# Translate a resource-relative path to the sandbox copy +sandboxed_path = sandbox.get_path("src/main.py") +# ... actor writes to sandboxed_path ... + +# Commit changes back to the original resource +result = sandbox.commit("Apply edits from plan 01ARZ3...") +# result.success, result.changed_files, result.added_files, result.deleted_files + +# Or discard changes +sandbox.rollback() + +# Always clean up when done +sandbox.cleanup() +``` + +**Key methods:** + +| Method | Description | +|--------|-------------| +| `create(plan_id) → SandboxContext` | Initialise the sandbox environment | +| `get_path(resource_path) → str` | Translate a resource-relative path to an absolute sandbox path | +| `commit(message=None) → CommitResult` | Apply changes to the original resource | +| `rollback() → None` | Discard all changes | +| `cleanup() → None` | Remove all sandbox artefacts (idempotent) | + +**Key properties:** + +| Property | Type | Description | +|----------|------|-------------| +| `sandbox_id` | `str` | Unique ULID identifier | +| `status` | `SandboxStatus` | Current lifecycle status | +| `context` | `SandboxContext \| None` | Context after `create()`, `None` before | + +--- + +### `SandboxManager` + +```python +from cleveragents.infrastructure.sandbox.manager import SandboxManager +``` + +Thread-safe manager for sandbox lifecycles across plan executions. Provides +lazy creation, batch operations, and automatic cleanup on process exit. + +Obtain via the DI container: + +```python +manager = container.sandbox_manager() +``` + +**Key methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `get_or_create_sandbox(plan_id, resource_id, original_path, sandbox_strategy)` | `Sandbox` | Return existing or create new sandbox (lazy) | +| `get_sandbox(plan_id, resource_id)` | `Sandbox \| None` | Look up without creating | +| `list_sandboxes(plan_id)` | `list[Sandbox]` | All sandboxes for a plan | +| `commit_all(plan_id)` | `list[CommitResult]` | Atomically commit all active sandboxes | +| `rollback_all(plan_id)` | `None` | Roll back all active sandboxes | +| `cleanup_all(plan_id)` | `None` | Clean up all sandboxes and remove tracking | +| `cleanup_abandoned()` | `int` | Clean up terminal-state sandboxes; returns count | +| `get_or_create_sandbox_for_resource(plan_id, resource, resource_registry)` | `Sandbox` | Boundary-aware sandbox creation | +| `resolve_sandbox_key(resource, resource_registry)` | `str` | Compute the sandbox boundary key for a resource | +| `clear_boundary_cache()` | `None` | Clear the boundary algebra cache | + +--- + +### `SandboxContext` + +```python +from cleveragents.infrastructure.sandbox.protocol import SandboxContext +``` + +Immutable frozen dataclass describing an initialised sandbox. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `sandbox_id` | `str` | Unique ULID identifier | +| `sandbox_path` | `str` | Root path where sandboxed files reside | +| `original_path` | `str` | Original resource location | +| `resource_id` | `str` | ID of the resource being sandboxed | +| `plan_id` | `str` | ID of the owning plan | +| `created_at` | `datetime` | When the sandbox was initialised | +| `metadata` | `dict[str, Any]` | Implementation-specific data (e.g. git branch name) | + +--- + +### `CommitResult` + +```python +from cleveragents.infrastructure.sandbox.protocol import CommitResult +``` + +Immutable frozen dataclass describing the outcome of a `commit()` call. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `sandbox_id` | `str` | Which sandbox was committed | +| `success` | `bool` | Whether the commit succeeded | +| `commit_ref` | `str \| None` | Git commit hash or equivalent, if applicable | +| `changed_files` | `list[str]` | Modified file paths | +| `added_files` | `list[str]` | Created file paths | +| `deleted_files` | `list[str]` | Removed file paths | +| `error` | `str \| None` | Error message when `success=False` | +| `timestamp` | `datetime` | When the commit occurred | +| `metadata` | `dict[str, Any]` | Arbitrary key-value metadata | + +--- + +## Atomic Commit (`commit_all`) + +`SandboxManager.commit_all()` implements the all-or-nothing atomicity +requirement from the specification (see **Atomic Commit Semantics** in the +Sandbox & Checkpoint section). + +**Algorithm:** + +1. Collect all sandboxes in `CREATED` or `ACTIVE` status for the plan. +2. Separate into *rollbackable* (git, copy-on-write, overlay) and + *non-rollbackable* (none, transaction) groups. +3. Commit rollbackable sandboxes first; non-rollbackable last. +4. If any commit fails, roll back all previously-committed sandboxes in + **LIFO** order (most recently committed first). +5. Return a list of `CommitResult` objects (all `success=True`) on success, + or a single `CommitResult(success=False)` on failure. + +**Error handling:** + +| Exception | Meaning | +|-----------|---------| +| `SandboxError` | A sandbox-level commit failure; rollback attempted; `CommitResult(success=False)` returned | +| `AtomicCommitError` | A non-sandbox exception during commit; rollback attempted; exception raised with `rolled_back_ids` and `failed_rollback_ids` attributes | + +```python +from cleveragents.infrastructure.sandbox.protocol import AtomicCommitError + +try: + results = manager.commit_all(plan_id) + if not all(r.success for r in results): + # Partial failure (SandboxError path) + failed = [r for r in results if not r.success] + print(f"Commit failed: {failed[0].error}") +except AtomicCommitError as exc: + # Unexpected exception during commit + print(f"Atomic commit failed: {exc}") + print(f"Rolled back: {exc.rolled_back_ids}") + print(f"Failed rollbacks: {exc.failed_rollback_ids}") +``` + +--- + +## Boundary Algebra + +The sandbox boundary algebra (introduced in M6, issue #548) determines which +sandbox governs a given resource. Resources that share a common ancestor +boundary share the same sandbox instance. + +```python +# Resources in the same git repo share one sandbox +sandbox_key = manager.resolve_sandbox_key(resource, resource_registry) +# → the resource_id of the boundary resource (e.g. the repo root) + +# Multiple resources → same sandbox +sandbox_a = manager.get_or_create_sandbox_for_resource(plan_id, file_a, registry) +sandbox_b = manager.get_or_create_sandbox_for_resource(plan_id, file_b, registry) +assert sandbox_a is sandbox_b # same boundary → same sandbox +``` + +Call `manager.clear_boundary_cache()` at the start of each plan execution +or when the resource DAG changes. + +--- + +## Exception Hierarchy + +``` +SandboxError +├── SandboxCreationError — sandbox cannot be initialised +├── SandboxCommitError — commit operation failed +├── SandboxRollbackError — rollback operation failed +├── SandboxStateError — operation invalid for current status +└── AtomicCommitError — non-sandbox exception during commit_all + ├── rolled_back_ids: list[str] + └── failed_rollback_ids: list[str] +``` + +--- + +## Sandbox Implementations + +### `GitWorktreeSandbox` + +Uses `git worktree add` to create an isolated branch for the plan. Changes +are committed to the worktree branch and then merged (or cherry-picked) back +to the original branch on `commit()`. + +- **Checkpoint:** `git tag` on the worktree branch +- **Rollback from COMMITTED:** `git reset --hard` to the pre-commit tag + +### `CopyOnWriteSandbox` + +Creates a full directory copy of the resource. Writes go to the copy; on +`commit()` the copy is merged back to the original using rename-based swap +(O(1) on the same filesystem). + +- **Checkpoint:** `shutil.copytree` snapshot +- **Rollback from COMMITTED:** restore from pre-commit backup + +### `OverlaySandbox` + +Uses Linux OverlayFS to create a layered view of the resource. The original +directory is the lower layer; writes go to the upper layer. Falls back to +copy-on-write when OverlayFS is unavailable. + +- **Checkpoint:** snapshot of the upper layer +- **Rollback from COMMITTED:** unmount and remount OverlayFS + +### `TransactionSandbox` + +Wraps database writes in a transaction. `commit()` issues `COMMIT`; +`rollback()` issues `ROLLBACK`. **Cannot be rolled back after `COMMIT`.** + +### `NoSandbox` + +Pass-through — writes go directly to the original resource. Used for +resources that cannot be sandboxed (e.g. external APIs). **Cannot be +rolled back.** + +!!! danger "Security Warning: NoSandbox bypasses all isolation" + `NoSandbox` writes are **immediately permanent** and **cannot be rolled + back**. The Apply-phase diff review is skipped entirely. Only choose + `NoSandbox` for resources that are inherently non-sandboxable (for + example, external API calls with guaranteed idempotency). Never use + `NoSandbox` for filesystem or database resources where rollback or + sandbox inspection may be required. + +--- + +## Filesystem Utilities + +**Module:** `cleveragents.infrastructure.sandbox._fs_utils` + +Shared utilities used by `CopyOnWriteSandbox` and `OverlaySandbox`: + +| Function | Description | +|----------|-------------| +| `backup_directory(src, dst)` | Copy a directory tree preserving permissions, timestamps, and symlinks | +| `safe_restore(backup, original)` | Rename-based atomic restore (backup → original) | +| `compute_diff(original, modified)` | Compute a unified diff between two directory trees | + +--- + +## Status Transition Validation + +```python +from cleveragents.infrastructure.sandbox.protocol import SandboxStatus + +# Check if a transition is valid +assert SandboxStatus.can_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED) +assert not SandboxStatus.can_transition(SandboxStatus.CLEANED_UP, SandboxStatus.ACTIVE) + +# Assert transition (raises SandboxStateError on invalid) +SandboxStatus.assert_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED) +``` + +--- + +## Testing + +The sandbox module is covered by BDD scenarios in `features/sandbox.feature` +and Robot Framework integration tests in `robot/sandbox.robot`. + +```python +from cleveragents.infrastructure.sandbox.protocol import Sandbox, SandboxStatus + +# Verify protocol conformance +assert isinstance(my_sandbox, Sandbox) + +# Verify status +assert my_sandbox.status == SandboxStatus.ACTIVE +``` + +--- + +## Related Documentation + +- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md) +- [ADR-038 Cross-Mechanism Sandbox Coordination](../adr/ADR-038-cross-mechanism-sandbox-coordination.md) +- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) +- [API — Resource System](../api/resource.md) diff --git a/mkdocs.yml b/mkdocs.yml index 07a5aba3a..3e33c3582 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,6 +25,8 @@ nav: - Modules: - Shell Safety: modules/shell-safety.md - UKO Provenance Tracking: modules/uko-provenance.md + - Sandbox & Checkpoint: modules/sandbox.md + - Correction Attempts: modules/correction-attempts.md - Invariant Reconciliation: modules/invariant-reconciliation.md - ACMS Context Hydration: modules/context-hydration.md - Git Worktree Sandbox: modules/git-worktree-sandbox.md