cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity
Files
master
cleveragents-core/docs/modules/sandbox.md
T
HAL9000 44652c216a docs: add sandbox, correction, and invariant module guides 
Document lifecycle details, security warnings, and CLI usage for the new module guides so implementors have a single reference.

ISSUES CLOSED: #4848
2026-06-16 18:18:18 -04:00

382 lines
13 KiB
Markdown
Raw Permalink Blame History

# Sandbox & Checkpoint Module
**Package:** `cleveragents.infrastructure.sandbox`
**Introduced:** M4 (v3.3.0); atomic commit added in M6 (v3.5.0, issue #925)
The sandbox module provides isolated write environments for plan execution.
Every resource write during a plan's Execute phase is routed through a sandbox
so that changes can be reviewed, rolled back, or atomically committed during
the Apply phase.
For the design rationale see
[ADR-015](../adr/ADR-015-sandbox-and-checkpoint.md) (Sandbox & Checkpoint) and
[ADR-038](../adr/ADR-038-cross-mechanism-sandbox-coordination.md)
(Cross-Mechanism Sandbox Coordination).
---
## Purpose
When a plan executes, actors write to resources (files, databases, git
repositories). Without sandboxing, every write would immediately modify the
original resource — making rollback impossible and preventing the diff-review
step in the Apply phase.
The sandbox module solves this by:
1. **Intercepting writes** — all resource paths are translated to sandbox
copies before the actor touches them.
2. **Enabling rollback** — if execution fails or the user rejects the diff,
the sandbox is discarded and the original resource is untouched.
3. **Atomic commit** — when the user approves the Apply phase, all sandboxes
for the plan are committed in an all-or-nothing batch.
---
## Sandbox Strategies
| Strategy | Class | Use case |
|----------|-------|---------|
| `git_worktree` | `GitWorktreeSandbox` | Git repositories — changes tracked as a worktree branch |
| `copy_on_write` | `CopyOnWriteSandbox` | Filesystem directories — full directory copy |
| `overlay` | `OverlaySandbox` | Linux OverlayFS — efficient layered filesystem (falls back to copy-on-write) |
| `transaction_rollback` | `TransactionSandbox` | Databases — wraps writes in a transaction |
| `none` | `NoSandbox` | Resources that cannot be sandboxed (writes are immediate) |
The strategy is selected per-resource via the `sandbox_strategy` field on the
resource definition. The `SandboxFactory` instantiates the correct class.
---
## Lifecycle
```
PENDING ──► CREATED ──► ACTIVE ──► COMMITTED ──► CLEANED_UP
│ │
│ └──► ROLLED_BACK ──► ACTIVE (re-use)
│ └──► CLEANED_UP
└──► ERRORED ──► CLEANED_UP
```
| Status | Meaning |
|--------|---------|
| `PENDING` | Sandbox object created but not yet initialised |
| `CREATED` | Sandbox environment exists; not yet activated for writes |
| `ACTIVE` | Sandbox is ready; actor writes are routed here |
| `COMMITTED` | Changes applied to the original resource |
| `ROLLED_BACK` | Changes discarded; sandbox may be re-activated |
| `CLEANED_UP` | All artefacts removed; terminal state |
| `ERRORED` | An unrecoverable error occurred |
---
## Core Classes
### `Sandbox` (Protocol)
```python
from cleveragents.infrastructure.sandbox.protocol import Sandbox
```
Runtime-checkable protocol that all sandbox implementations satisfy.
```python
# Typical usage in plan execution
sandbox = manager.get_or_create_sandbox(
plan_id="01ARZ3...",
resource_id="res-001",
original_path="/path/to/repo",
sandbox_strategy="git_worktree",
)
# Translate a resource-relative path to the sandbox copy
sandboxed_path = sandbox.get_path("src/main.py")
# ... actor writes to sandboxed_path ...
# Commit changes back to the original resource
result = sandbox.commit("Apply edits from plan 01ARZ3...")
# result.success, result.changed_files, result.added_files, result.deleted_files
# Or discard changes
sandbox.rollback()
# Always clean up when done
sandbox.cleanup()
```
**Key methods:**
| Method | Description |
|--------|-------------|
| `create(plan_id) → SandboxContext` | Initialise the sandbox environment |
| `get_path(resource_path) → str` | Translate a resource-relative path to an absolute sandbox path |
| `commit(message=None) → CommitResult` | Apply changes to the original resource |
| `rollback() → None` | Discard all changes |
| `cleanup() → None` | Remove all sandbox artefacts (idempotent) |
**Key properties:**
| Property | Type | Description |
|----------|------|-------------|
| `sandbox_id` | `str` | Unique ULID identifier |
| `status` | `SandboxStatus` | Current lifecycle status |
| `context` | `SandboxContext \| None` | Context after `create()`, `None` before |
---
### `SandboxManager`
```python
from cleveragents.infrastructure.sandbox.manager import SandboxManager
```
Thread-safe manager for sandbox lifecycles across plan executions. Provides
lazy creation, batch operations, and automatic cleanup on process exit.
Obtain via the DI container:
```python
manager = container.sandbox_manager()
```
**Key methods:**
| Method | Returns | Description |
|--------|---------|-------------|
| `get_or_create_sandbox(plan_id, resource_id, original_path, sandbox_strategy)` | `Sandbox` | Return existing or create new sandbox (lazy) |
| `get_sandbox(plan_id, resource_id)` | `Sandbox \| None` | Look up without creating |
| `list_sandboxes(plan_id)` | `list[Sandbox]` | All sandboxes for a plan |
| `commit_all(plan_id)` | `list[CommitResult]` | Atomically commit all active sandboxes |
| `rollback_all(plan_id)` | `None` | Roll back all active sandboxes |
| `cleanup_all(plan_id)` | `None` | Clean up all sandboxes and remove tracking |
| `cleanup_abandoned()` | `int` | Clean up terminal-state sandboxes; returns count |
| `get_or_create_sandbox_for_resource(plan_id, resource, resource_registry)` | `Sandbox` | Boundary-aware sandbox creation |
| `resolve_sandbox_key(resource, resource_registry)` | `str` | Compute the sandbox boundary key for a resource |
| `clear_boundary_cache()` | `None` | Clear the boundary algebra cache |
---
### `SandboxContext`
```python
from cleveragents.infrastructure.sandbox.protocol import SandboxContext
```
Immutable frozen dataclass describing an initialised sandbox.
| Attribute | Type | Description |
|-----------|------|-------------|
| `sandbox_id` | `str` | Unique ULID identifier |
| `sandbox_path` | `str` | Root path where sandboxed files reside |
| `original_path` | `str` | Original resource location |
| `resource_id` | `str` | ID of the resource being sandboxed |
| `plan_id` | `str` | ID of the owning plan |
| `created_at` | `datetime` | When the sandbox was initialised |
| `metadata` | `dict[str, Any]` | Implementation-specific data (e.g. git branch name) |
---
### `CommitResult`
```python
from cleveragents.infrastructure.sandbox.protocol import CommitResult
```
Immutable frozen dataclass describing the outcome of a `commit()` call.
| Attribute | Type | Description |
|-----------|------|-------------|
| `sandbox_id` | `str` | Which sandbox was committed |
| `success` | `bool` | Whether the commit succeeded |
| `commit_ref` | `str \| None` | Git commit hash or equivalent, if applicable |
| `changed_files` | `list[str]` | Modified file paths |
| `added_files` | `list[str]` | Created file paths |
| `deleted_files` | `list[str]` | Removed file paths |
| `error` | `str \| None` | Error message when `success=False` |
| `timestamp` | `datetime` | When the commit occurred |
| `metadata` | `dict[str, Any]` | Arbitrary key-value metadata |
---
## Atomic Commit (`commit_all`)
`SandboxManager.commit_all()` implements the all-or-nothing atomicity
requirement from the specification (see **Atomic Commit Semantics** in the
Sandbox & Checkpoint section).
**Algorithm:**
1. Collect all sandboxes in `CREATED` or `ACTIVE` status for the plan.
2. Separate into *rollbackable* (git, copy-on-write, overlay) and
*non-rollbackable* (none, transaction) groups.
3. Commit rollbackable sandboxes first; non-rollbackable last.
4. If any commit fails, roll back all previously-committed sandboxes in
**LIFO** order (most recently committed first).
5. Return a list of `CommitResult` objects (all `success=True`) on success,
or a single `CommitResult(success=False)` on failure.
**Error handling:**
| Exception | Meaning |
|-----------|---------|
| `SandboxError` | A sandbox-level commit failure; rollback attempted; `CommitResult(success=False)` returned |
| `AtomicCommitError` | A non-sandbox exception during commit; rollback attempted; exception raised with `rolled_back_ids` and `failed_rollback_ids` attributes |
```python
from cleveragents.infrastructure.sandbox.protocol import AtomicCommitError
try:
results = manager.commit_all(plan_id)
if not all(r.success for r in results):
# Partial failure (SandboxError path)
failed = [r for r in results if not r.success]
print(f"Commit failed: {failed[0].error}")
except AtomicCommitError as exc:
# Unexpected exception during commit
print(f"Atomic commit failed: {exc}")
print(f"Rolled back: {exc.rolled_back_ids}")
print(f"Failed rollbacks: {exc.failed_rollback_ids}")
```
---
## Boundary Algebra
The sandbox boundary algebra (introduced in M6, issue #548) determines which
sandbox governs a given resource. Resources that share a common ancestor
boundary share the same sandbox instance.
```python
# Resources in the same git repo share one sandbox
sandbox_key = manager.resolve_sandbox_key(resource, resource_registry)
# → the resource_id of the boundary resource (e.g. the repo root)
# Multiple resources → same sandbox
sandbox_a = manager.get_or_create_sandbox_for_resource(plan_id, file_a, registry)
sandbox_b = manager.get_or_create_sandbox_for_resource(plan_id, file_b, registry)
assert sandbox_a is sandbox_b # same boundary → same sandbox
```
Call `manager.clear_boundary_cache()` at the start of each plan execution
or when the resource DAG changes.
---
## Exception Hierarchy
```
SandboxError
├── SandboxCreationError — sandbox cannot be initialised
├── SandboxCommitError — commit operation failed
├── SandboxRollbackError — rollback operation failed
├── SandboxStateError — operation invalid for current status
└── AtomicCommitError — non-sandbox exception during commit_all
├── rolled_back_ids: list[str]
└── failed_rollback_ids: list[str]
```
---
## Sandbox Implementations
### `GitWorktreeSandbox`
Uses `git worktree add` to create an isolated branch for the plan. Changes
are committed to the worktree branch and then merged (or cherry-picked) back
to the original branch on `commit()`.
- **Checkpoint:** `git tag` on the worktree branch
- **Rollback from COMMITTED:** `git reset --hard` to the pre-commit tag
### `CopyOnWriteSandbox`
Creates a full directory copy of the resource. Writes go to the copy; on
`commit()` the copy is merged back to the original using rename-based swap
(O(1) on the same filesystem).
- **Checkpoint:** `shutil.copytree` snapshot
- **Rollback from COMMITTED:** restore from pre-commit backup
### `OverlaySandbox`
Uses Linux OverlayFS to create a layered view of the resource. The original
directory is the lower layer; writes go to the upper layer. Falls back to
copy-on-write when OverlayFS is unavailable.
- **Checkpoint:** snapshot of the upper layer
- **Rollback from COMMITTED:** unmount and remount OverlayFS
### `TransactionSandbox`
Wraps database writes in a transaction. `commit()` issues `COMMIT`;
`rollback()` issues `ROLLBACK`. **Cannot be rolled back after `COMMIT`.**
### `NoSandbox`
Pass-through — writes go directly to the original resource. Used for
resources that cannot be sandboxed (e.g. external APIs). **Cannot be
rolled back.**
!!! danger "Security Warning: NoSandbox bypasses all isolation"
`NoSandbox` writes are **immediately permanent** and **cannot be rolled
back**. The Apply-phase diff review is skipped entirely. Only choose
`NoSandbox` for resources that are inherently non-sandboxable (for
example, external API calls with guaranteed idempotency). Never use
`NoSandbox` for filesystem or database resources where rollback or
sandbox inspection may be required.
---
## Filesystem Utilities
**Module:** `cleveragents.infrastructure.sandbox._fs_utils`
Shared utilities used by `CopyOnWriteSandbox` and `OverlaySandbox`:
| Function | Description |
|----------|-------------|
| `backup_directory(src, dst)` | Copy a directory tree preserving permissions, timestamps, and symlinks |
| `safe_restore(backup, original)` | Rename-based atomic restore (backup → original) |
| `compute_diff(original, modified)` | Compute a unified diff between two directory trees |
---
## Status Transition Validation
```python
from cleveragents.infrastructure.sandbox.protocol import SandboxStatus
# Check if a transition is valid
assert SandboxStatus.can_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)
assert not SandboxStatus.can_transition(SandboxStatus.CLEANED_UP, SandboxStatus.ACTIVE)
# Assert transition (raises SandboxStateError on invalid)
SandboxStatus.assert_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)
```
---
## Testing
The sandbox module is covered by BDD scenarios in `features/sandbox.feature`
and Robot Framework integration tests in `robot/sandbox.robot`.
```python
from cleveragents.infrastructure.sandbox.protocol import Sandbox, SandboxStatus
# Verify protocol conformance
assert isinstance(my_sandbox, Sandbox)
# Verify status
assert my_sandbox.status == SandboxStatus.ACTIVE
```
---
## Related Documentation
- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md)
- [ADR-038 Cross-Mechanism Sandbox Coordination](../adr/ADR-038-cross-mechanism-sandbox-coordination.md)
- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle)
- [API — Resource System](../api/resource.md)
Reference in New Issue View Git Blame Copy Permalink