# Sandbox & Checkpoint Module

**Package:** `cleveragents.infrastructure.sandbox`
**Introduced:** M4 (v3.3.0); atomic commit added in M6 (v3.5.0, issue #925)

The sandbox module provides isolated write environments for plan execution.
Every resource write during a plan's Execute phase is routed through a sandbox
so that changes can be reviewed, rolled back, or atomically committed during
the Apply phase.

For the design rationale see
[ADR-015](../adr/ADR-015-sandbox-and-checkpoint.md) (Sandbox & Checkpoint) and
[ADR-038](../adr/ADR-038-cross-mechanism-sandbox-coordination.md)
(Cross-Mechanism Sandbox Coordination).

---

## Purpose

When a plan executes, actors write to resources (files, databases, git
repositories).  Without sandboxing, every write would immediately modify the
original resource — making rollback impossible and preventing the diff-review
step in the Apply phase.

The sandbox module solves this by:

1. **Intercepting writes** — all resource paths are translated to sandbox
   copies before the actor touches them.
2. **Enabling rollback** — if execution fails or the user rejects the diff,
   the sandbox is discarded and the original resource is untouched.
3. **Atomic commit** — when the user approves the Apply phase, all sandboxes
   for the plan are committed in an all-or-nothing batch.

---

## Sandbox Strategies

| Strategy | Class | Use case |
|----------|-------|---------|
| `git_worktree` | `GitWorktreeSandbox` | Git repositories — changes tracked as a worktree branch |
| `copy_on_write` | `CopyOnWriteSandbox` | Filesystem directories — full directory copy |
| `overlay` | `OverlaySandbox` | Linux OverlayFS — efficient layered filesystem (falls back to copy-on-write) |
| `transaction_rollback` | `TransactionSandbox` | Databases — wraps writes in a transaction |
| `none` | `NoSandbox` | Resources that cannot be sandboxed (writes are immediate) |

The strategy is selected per-resource via the `sandbox_strategy` field on the
resource definition.  The `SandboxFactory` instantiates the correct class.

---

## Lifecycle

```
PENDING ──► CREATED ──► ACTIVE ──► COMMITTED ──► CLEANED_UP
                │           │
                │           └──► ROLLED_BACK ──► ACTIVE (re-use)
                │                              └──► CLEANED_UP
                └──► ERRORED ──► CLEANED_UP
```

| Status | Meaning |
|--------|---------|
| `PENDING` | Sandbox object created but not yet initialised |
| `CREATED` | Sandbox environment exists; not yet activated for writes |
| `ACTIVE` | Sandbox is ready; actor writes are routed here |
| `COMMITTED` | Changes applied to the original resource |
| `ROLLED_BACK` | Changes discarded; sandbox may be re-activated |
| `CLEANED_UP` | All artefacts removed; terminal state |
| `ERRORED` | An unrecoverable error occurred |

---

## Core Classes

### `Sandbox` (Protocol)

```python
from cleveragents.infrastructure.sandbox.protocol import Sandbox
```

Runtime-checkable protocol that all sandbox implementations satisfy.

```python
# Typical usage in plan execution
sandbox = manager.get_or_create_sandbox(
    plan_id="01ARZ3...",
    resource_id="res-001",
    original_path="/path/to/repo",
    sandbox_strategy="git_worktree",
)

# Translate a resource-relative path to the sandbox copy
sandboxed_path = sandbox.get_path("src/main.py")
# ... actor writes to sandboxed_path ...

# Commit changes back to the original resource
result = sandbox.commit("Apply edits from plan 01ARZ3...")
# result.success, result.changed_files, result.added_files, result.deleted_files

# Or discard changes
sandbox.rollback()

# Always clean up when done
sandbox.cleanup()
```

**Key methods:**

| Method | Description |
|--------|-------------|
| `create(plan_id) → SandboxContext` | Initialise the sandbox environment |
| `get_path(resource_path) → str` | Translate a resource-relative path to an absolute sandbox path |
| `commit(message=None) → CommitResult` | Apply changes to the original resource |
| `rollback() → None` | Discard all changes |
| `cleanup() → None` | Remove all sandbox artefacts (idempotent) |

**Key properties:**

| Property | Type | Description |
|----------|------|-------------|
| `sandbox_id` | `str` | Unique ULID identifier |
| `status` | `SandboxStatus` | Current lifecycle status |
| `context` | `SandboxContext \| None` | Context after `create()`, `None` before |

---

### `SandboxManager`

```python
from cleveragents.infrastructure.sandbox.manager import SandboxManager
```

Thread-safe manager for sandbox lifecycles across plan executions.  Provides
lazy creation, batch operations, and automatic cleanup on process exit.

Obtain via the DI container:

```python
manager = container.sandbox_manager()
```

**Key methods:**

| Method | Returns | Description |
|--------|---------|-------------|
| `get_or_create_sandbox(plan_id, resource_id, original_path, sandbox_strategy)` | `Sandbox` | Return existing or create new sandbox (lazy) |
| `get_sandbox(plan_id, resource_id)` | `Sandbox \| None` | Look up without creating |
| `list_sandboxes(plan_id)` | `list[Sandbox]` | All sandboxes for a plan |
| `commit_all(plan_id)` | `list[CommitResult]` | Atomically commit all active sandboxes |
| `rollback_all(plan_id)` | `None` | Roll back all active sandboxes |
| `cleanup_all(plan_id)` | `None` | Clean up all sandboxes and remove tracking |
| `cleanup_abandoned()` | `int` | Clean up terminal-state sandboxes; returns count |
| `get_or_create_sandbox_for_resource(plan_id, resource, resource_registry)` | `Sandbox` | Boundary-aware sandbox creation |
| `resolve_sandbox_key(resource, resource_registry)` | `str` | Compute the sandbox boundary key for a resource |
| `clear_boundary_cache()` | `None` | Clear the boundary algebra cache |

---

### `SandboxContext`

```python
from cleveragents.infrastructure.sandbox.protocol import SandboxContext
```

Immutable frozen dataclass describing an initialised sandbox.

| Attribute | Type | Description |
|-----------|------|-------------|
| `sandbox_id` | `str` | Unique ULID identifier |
| `sandbox_path` | `str` | Root path where sandboxed files reside |
| `original_path` | `str` | Original resource location |
| `resource_id` | `str` | ID of the resource being sandboxed |
| `plan_id` | `str` | ID of the owning plan |
| `created_at` | `datetime` | When the sandbox was initialised |
| `metadata` | `dict[str, Any]` | Implementation-specific data (e.g. git branch name) |

---

### `CommitResult`

```python
from cleveragents.infrastructure.sandbox.protocol import CommitResult
```

Immutable frozen dataclass describing the outcome of a `commit()` call.

| Attribute | Type | Description |
|-----------|------|-------------|
| `sandbox_id` | `str` | Which sandbox was committed |
| `success` | `bool` | Whether the commit succeeded |
| `commit_ref` | `str \| None` | Git commit hash or equivalent, if applicable |
| `changed_files` | `list[str]` | Modified file paths |
| `added_files` | `list[str]` | Created file paths |
| `deleted_files` | `list[str]` | Removed file paths |
| `error` | `str \| None` | Error message when `success=False` |
| `timestamp` | `datetime` | When the commit occurred |
| `metadata` | `dict[str, Any]` | Arbitrary key-value metadata |

---

## Atomic Commit (`commit_all`)

`SandboxManager.commit_all()` implements the all-or-nothing atomicity
requirement from the specification (see **Atomic Commit Semantics** in the
Sandbox & Checkpoint section).

**Algorithm:**

1. Collect all sandboxes in `CREATED` or `ACTIVE` status for the plan.
2. Separate into *rollbackable* (git, copy-on-write, overlay) and
   *non-rollbackable* (none, transaction) groups.
3. Commit rollbackable sandboxes first; non-rollbackable last.
4. If any commit fails, roll back all previously-committed sandboxes in
   **LIFO** order (most recently committed first).
5. Return a list of `CommitResult` objects (all `success=True`) on success,
   or a single `CommitResult(success=False)` on failure.

**Error handling:**

| Exception | Meaning |
|-----------|---------|
| `SandboxError` | A sandbox-level commit failure; rollback attempted; `CommitResult(success=False)` returned |
| `AtomicCommitError` | A non-sandbox exception during commit; rollback attempted; exception raised with `rolled_back_ids` and `failed_rollback_ids` attributes |

```python
from cleveragents.infrastructure.sandbox.protocol import AtomicCommitError

try:
    results = manager.commit_all(plan_id)
    if not all(r.success for r in results):
        # Partial failure (SandboxError path)
        failed = [r for r in results if not r.success]
        print(f"Commit failed: {failed[0].error}")
except AtomicCommitError as exc:
    # Unexpected exception during commit
    print(f"Atomic commit failed: {exc}")
    print(f"Rolled back: {exc.rolled_back_ids}")
    print(f"Failed rollbacks: {exc.failed_rollback_ids}")
```

---

## Boundary Algebra

The sandbox boundary algebra (introduced in M6, issue #548) determines which
sandbox governs a given resource.  Resources that share a common ancestor
boundary share the same sandbox instance.

```python
# Resources in the same git repo share one sandbox
sandbox_key = manager.resolve_sandbox_key(resource, resource_registry)
# → the resource_id of the boundary resource (e.g. the repo root)

# Multiple resources → same sandbox
sandbox_a = manager.get_or_create_sandbox_for_resource(plan_id, file_a, registry)
sandbox_b = manager.get_or_create_sandbox_for_resource(plan_id, file_b, registry)
assert sandbox_a is sandbox_b  # same boundary → same sandbox
```

Call `manager.clear_boundary_cache()` at the start of each plan execution
or when the resource DAG changes.

---

## Exception Hierarchy

```
SandboxError
├── SandboxCreationError   — sandbox cannot be initialised
├── SandboxCommitError     — commit operation failed
├── SandboxRollbackError   — rollback operation failed
├── SandboxStateError      — operation invalid for current status
└── AtomicCommitError      — non-sandbox exception during commit_all
    ├── rolled_back_ids: list[str]
    └── failed_rollback_ids: list[str]
```

---

## Sandbox Implementations

### `GitWorktreeSandbox`

Uses `git worktree add` to create an isolated branch for the plan.  Changes
are committed to the worktree branch and then merged (or cherry-picked) back
to the original branch on `commit()`.

- **Checkpoint:** `git tag` on the worktree branch
- **Rollback from COMMITTED:** `git reset --hard` to the pre-commit tag

### `CopyOnWriteSandbox`

Creates a full directory copy of the resource.  Writes go to the copy; on
`commit()` the copy is merged back to the original using rename-based swap
(O(1) on the same filesystem).

- **Checkpoint:** `shutil.copytree` snapshot
- **Rollback from COMMITTED:** restore from pre-commit backup

### `OverlaySandbox`

Uses Linux OverlayFS to create a layered view of the resource.  The original
directory is the lower layer; writes go to the upper layer.  Falls back to
copy-on-write when OverlayFS is unavailable.

- **Checkpoint:** snapshot of the upper layer
- **Rollback from COMMITTED:** unmount and remount OverlayFS

### `TransactionSandbox`

Wraps database writes in a transaction.  `commit()` issues `COMMIT`;
`rollback()` issues `ROLLBACK`.  **Cannot be rolled back after `COMMIT`.**

### `NoSandbox`

Pass-through — writes go directly to the original resource.  Used for
resources that cannot be sandboxed (e.g. external APIs).  **Cannot be
rolled back.**

!!! danger "Security Warning: NoSandbox bypasses all isolation"
    `NoSandbox` writes are **immediately permanent** and **cannot be rolled
    back**. The Apply-phase diff review is skipped entirely. Only choose
    `NoSandbox` for resources that are inherently non-sandboxable (for
    example, external API calls with guaranteed idempotency). Never use
    `NoSandbox` for filesystem or database resources where rollback or
    sandbox inspection may be required.

---

## Filesystem Utilities

**Module:** `cleveragents.infrastructure.sandbox._fs_utils`

Shared utilities used by `CopyOnWriteSandbox` and `OverlaySandbox`:

| Function | Description |
|----------|-------------|
| `backup_directory(src, dst)` | Copy a directory tree preserving permissions, timestamps, and symlinks |
| `safe_restore(backup, original)` | Rename-based atomic restore (backup → original) |
| `compute_diff(original, modified)` | Compute a unified diff between two directory trees |

---

## Status Transition Validation

```python
from cleveragents.infrastructure.sandbox.protocol import SandboxStatus

# Check if a transition is valid
assert SandboxStatus.can_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)
assert not SandboxStatus.can_transition(SandboxStatus.CLEANED_UP, SandboxStatus.ACTIVE)

# Assert transition (raises SandboxStateError on invalid)
SandboxStatus.assert_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)
```

---

## Testing

The sandbox module is covered by BDD scenarios in `features/sandbox.feature`
and Robot Framework integration tests in `robot/sandbox.robot`.

```python
from cleveragents.infrastructure.sandbox.protocol import Sandbox, SandboxStatus

# Verify protocol conformance
assert isinstance(my_sandbox, Sandbox)

# Verify status
assert my_sandbox.status == SandboxStatus.ACTIVE
```

---

## Related Documentation

- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md)
- [ADR-038 Cross-Mechanism Sandbox Coordination](../adr/ADR-038-cross-mechanism-sandbox-coordination.md)
- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle)
- [API — Resource System](../api/resource.md)