cleveragents-core/docs/modules/sandbox.md

Sandbox & Checkpoint Module

Package: cleveragents.infrastructure.sandbox Introduced: M4 (v3.3.0); atomic commit added in M6 (v3.5.0, issue #925)

The sandbox module provides isolated write environments for plan execution. Every resource write during a plan's Execute phase is routed through a sandbox so that changes can be reviewed, rolled back, or atomically committed during the Apply phase.

For the design rationale see ADR-015 (Sandbox & Checkpoint) and ADR-038 (Cross-Mechanism Sandbox Coordination).

---

Purpose

When a plan executes, actors write to resources (files, databases, git repositories). Without sandboxing, every write would immediately modify the original resource — making rollback impossible and preventing the diff-review step in the Apply phase.

The sandbox module solves this by:

1. Intercepting writes — all resource paths are translated to sandbox copies before the actor touches them.
2. Enabling rollback — if execution fails or the user rejects the diff, the sandbox is discarded and the original resource is untouched.
3. Atomic commit — when the user approves the Apply phase, all sandboxes for the plan are committed in an all-or-nothing batch.

---

Sandbox Strategies

Strategy	Class	Use case
git_worktree	GitWorktreeSandbox	Git repositories — changes tracked as a worktree branch
copy_on_write	CopyOnWriteSandbox	Filesystem directories — full directory copy
overlay	OverlaySandbox	Linux OverlayFS — efficient layered filesystem (falls back to copy-on-write)
transaction_rollback	TransactionSandbox	Databases — wraps writes in a transaction
none	NoSandbox	Resources that cannot be sandboxed (writes are immediate)

The strategy is selected per-resource via the sandbox_strategy field on the resource definition. The SandboxFactory instantiates the correct class.

---

Lifecycle

PENDING ──► CREATED ──► ACTIVE ──► COMMITTED ──► CLEANED_UP
                │           │
                │           └──► ROLLED_BACK ──► ACTIVE (re-use)
                │                              └──► CLEANED_UP
                └──► ERRORED ──► CLEANED_UP

Status	Meaning
PENDING	Sandbox object created but not yet initialised
CREATED	Sandbox environment exists; not yet activated for writes
ACTIVE	Sandbox is ready; actor writes are routed here
COMMITTED	Changes applied to the original resource
ROLLED_BACK	Changes discarded; sandbox may be re-activated
CLEANED_UP	All artefacts removed; terminal state
ERRORED	An unrecoverable error occurred

---

Core Classes

Sandbox (Protocol)

from cleveragents.infrastructure.sandbox.protocol import Sandbox

Runtime-checkable protocol that all sandbox implementations satisfy.

# Typical usage in plan execution
sandbox = manager.get_or_create_sandbox(
    plan_id="01ARZ3...",
    resource_id="res-001",
    original_path="/path/to/repo",
    sandbox_strategy="git_worktree",
)

# Translate a resource-relative path to the sandbox copy
sandboxed_path = sandbox.get_path("src/main.py")
# ... actor writes to sandboxed_path ...

# Commit changes back to the original resource
result = sandbox.commit("Apply edits from plan 01ARZ3...")
# result.success, result.changed_files, result.added_files, result.deleted_files

# Or discard changes
sandbox.rollback()

# Always clean up when done
sandbox.cleanup()

Key methods:

Method	Description
create(plan_id) → SandboxContext	Initialise the sandbox environment
get_path(resource_path) → str	Translate a resource-relative path to an absolute sandbox path
commit(message=None) → CommitResult	Apply changes to the original resource
rollback() → None	Discard all changes
cleanup() → None	Remove all sandbox artefacts (idempotent)

Key properties:

Property	Type	Description
sandbox_id	str	Unique ULID identifier
status	SandboxStatus	Current lifecycle status
context	SandboxContext | None	Context after create(), None before

---

SandboxManager

from cleveragents.infrastructure.sandbox.manager import SandboxManager

Thread-safe manager for sandbox lifecycles across plan executions. Provides lazy creation, batch operations, and automatic cleanup on process exit.

Obtain via the DI container:

manager = container.sandbox_manager()

Key methods:

Method	Returns	Description
get_or_create_sandbox(plan_id, resource_id, original_path, sandbox_strategy)	Sandbox	Return existing or create new sandbox (lazy)
get_sandbox(plan_id, resource_id)	Sandbox | None	Look up without creating
list_sandboxes(plan_id)	list[Sandbox]	All sandboxes for a plan
commit_all(plan_id)	list[CommitResult]	Atomically commit all active sandboxes
rollback_all(plan_id)	None	Roll back all active sandboxes
cleanup_all(plan_id)	None	Clean up all sandboxes and remove tracking
cleanup_abandoned()	int	Clean up terminal-state sandboxes; returns count
get_or_create_sandbox_for_resource(plan_id, resource, resource_registry)	Sandbox	Boundary-aware sandbox creation
resolve_sandbox_key(resource, resource_registry)	str	Compute the sandbox boundary key for a resource
clear_boundary_cache()	None	Clear the boundary algebra cache

---

SandboxContext

from cleveragents.infrastructure.sandbox.protocol import SandboxContext

Immutable frozen dataclass describing an initialised sandbox.

Attribute	Type	Description
sandbox_id	str	Unique ULID identifier
sandbox_path	str	Root path where sandboxed files reside
original_path	str	Original resource location
resource_id	str	ID of the resource being sandboxed
plan_id	str	ID of the owning plan
created_at	datetime	When the sandbox was initialised
metadata	dict[str, Any]	Implementation-specific data (e.g. git branch name)

---

CommitResult

from cleveragents.infrastructure.sandbox.protocol import CommitResult

Immutable frozen dataclass describing the outcome of a commit() call.

Attribute	Type	Description
sandbox_id	str	Which sandbox was committed
success	bool	Whether the commit succeeded
commit_ref	str | None	Git commit hash or equivalent, if applicable
changed_files	list[str]	Modified file paths
added_files	list[str]	Created file paths
deleted_files	list[str]	Removed file paths
error	str | None	Error message when success=False
timestamp	datetime	When the commit occurred
metadata	dict[str, Any]	Arbitrary key-value metadata

---

Atomic Commit (commit_all)

SandboxManager.commit_all() implements the all-or-nothing atomicity requirement from the specification (see Atomic Commit Semantics in the Sandbox & Checkpoint section).

Algorithm:

1. Collect all sandboxes in CREATED or ACTIVE status for the plan.
2. Separate into rollbackable (git, copy-on-write, overlay) and non-rollbackable (none, transaction) groups.
3. Commit rollbackable sandboxes first; non-rollbackable last.
4. If any commit fails, roll back all previously-committed sandboxes in LIFO order (most recently committed first).
5. Return a list of CommitResult objects (all success=True) on success, or a single CommitResult(success=False) on failure.

Error handling:

Exception	Meaning
SandboxError	A sandbox-level commit failure; rollback attempted; CommitResult(success=False) returned
AtomicCommitError	A non-sandbox exception during commit; rollback attempted; exception raised with rolled_back_ids and failed_rollback_ids attributes

from cleveragents.infrastructure.sandbox.protocol import AtomicCommitError

try:
    results = manager.commit_all(plan_id)
    if not all(r.success for r in results):
        # Partial failure (SandboxError path)
        failed = [r for r in results if not r.success]
        print(f"Commit failed: {failed[0].error}")
except AtomicCommitError as exc:
    # Unexpected exception during commit
    print(f"Atomic commit failed: {exc}")
    print(f"Rolled back: {exc.rolled_back_ids}")
    print(f"Failed rollbacks: {exc.failed_rollback_ids}")

---

Boundary Algebra

The sandbox boundary algebra (introduced in M6, issue #548) determines which sandbox governs a given resource. Resources that share a common ancestor boundary share the same sandbox instance.

# Resources in the same git repo share one sandbox
sandbox_key = manager.resolve_sandbox_key(resource, resource_registry)
# → the resource_id of the boundary resource (e.g. the repo root)

# Multiple resources → same sandbox
sandbox_a = manager.get_or_create_sandbox_for_resource(plan_id, file_a, registry)
sandbox_b = manager.get_or_create_sandbox_for_resource(plan_id, file_b, registry)
assert sandbox_a is sandbox_b  # same boundary → same sandbox

Call manager.clear_boundary_cache() at the start of each plan execution or when the resource DAG changes.

---

Exception Hierarchy

SandboxError
├── SandboxCreationError   — sandbox cannot be initialised
├── SandboxCommitError     — commit operation failed
├── SandboxRollbackError   — rollback operation failed
├── SandboxStateError      — operation invalid for current status
└── AtomicCommitError      — non-sandbox exception during commit_all
    ├── rolled_back_ids: list[str]
    └── failed_rollback_ids: list[str]

---

Sandbox Implementations

GitWorktreeSandbox

Uses git worktree add to create an isolated branch for the plan. Changes are committed to the worktree branch and then merged (or cherry-picked) back to the original branch on commit().

• Checkpoint: git tag on the worktree branch
• Rollback from COMMITTED: git reset --hard to the pre-commit tag

CopyOnWriteSandbox

Creates a full directory copy of the resource. Writes go to the copy; on commit() the copy is merged back to the original using rename-based swap (O(1) on the same filesystem).

• Checkpoint: shutil.copytree snapshot
• Rollback from COMMITTED: restore from pre-commit backup

OverlaySandbox

Uses Linux OverlayFS to create a layered view of the resource. The original directory is the lower layer; writes go to the upper layer. Falls back to copy-on-write when OverlayFS is unavailable.

• Checkpoint: snapshot of the upper layer
• Rollback from COMMITTED: unmount and remount OverlayFS

TransactionSandbox

Wraps database writes in a transaction. commit() issues COMMIT; rollback() issues ROLLBACK. Cannot be rolled back after COMMIT.

NoSandbox

Pass-through — writes go directly to the original resource. Used for resources that cannot be sandboxed (e.g. external APIs). Cannot be rolled back.

!!! danger "Security Warning: NoSandbox bypasses all isolation" NoSandbox writes are immediately permanent and cannot be rolled back. The Apply-phase diff review is skipped entirely. Only choose NoSandbox for resources that are inherently non-sandboxable (for example, external API calls with guaranteed idempotency). Never use NoSandbox for filesystem or database resources where rollback or sandbox inspection may be required.

---

Filesystem Utilities

Module: cleveragents.infrastructure.sandbox._fs_utils

Shared utilities used by CopyOnWriteSandbox and OverlaySandbox:

Function	Description
backup_directory(src, dst)	Copy a directory tree preserving permissions, timestamps, and symlinks
safe_restore(backup, original)	Rename-based atomic restore (backup → original)
compute_diff(original, modified)	Compute a unified diff between two directory trees

---

Status Transition Validation

from cleveragents.infrastructure.sandbox.protocol import SandboxStatus

# Check if a transition is valid
assert SandboxStatus.can_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)
assert not SandboxStatus.can_transition(SandboxStatus.CLEANED_UP, SandboxStatus.ACTIVE)

# Assert transition (raises SandboxStateError on invalid)
SandboxStatus.assert_transition(SandboxStatus.ACTIVE, SandboxStatus.COMMITTED)

---

Testing

The sandbox module is covered by BDD scenarios in features/sandbox.feature and Robot Framework integration tests in robot/sandbox.robot.

from cleveragents.infrastructure.sandbox.protocol import Sandbox, SandboxStatus

# Verify protocol conformance
assert isinstance(my_sandbox, Sandbox)

# Verify status
assert my_sandbox.status == SandboxStatus.ACTIVE

---

Related Documentation

• ADR-015 Sandbox & Checkpoint
• ADR-038 Cross-Mechanism Sandbox Coordination
• Architecture — Plan Lifecycle
• API — Resource System