cleveragents-core/docs/adr/ADR-038-cross-mechanism-sandbox-coordination.md

adr_number, title, status_history, tier, authors, superseded_by, related_adrs, acceptance

adr_number	title	status_history	tier	authors	superseded_by	related_adrs	acceptance
38	Cross-Mechanism Sandbox Coordination	2026-02-21 Proposed Jeffrey Phillips Freeman 2026-02-21 Accepted Jeffrey Phillips Freeman	3	Jeffrey Phillips Freeman	null	number title relationship 8 Resource System Virtual resources and equivalence linking are defined here; this ADR adds coherence and coordination on top number title relationship 15 Sandbox and Checkpoint Sandbox strategies and lazy sandboxing; this ADR extends to cross-mechanism coordination at commit time number title relationship 36 Resource DAG Operational Semantics Defines sandbox boundary algebra; this ADR addresses what happens when boundaries overlap through equivalence number title relationship 37 Tool Reachability and Access Projection Routing decisions must account for coherence and canonical write targets defined here number title relationship 39 Container Resource Types Container mounts create the primary cross-mechanism equivalence scenarios number title relationship 40 LSP Resource Types LSP documents have `cached` coherence with backing filesystem files	votes_for votes_against abstentions voter comment Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> Coherence + write-then-sync + lazy materialization is the right balance between correctness and efficiency for cross-mechanism coordination

Context

ADR-015 defines per-plan sandboxes with five isolation strategies and lazy sandboxing. ADR-036 formalizes sandbox boundaries as a DAG property. However, when the same virtual resource has physical manifestations in different sandbox domains — for example, a file accessible through both a git-checkout sandbox (git_worktree) and a Docker container sandbox (snapshot) — the current design does not address:

1. How do writes in one sandbox domain affect the other? A file modified through git_worktree is not automatically visible in the Docker container's snapshot, even though both represent the same logical file.
2. How does sandbox commit coordinate across domains? When a plan commits, changes in the git sandbox must be reflected in other domains that share virtual resources.
3. What happens when the same virtual resource is written through different domains? This is a conflict that must be detected and resolved.
4. When should virtual resource nodes be created and destroyed? Eagerly creating virtual hubs for every resource wastes storage; never creating them loses equivalence information.

Decision Drivers

• The same virtual resource may have physical manifestations in different sandbox domains (e.g., git-checkout sandbox and Docker container snapshot), and writes in one domain are not automatically visible in the other
• Sandbox commit must coordinate across domains so that changes in a git sandbox are reflected in container domains sharing the same virtual resource
• Cross-domain write conflicts (same virtual resource modified through different sandbox domains) must be detected before changes are applied to real resources
• Virtual resource nodes for every physical resource would waste storage; equivalence information is only needed when a second manifestation sharing the same identity is discovered
• The relationship between physical manifestations varies — bind mounts are transparent, LSP buffers are cached, volume copies are independent — and the coordination protocol must account for these differences

Decision

CleverAgents defines a coherence property on physical-to-virtual edges that describes the relationship between a physical resource and its virtual identity, a write-then-sync coordination protocol for sandbox commits, lazy virtual node materialization for efficient equivalence tracking, and conflict detection for cross-domain writes.

Design

Coherence Property

Every edge from a physical resource to its virtual parent carries a coherence property describing how changes to the physical resource relate to changes visible through other physical manifestations of the same virtual resource.

Three Coherence Modes

Coherence	Description	Sync Action at Commit	Example
transparent	Changes to one physical manifestation are immediately visible in the other. They share the same underlying storage.	No sync needed — changes are already visible.	Bind mount: host fs-file ↔ container fs-file through bind-mounted directory. Both paths refer to the same inode.
cached	The physical resource caches content from the shared storage. Changes to the backing store require an explicit refresh signal.	Send invalidation/refresh signal (e.g., LSP workspace/didChangeWatchedFiles).	LSP document: the LSP server buffers file content. After a filesystem write, the server must be notified to re-read and re-analyze.
independent	The physical resources are fully independent copies. Changes in one are invisible in the other until explicit synchronization.	Full content sync: copy changed content from canonical target to other manifestations, or mark them as diverged.	Volume-mounted file where the volume is a separate copy. Separate git clones of the same repo.

Coherence Determination

The coherence of a physical-to-virtual edge is determined by the relationship between the physical resource's storage and the virtual identity's other manifestations:

Physical Resource Type	Scenario	Coherence
fs-file via git-checkout	Same checkout, different access path	transparent
fs-file via container bind mount	Host file = container file (same inode)	transparent
fs-file via container volume mount	Volume is a separate copy	independent
lsp-document	LSP caches file content	cached
git-tree-entry	Git object store (immutable)	independent (immutable — content never changes)
git-commit in different repos	Same commit hash, different repos	independent
fs-file in different directories	Same content, different locations	independent

The coherence property is set when the physical-to-virtual edge is created (during auto-discovery or equivalence linking). It can be explicitly overridden via resource configuration for cases where the system cannot automatically determine the relationship (e.g., custom mount drivers with pass-through semantics).

Coherence on the Edge Model

The coherence property is stored on the DAG edge (the resource_edges or resource_links table):

Column	Type	Description
coherence	String(20)	transparent | cached | independent | null (only set on physical-to-virtual edges)

Write-Then-Sync Coordination Protocol

The coordination protocol governs how writes through one physical manifestation are propagated to equivalent physical manifestations at sandbox commit time.

During Plan Execution

1. Tools write through their bound physical resource. The write goes into the physical resource's sandbox domain. No cross-domain propagation occurs during execution.
2. Dirty tracking: The system records which physical resources have been modified during the plan. This is already done by the change recording system (tool invocations record changes to the ChangeSet). The extension is: for each changed physical resource, the system also records which virtual resource it is a child of (if any).
3. No cross-boundary writes during execution: Writes stay within their sandbox domain. Other physical manifestations of the same virtual resource see the original (pre-sandbox) content until commit time.

At Sandbox Commit Time

When a plan's sandboxes are committed (Apply phase), the system performs cross-mechanism coordination:

Step 1: Identify dirty virtual resources.

dirty_virtuals = {virtual_parent(p) for p in dirty_physical_resources
                  if virtual_parent(p) is not None}

Step 2: For each dirty virtual resource, check for conflicts.

for v in dirty_virtuals:
    dirty_manifestations = {p for p in physical_children(v) if is_dirty(p)}
    if len(dirty_manifestations) > 1:
        # CONFLICT: multiple manifestations modified
        resolve_conflict(v, dirty_manifestations)
    else:
        # Single manifestation modified — propagate
        propagate_changes(v, dirty_manifestations.pop())

Step 3: Propagate changes based on coherence.

For each non-dirty physical sibling s of the dirty manifestation d:

Coherence of s	Propagation Action
transparent	No action — s already sees the changes (shared storage). Verify by content hash.
cached	Send refresh signal to s's handler (e.g., LSP workspace/didChangeWatchedFiles).
independent	Copy content from d to s via s's handler. Or: unlink s from the virtual parent (it has diverged).

Step 4: Re-evaluate virtual resource identity.

After propagation, re-compute content hashes for all physical manifestations. If any manifestation no longer matches the virtual identity, unlink it (divergence detection per specification lines 22894–22901).

Conflict Detection and Resolution

A conflict occurs when two or more physical manifestations of the same virtual resource are modified during the same plan execution (through different tools or different sandbox domains).

Detection

conflict_detected(v) = |{p ∈ physical_children(v) : is_dirty(p)}| > 1

Resolution Strategies

Strategy	Description	When to Use
canonical-wins (default)	The canonical write target's changes are accepted. Non-canonical changes are discarded with a warning.	When one path is authoritative (e.g., git-tracked file wins over Docker volume copy).
merge	Attempt content-level merge (three-way merge for text files).	When both changes are intentional and the content type supports merging.
fail	Reject the commit. Surface the conflict for human resolution.	When automatic resolution is too risky (e.g., database records, binary files).
last-writer-wins	Accept the most recent modification (by timestamp).	Low-stakes resources where any version is acceptable.

The default strategy is canonical-wins. Per-virtual-resource or per-resource-type overrides can be configured:

# Resource type configuration
conflict_resolution: canonical-wins  # or: merge, fail, last-writer-wins

Canonical Write Target

Every virtual resource with multiple physical manifestations designates one as the canonical write target — the manifestation through which writes should be routed to ensure sandbox and checkpoint integrity.

Selection rules (priority order):

1. If exactly one physical manifestation is in a sandboxed domain, it is canonical.
2. If multiple are sandboxed, prefer strongest strategy: git_worktree > snapshot > copy_on_write > transaction_rollback.
3. If still ambiguous, prefer the manifestation linked to the plan's primary project.
4. Explicit user override via virtual resource configuration.

The canonical write target is stored as metadata on the virtual resource:

Field	Description
canonical_write_resource_id	Resource ID of the canonical write target physical resource
canonical_write_auto	true if auto-selected, false if user-overridden

Lazy Virtual Node Materialization

Virtual resource nodes are not created eagerly for every physical resource. Instead, they are materialized lazily when a second physical manifestation sharing the same identity is discovered.

Lifecycle

Phase 1: Single manifestation (no virtual node)

git-checkout "/repo"
  └── fs-file "/repo/src/main.py"    ← physical resource, no virtual parent

No virtual node exists. The fs-file has a content hash but no equivalence link.

Phase 2: Second manifestation discovered → virtual node created

When a Docker container is registered and its bind mount overlaps with the git checkout, auto-discovery detects that container-mount-file "/app/src/main.py" has the same content hash and relative path as fs-file "/repo/src/main.py".

The system:

1. Computes the canonical identity using the virtual type's equivalence rule (for file: SHA-256 + filename + permissions).
2. Searches for existing virtual resources matching this identity.
3. If none found: creates a new virtual file resource, links both physical resources as children, determines coherence for each edge.
4. If found: links the new physical resource as a child of the existing virtual resource.

file (virtual: "main.py@sha256:abc...")  ← created
  ├── fs-file "/repo/src/main.py"        ← re-linked as child
  └── fs-file "/app/src/main.py"         ← newly linked (via container-mount)
       (coherence: transparent — bind mount)

Phase 3: Manifestation removed → potential cleanup

When a physical resource is deregistered or its equivalence link is broken (divergence):

• If the virtual node still has 2+ physical children: remove the edge, virtual node persists.
• If only 1 physical child remains: remove the virtual node and the last edge (collapse back to single manifestation).
• If 0 physical children remain: remove the virtual node entirely.

Identity Computation

Each virtual resource type defines an identity function that maps a physical resource to a canonical identity key:

Virtual Type	Identity Function	Identity Key
file	SHA-256(content) + filename + permissions	file:sha256:abc...:main.py:0644
directory	Merkle hash of child identities	dir:merkle:def...
commit	Commit hash	commit:a1b2c3d
branch	Branch name + HEAD commit hash	branch:main:a1b2c3d
tag	Tag name + target object	tag:v1.0.0:a1b2c3d
remote	Remote URL (normalized)	remote:github.com/org/repo
submodule	Submodule URL + path	submodule:github.com/lib:vendor/lib
tree	Tree hash	tree:e8f1...9d2a
symlink	Symlink name + target	symlink:link.txt:/etc/target

The identity key is stored on the virtual resource as content_hash (reusing the existing field).

Materialization Triggers

Virtual node materialization is triggered by:

1. Resource registration: When a new physical resource is registered, compute its identity key and check for existing virtual resources with the same key.
2. Auto-discovery: When auto-discovery creates physical child resources, check each against existing virtual identities.
3. Content change detection: When a physical resource's content changes (detected by content hash mismatch), re-compute its identity and check for new or broken equivalence links.
4. On-demand refresh: When the system needs equivalence information (e.g., read/write routing queries), trigger identity computation for resources that haven't been checked recently.

Cross-Mechanism Coordination and the Sandbox Manager

The SandboxManager is extended to support cross-mechanism coordination:

1. At sandbox creation: When a sandbox is created for a resource, check if any of the resource's descendants have virtual parents with physical siblings in other sandbox domains. Record these cross-domain relationships.

2. During execution: No cross-domain coordination. Tools operate within their sandbox domain.

3. At commit time: Before committing a sandbox, run the write-then-sync protocol: a. Identify dirty virtual resources (from changeset). b. Check for conflicts across sandbox domains. c. Propagate changes based on coherence. d. Re-evaluate virtual identities (divergence detection).

4. At rollback time: Rolling back a sandbox only affects resources within its domain. Cross-domain physical manifestations retain their pre-execution state (which is correct, since no cross-domain propagation happens during execution).

Constraints

• Coherence determination must be automatic for built-in resource type combinations. Custom resource types may require explicit coherence configuration.
• Write-then-sync propagation must complete before the sandbox commit is reported as successful. Propagation failures are reported as commit warnings (not failures) for cached and transparent coherence, but as commit failures for independent coherence.
• Conflict detection runs at commit time, not during execution. This means conflicts are not detected early — they surface only at Apply. This is acceptable because cross-domain writes to the same virtual resource are rare and should be flagged during the Strategize phase (when the system can analyze the plan's write scope).
• Lazy materialization means equivalence information is not always available. Systems that need equivalence data should trigger materialization explicitly (via the identity computation / on-demand refresh).
• Virtual node cleanup (collapsing to single manifestation) must be idempotent and safe to run concurrently with other operations.

Consequences

Positive

• Cross-mechanism writes are handled explicitly, preventing silent divergence between equivalent resources in different sandbox domains.
• The coherence property makes the relationship between physical manifestations explicit and queryable, instead of implicit in the mount configuration.
• Lazy virtual materialization avoids the storage overhead of creating virtual nodes for every resource while still providing equivalence tracking when it matters.
• Conflict detection at commit time catches unintended cross-domain writes before they are applied to real resources.

Negative

• The write-then-sync protocol adds complexity to the sandbox commit path.
• Conflict detection at commit time (not during execution) means wasted work if a conflict is detected after extensive execution.
• Coherence determination for custom resource types may require manual configuration, adding setup overhead.

Risks

• transparent coherence assumes bind mounts provide true transparent access. Some container runtimes or mount configurations may introduce caching or delays that break this assumption.
• Content synchronization for independent coherence may fail if the target resource's handler does not support write operations.
• Lazy materialization could miss equivalence links if the identity computation is not triggered at the right time (e.g., a resource is registered but its content is not yet available for hashing).

Alternatives Considered

Exclusive write coordination — Only one physical manifestation accepts writes at a time; others become read-only. Simpler but prevents tools bound to different sandbox domains from writing to the same logical resource within a single plan. This is too restrictive for complex plans that span host and container environments.

Eager virtual node creation — Create a virtual node for every physical resource at registration time. Ensures equivalence is always available but creates significant storage overhead (every file gets both a physical and virtual node, doubling the node count).

No cross-mechanism coordination — Treat different sandbox domains as fully independent. Simpler but allows silent divergence between equivalent resources, which can lead to inconsistent state after commit.

Compliance

• Coherence assignment tests: Verify correct coherence assignment for all built-in resource type combinations (bind mount = transparent, LSP = cached, volume = independent, etc.).
• Write-then-sync tests: End-to-end tests verify that changes in one sandbox domain are correctly propagated to equivalent resources in other domains at commit time.
• Conflict detection tests: Verify that cross-domain writes to the same virtual resource are detected and resolved according to the configured strategy.
• Lazy materialization tests: Verify that virtual nodes are created when a second manifestation is discovered, and cleaned up when collapsed back to single manifestation.
• Coherence refresh tests: Verify that cached coherence triggers the correct refresh signal (e.g., LSP notification) and that transparent coherence requires no action.