cleveragents/cleveragents-core

Fork 3

feat(sandbox): implement overlay filesystem sandbox strategy #880

New issue

Closed

opened 2026-03-13 22:59:37 +00:00 by freemo · 1 comment

freemo commented

2026-03-13 22:59:37 +00:00

Owner

Metadata

Commit Message: feat(sandbox): implement overlay filesystem sandbox strategy
Branch: feature/m5-overlay-sandbox

Background and Context

The specification defines 5 sandbox strategies: git_worktree, copy_on_write, overlay, transaction_rollback, and none. The factory (infrastructure/sandbox/factory.py) defines 5 strategy constants but only 4 have implementations. The overlay strategy does not exist; instead, a snapshot constant is defined that raises NotImplementedError when requested.

The overlay strategy would use OverlayFS (or a userspace equivalent) to create a lightweight, copy-on-write filesystem layer over the original resource, capturing all writes in an upper layer that can be committed or discarded.

Expected Behavior

An OverlaySandbox class implementing the Sandbox protocol that:

Creates an overlay mount (upper + work + lower = merged) over the resource path
Captures all file modifications in the upper layer
Supports commit() by merging upper layer changes into the lower layer
Supports rollback() by discarding the upper layer
Falls back to CopyOnWriteSandbox on systems without OverlayFS support

Acceptance Criteria

OverlaySandbox implements the Sandbox protocol (create, commit, rollback, cleanup)
Strategy is registered as overlay in the SandboxFactory
Upper/lower/work directory structure is created correctly
Commit merges overlay changes into original
Rollback discards overlay changes
Graceful fallback to copy_on_write when OverlayFS is not available
Resource type compatibility matrix updated

Subtasks

Implement OverlaySandbox class in infrastructure/sandbox/
Detect OverlayFS availability at runtime
Implement overlay mount/unmount lifecycle
Implement commit (merge upper into lower)
Implement rollback (discard upper)
Register in SandboxFactory and update strategy constants
Implement fallback to CopyOnWriteSandbox
Tests (Behave): Add scenarios for overlay create, commit, rollback, fallback
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Commit Message**: `feat(sandbox): implement overlay filesystem sandbox strategy` - **Branch**: `feature/m5-overlay-sandbox` ## Background and Context The specification defines 5 sandbox strategies: `git_worktree`, `copy_on_write`, `overlay`, `transaction_rollback`, and `none`. The factory (`infrastructure/sandbox/factory.py`) defines 5 strategy constants but only 4 have implementations. The `overlay` strategy does not exist; instead, a `snapshot` constant is defined that raises `NotImplementedError` when requested. The `overlay` strategy would use OverlayFS (or a userspace equivalent) to create a lightweight, copy-on-write filesystem layer over the original resource, capturing all writes in an upper layer that can be committed or discarded. ## Expected Behavior An `OverlaySandbox` class implementing the `Sandbox` protocol that: - Creates an overlay mount (upper + work + lower = merged) over the resource path - Captures all file modifications in the upper layer - Supports `commit()` by merging upper layer changes into the lower layer - Supports `rollback()` by discarding the upper layer - Falls back to `CopyOnWriteSandbox` on systems without OverlayFS support ## Acceptance Criteria - [ ] `OverlaySandbox` implements the `Sandbox` protocol (create, commit, rollback, cleanup) - [ ] Strategy is registered as `overlay` in the `SandboxFactory` - [ ] Upper/lower/work directory structure is created correctly - [ ] Commit merges overlay changes into original - [ ] Rollback discards overlay changes - [ ] Graceful fallback to copy_on_write when OverlayFS is not available - [ ] Resource type compatibility matrix updated ## Subtasks - [ ] Implement `OverlaySandbox` class in `infrastructure/sandbox/` - [ ] Detect OverlayFS availability at runtime - [ ] Implement overlay mount/unmount lifecycle - [ ] Implement commit (merge upper into lower) - [ ] Implement rollback (discard upper) - [ ] Register in SandboxFactory and update strategy constants - [ ] Implement fallback to CopyOnWriteSandbox - [ ] Tests (Behave): Add scenarios for overlay create, commit, rollback, fallback - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

freemo added the

labels

2026-03-13 23:00:02 +00:00

freemo added this to the v3.5.0 milestone

2026-03-13 23:00:14 +00:00

freemo added a new dependency

2026-03-13 23:11:12 +00:00

#369 Epic: Large Project Autonomy & Context

brent.edwards was assigned by freemo

2026-03-14 04:27:22 +00:00

brent.edwards added

and removed

labels

2026-03-16 22:40:49 +00:00

brent.edwards referenced this issue from a commit

2026-03-17 00:59:59 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy

brent.edwards referenced this issue from a pull request that will close it,

2026-03-17 01:01:53 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy #994

brent.edwards referenced this issue from a commit

2026-03-17 01:03:56 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy

brent.edwards added

and removed

labels

2026-03-17 01:04:18 +00:00

brent.edwards commented

2026-03-17 01:04:38 +00:00

Member

Implementation Notes

Architecture

OverlaySandbox (infrastructure/sandbox/overlay.py, 497 lines): Dual-mode implementation:

Real OverlayFS (root + kernel support): Creates upper/, work/, merged/ dirs. Mounts via mount -t overlay overlay -o lowerdir=<original>,upperdir=<upper>,workdir=<work> <merged>. Writes go to upper layer only. Commit copies upper→original. Rollback unmounts and recreates.
Userspace fallback (default in CI/containers): shutil.copytree(original→merged). Commit diffs merged vs original via filecmp. Rollback removes merged and re-copies. Semantically equivalent; no kernel requirements.

Detection: _is_overlayfs_available() checks /proc/filesystems for "overlay" AND os.geteuid() == 0.

Domain Changes

OVERLAY = "overlay" added to SandboxStrategy enum in both resource_type.py and resource.py
STRATEGY_OVERLAY registered in SandboxFactory for fs-mount, fs-directory, fs-file resource types
Updated _SUPPORTED_STRATEGIES compatibility matrix

Quality Gates

Session	Result
lint	PASS
typecheck	PASS (0 errors)
unit_tests	PASS (10,917 scenarios, 22 new)
coverage_report	97%

Commit

551e2801 on branch feature/m5-overlay-sandbox

PR

PR #994

## Implementation Notes ### Architecture **OverlaySandbox** (`infrastructure/sandbox/overlay.py`, 497 lines): Dual-mode implementation: 1. **Real OverlayFS** (root + kernel support): Creates `upper/`, `work/`, `merged/` dirs. Mounts via `mount -t overlay overlay -o lowerdir=<original>,upperdir=<upper>,workdir=<work> <merged>`. Writes go to upper layer only. Commit copies upper→original. Rollback unmounts and recreates. 2. **Userspace fallback** (default in CI/containers): `shutil.copytree(original→merged)`. Commit diffs merged vs original via `filecmp`. Rollback removes merged and re-copies. Semantically equivalent; no kernel requirements. Detection: `_is_overlayfs_available()` checks `/proc/filesystems` for "overlay" AND `os.geteuid() == 0`. ### Domain Changes - `OVERLAY = "overlay"` added to `SandboxStrategy` enum in both `resource_type.py` and `resource.py` - `STRATEGY_OVERLAY` registered in `SandboxFactory` for `fs-mount`, `fs-directory`, `fs-file` resource types - Updated `_SUPPORTED_STRATEGIES` compatibility matrix ### Quality Gates | Session | Result | |---|---| | lint | PASS | | typecheck | PASS (0 errors) | | unit_tests | PASS (10,917 scenarios, 22 new) | | coverage_report | 97% | ### Commit `551e2801` on branch `feature/m5-overlay-sandbox` ### PR [PR #994](https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/994)

brent.edwards referenced this issue from a commit

2026-03-17 02:45:00 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy

brent.edwards referenced this issue from a commit

2026-03-18 03:29:43 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy

brent.edwards referenced this issue from a commit

2026-03-18 19:03:44 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy

brent.edwards referenced this issue from a commit

2026-03-18 19:05:41 +00:00

feat(sandbox): implement overlay filesystem sandbox strategy