UAT: filesystem_copy sandbox strategy documented in spec but not implemented in code #2823

Open
opened 2026-04-04 20:41:16 +00:00 by freemo · 1 comment
Owner

Bug Report

What Was Tested

SandboxFactory.create_sandbox() in src/cleveragents/infrastructure/sandbox/factory.py and SandboxStrategy enum in src/cleveragents/domain/models/core/resource.py.

Expected Behavior (from spec)

The specification (docs/specification.md, line 30588) documents filesystem_copy as a valid value for the sandbox.strategy configuration key:

sandbox.strategy | string | git_worktree | CLEVERAGENTS_SANDBOX_STRATEGY | Default sandbox isolation strategy for the Execute phase. Accepted values: git_worktree (create a git worktree for isolation), filesystem_copy (copy the project directory), transaction_rollback (for database resources), none (no isolation — requires require_sandbox: false in the automation profile).

The spec also describes filesystem_copy as a distinct strategy in multiple places:

  • Line 24969: fs-mount supports copy_on_write, filesystem_copy, or overlay
  • Line 24970: fs-directory supports copy_on_write or filesystem_copy
  • Line 24977: "filesystem_copy: Performs an explicit full copy of the resource directory (e.g., via cp). Works on all writable filesystems regardless of CoW support, at the cost of duplicating data upfront."
  • Line 46054: filesystem_copy described as "Process-level filesystem isolation"
  • Line 28689: Listed as a rollbackable strategy alongside git_worktree, overlay, transaction_rollback

Actual Behavior

filesystem_copy is completely absent from the implementation:

  • SandboxStrategyStr Literal type in src/cleveragents/infrastructure/sandbox/factory.py does not include "filesystem_copy"
  • SandboxStrategy enum in src/cleveragents/domain/models/core/resource.py does not include FILESYSTEM_COPY
  • No FilesystemCopySandbox class exists anywhere in the codebase
  • SandboxFactory.create_sandbox() raises ValueError: Unknown sandbox strategy: filesystem_copy if called with this strategy

Steps to Reproduce

  1. Set CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy or configure sandbox.strategy = filesystem_copy
  2. Attempt to run a plan that requires sandbox isolation
  3. Observe ValueError: Unknown sandbox strategy: filesystem_copy

Code Location

  • src/cleveragents/infrastructure/sandbox/factory.pySandboxStrategyStr Literal type and create_sandbox() method
  • src/cleveragents/domain/models/core/resource.pySandboxStrategy enum

Impact

If a user sets CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy (as documented in the spec), the system will raise a ValueError at runtime when attempting to create a sandbox, causing plan execution to fail entirely. This is a spec-compliance gap: a documented, user-facing configuration value produces a hard crash.

Expected Fix

Per the spec (line 24977), filesystem_copy should perform an explicit full copy of the resource directory (equivalent to cp -r), providing isolation on all writable filesystems regardless of copy-on-write support. The implementation should:

  1. Add FILESYSTEM_COPY = "filesystem_copy" to the SandboxStrategy enum
  2. Add "filesystem_copy" to the SandboxStrategyStr Literal type
  3. Implement a FilesystemCopySandbox class that performs an explicit directory copy on create() and restores/discards on rollback()/commit()
  4. Wire FilesystemCopySandbox into SandboxFactory.create_sandbox()

Metadata

  • Branch: fix/sandbox-filesystem-copy-strategy
  • Commit Message: fix(sandbox): implement filesystem_copy sandbox strategy
  • Milestone: v3.7.0
  • Parent Epic: #825

Subtasks

  • Add FILESYSTEM_COPY = "filesystem_copy" to SandboxStrategy enum in src/cleveragents/domain/models/core/resource.py
  • Add "filesystem_copy" to SandboxStrategyStr Literal type in src/cleveragents/infrastructure/sandbox/factory.py
  • Implement FilesystemCopySandbox class with create(), commit(), rollback(), and get_path() methods using explicit directory copy semantics
  • Wire FilesystemCopySandbox into SandboxFactory.create_sandbox() dispatch
  • Add Behave unit tests for FilesystemCopySandbox covering create, commit, rollback, and error paths
  • Update Robot Framework integration tests to cover filesystem_copy strategy end-to-end
  • Verify CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy no longer raises ValueError

Definition of Done

  • SandboxStrategy.FILESYSTEM_COPY exists in the domain model enum
  • SandboxStrategyStr Literal includes "filesystem_copy"
  • FilesystemCopySandbox performs an explicit full directory copy on create() and correctly restores on rollback()
  • SandboxFactory.create_sandbox("filesystem_copy", ...) returns a valid FilesystemCopySandbox instance without raising
  • Setting CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy and running a plan does not raise ValueError
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Bug Report ### What Was Tested `SandboxFactory.create_sandbox()` in `src/cleveragents/infrastructure/sandbox/factory.py` and `SandboxStrategy` enum in `src/cleveragents/domain/models/core/resource.py`. ### Expected Behavior (from spec) The specification (`docs/specification.md`, line 30588) documents `filesystem_copy` as a valid value for the `sandbox.strategy` configuration key: > `sandbox.strategy` | string | `git_worktree` | `CLEVERAGENTS_SANDBOX_STRATEGY` | Default sandbox isolation strategy for the Execute phase. Accepted values: `git_worktree` (create a git worktree for isolation), `filesystem_copy` (copy the project directory), `transaction_rollback` (for database resources), `none` (no isolation — requires `require_sandbox: false` in the automation profile). The spec also describes `filesystem_copy` as a distinct strategy in multiple places: - Line 24969: `fs-mount` supports `copy_on_write`, `filesystem_copy`, or `overlay` - Line 24970: `fs-directory` supports `copy_on_write` or `filesystem_copy` - Line 24977: "filesystem_copy: Performs an explicit full copy of the resource directory (e.g., via cp). Works on all writable filesystems regardless of CoW support, at the cost of duplicating data upfront." - Line 46054: `filesystem_copy` described as "Process-level filesystem isolation" - Line 28689: Listed as a rollbackable strategy alongside `git_worktree`, `overlay`, `transaction_rollback` ### Actual Behavior `filesystem_copy` is **completely absent** from the implementation: - `SandboxStrategyStr` Literal type in `src/cleveragents/infrastructure/sandbox/factory.py` does not include `"filesystem_copy"` - `SandboxStrategy` enum in `src/cleveragents/domain/models/core/resource.py` does not include `FILESYSTEM_COPY` - No `FilesystemCopySandbox` class exists anywhere in the codebase - `SandboxFactory.create_sandbox()` raises `ValueError: Unknown sandbox strategy: filesystem_copy` if called with this strategy ### Steps to Reproduce 1. Set `CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy` or configure `sandbox.strategy = filesystem_copy` 2. Attempt to run a plan that requires sandbox isolation 3. Observe `ValueError: Unknown sandbox strategy: filesystem_copy` ### Code Location - `src/cleveragents/infrastructure/sandbox/factory.py` — `SandboxStrategyStr` Literal type and `create_sandbox()` method - `src/cleveragents/domain/models/core/resource.py` — `SandboxStrategy` enum ### Impact If a user sets `CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy` (as documented in the spec), the system will raise a `ValueError` at runtime when attempting to create a sandbox, causing plan execution to fail entirely. This is a spec-compliance gap: a documented, user-facing configuration value produces a hard crash. ### Expected Fix Per the spec (line 24977), `filesystem_copy` should perform an explicit full copy of the resource directory (equivalent to `cp -r`), providing isolation on all writable filesystems regardless of copy-on-write support. The implementation should: 1. Add `FILESYSTEM_COPY = "filesystem_copy"` to the `SandboxStrategy` enum 2. Add `"filesystem_copy"` to the `SandboxStrategyStr` Literal type 3. Implement a `FilesystemCopySandbox` class that performs an explicit directory copy on `create()` and restores/discards on `rollback()`/`commit()` 4. Wire `FilesystemCopySandbox` into `SandboxFactory.create_sandbox()` --- ## Metadata - **Branch**: `fix/sandbox-filesystem-copy-strategy` - **Commit Message**: `fix(sandbox): implement filesystem_copy sandbox strategy` - **Milestone**: v3.7.0 - **Parent Epic**: #825 ## Subtasks - [ ] Add `FILESYSTEM_COPY = "filesystem_copy"` to `SandboxStrategy` enum in `src/cleveragents/domain/models/core/resource.py` - [ ] Add `"filesystem_copy"` to `SandboxStrategyStr` Literal type in `src/cleveragents/infrastructure/sandbox/factory.py` - [ ] Implement `FilesystemCopySandbox` class with `create()`, `commit()`, `rollback()`, and `get_path()` methods using explicit directory copy semantics - [ ] Wire `FilesystemCopySandbox` into `SandboxFactory.create_sandbox()` dispatch - [ ] Add Behave unit tests for `FilesystemCopySandbox` covering create, commit, rollback, and error paths - [ ] Update Robot Framework integration tests to cover `filesystem_copy` strategy end-to-end - [ ] Verify `CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy` no longer raises `ValueError` ## Definition of Done - [ ] `SandboxStrategy.FILESYSTEM_COPY` exists in the domain model enum - [ ] `SandboxStrategyStr` Literal includes `"filesystem_copy"` - [ ] `FilesystemCopySandbox` performs an explicit full directory copy on `create()` and correctly restores on `rollback()` - [ ] `SandboxFactory.create_sandbox("filesystem_copy", ...)` returns a valid `FilesystemCopySandbox` instance without raising - [ ] Setting `CLEVERAGENTS_SANDBOX_STRATEGY=filesystem_copy` and running a plan does not raise `ValueError` - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester
freemo added this to the v3.7.0 milestone 2026-04-04 20:41:21 +00:00
Author
Owner

Starting implementation on branch fix/sandbox-filesystem-copy-strategy.

Difficulty assessment: Medium → starting at sonnet tier.

Wave plan:

  • Wave 1 (parallel): Add enum value + Add Literal type + Implement FilesystemCopySandbox class
  • Wave 2 (sequential): Wire into SandboxFactory dispatch
  • Wave 3 (parallel): Add Behave unit tests + Update Robot Framework integration tests
  • Wave 4: Verify end-to-end

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-worker

Starting implementation on branch `fix/sandbox-filesystem-copy-strategy`. Difficulty assessment: Medium → starting at sonnet tier. Wave plan: - Wave 1 (parallel): Add enum value + Add Literal type + Implement FilesystemCopySandbox class - Wave 2 (sequential): Wire into SandboxFactory dispatch - Wave 3 (parallel): Add Behave unit tests + Update Robot Framework integration tests - Wave 4: Verify end-to-end --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-worker
freemo removed this from the v3.7.0 milestone 2026-04-07 00:42:03 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
Reference
cleveragents/cleveragents-core#2823
No description provided.