7f078f75a5
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 28s
CI / security (pull_request) Successful in 1m8s
CI / build (pull_request) Successful in 20s
CI / helm (pull_request) Successful in 34s
CI / quality (pull_request) Successful in 3m53s
CI / integration_tests (pull_request) Successful in 4m2s
CI / typecheck (pull_request) Successful in 4m9s
CI / unit_tests (pull_request) Successful in 4m48s
CI / docker (pull_request) Successful in 1m19s
CI / e2e_tests (pull_request) Successful in 9m47s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / quality (push) Successful in 52s
CI / lint (push) Successful in 3m18s
CI / build (push) Successful in 20s
CI / typecheck (push) Successful in 3m56s
CI / helm (push) Successful in 22s
CI / security (push) Successful in 4m15s
CI / unit_tests (push) Successful in 4m39s
CI / docker (push) Successful in 18s
CI / integration_tests (push) Successful in 6m56s
CI / e2e_tests (push) Successful in 12m34s
CI / coverage (push) Successful in 11m39s
CI / status-check (push) Successful in 1s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Successful in 30m10s
CI / benchmark-regression (pull_request) Successful in 57m33s
Changed SandboxManager.commit_all() from partial-commit semantics to all-or-nothing atomic operation per specification requirement. On partial failure, already-committed sandboxes are rolled back. Error reporting indicates which sandbox failed and what was rolled back. Added Behave scenarios verifying atomicity guarantee. Hardened atomicity guarantees after code review: - commit_all and _rollback_committed catch Exception (not just SandboxError) so unexpected errors cannot bypass rollback. Non-SandboxError exceptions are wrapped in a new AtomicCommitError (chaining the original as __cause__) that carries rolled_back_ids and failed_rollback_ids attributes so callers can programmatically determine rollback outcomes. - _rollback_committed returns both rolled_back_ids and failed_rollback_ids; the error result metadata now carries both "rolled_back" and "rollback_failed" keys. - _rollback_committed iterates in reverse (LIFO) order following the standard transaction-log undo pattern. Clarified in docstring that this is distinct from the specification DAG-based "top-down" rollback ordering (line 24632). - TransactionSandbox.rollback() from COMMITTED now raises SandboxRollbackError (database commits are irreversible) instead of silently transitioning to ROLLED_BACK. - TransactionSandbox is now classified as non-rollbackable in commit_all alongside NoSandbox and committed last in the batch, since database COMMIT is irreversible. Docstring corrected to match the raising behavior. - CopyOnWriteSandbox and OverlaySandbox rollback-from-COMMITTED uses rename-based safe_restore() to prevent data loss when the copytree step fails after the original was removed. - CopyOnWriteSandbox and OverlaySandbox rollback() now catches Exception (not just OSError), matching the broader catch used in _rollback_committed, so non-OSError exceptions from safe_restore set the status to ERRORED correctly. - Extracted shared _fs_utils module (backup_directory, safe_restore, compute_diff) with symlink, permission, and timestamp preservation (including directory timestamps), replacing duplicated per-class _backup_directory and _compute_diff methods. - backup_directory defers directory permissions and timestamps to a bottom-up post-walk pass, fixing incorrect mtime preservation (POSIX file creation inside a directory overwrites its mtime) and preventing restrictive source permissions from blocking backup writes. - backup_directory skips non-regular files (FIFOs, sockets, device files) with a warning to prevent hangs on special files. - CopyOnWriteSandbox and OverlaySandbox commit() now attempts to restore the original from the pre-commit backup when the file-copy phase fails midway, preventing partial corruption. If the restore itself fails the backup is preserved for manual recovery (cleanup() still removes it). - CopyOnWriteSandbox and OverlaySandbox commit() error handler now catches Exception (not just OSError) so that unexpected errors during the file-copy phase also trigger pre-commit backup restoration, preventing partial corruption of the original directory. - Pre-commit backup exception handler catches Exception (not just OSError) preventing temp directory leaks on non-OSError failures from backup_directory. - Fixed _pre_commit_backup assignment timing: the backup reference is now assigned only AFTER backup_directory() succeeds, preventing safe_restore() from corrupting an intact original with a partial backup when backup_directory() fails (e.g. disk full). - Rollback from COMMITTED with no pre-commit backup (no changes were applied) is now a no-op instead of raising SandboxRollbackError, preventing false rollback-failure reports in commit_all error metadata. - commit_all logs a warning when NoSandbox or TransactionSandbox instances are present in the batch since their changes cannot be rolled back, which breaks the atomicity guarantee. - Pre-commit backup is skipped when compute_diff returns no changes, avoiding a full directory copy for no-op commits. - GitWorktreeSandbox clears _pre_merge_commit on commit failure so the stale value cannot be used by future code. - commit_all docstring documents Raises clause for AtomicCommitError exception wrapping behavior. - GitWorktreeSandbox.rollback() docstring warns about multi-worktree safety when rolling back from COMMITTED. - Updated SandboxStatus transition diagram in protocol.py to clearly show the COMMITTED -> ROLLED_BACK path. - Added spec-contradiction note (line 45938 vs 19193) in commit_all docstring. - OverlaySandbox rollback from COMMITTED now properly remounts OverlayFS for real overlay (unmount, clean upper/work dirs, remount) and uses dirs_exist_ok=True for userspace fallback to prevent FileExistsError if rmtree silently fails. The merged directory is reset from the restored original, preventing stale pre-rollback data from being exposed on re-activation via get_path() (which allows ROLLED_BACK status). - OverlaySandbox rollback from COMMITTED now raises SandboxRollbackError if the OverlayFS unmount fails, preventing a double-mount attempt that would leave the sandbox in an inconsistent state. - OverlaySandbox rollback from ACTIVE now uses dirs_exist_ok=True for userspace fallback to prevent FileExistsError when rmtree with ignore_errors=True silently fails. - CopyOnWriteSandbox rollback from ACTIVE now uses dirs_exist_ok=True in copytree to prevent FileExistsError when rmtree with ignore_errors=True silently fails, matching the fix already applied to OverlaySandbox. - Non-rollbackable sandboxes (NoSandbox, TransactionSandbox) are committed last in the batch so that all rollbackable sandboxes commit first; if any rollbackable sandbox fails, none of the non-rollbackable sandboxes will have committed yet. - Moved NoSandbox and TransactionSandbox imports to module level in manager.py (no circular dependency exists). - Pre-commit backups are now created on the same filesystem as the original directory (using dir= argument to mkdtemp), avoiding cross-device copy overhead and ensuring os.rename compatibility. - safe_restore now renames the target into the mkdtemp directory instead of removing the mkdtemp dir first, eliminating the residual TOCTOU window between rmdir and rename. - safe_restore catches BaseException (not just OSError) to ensure the original directory is always renamed back on unexpected errors, preventing the original from being left in the renamed-aside state. - Added AtomicCommitError exception class to protocol.py carrying rolled_back_ids and failed_rollback_ids attributes. - Exported AtomicCommitError from sandbox package __init__.py so callers can import it from the public API. - Added BDD scenarios: LIFO rollback order, AtomicCommitError wrapping with RuntimeError cause and rollback metadata, _fs_utils backup/restore coverage, no-change commit rollback success, directory timestamp preservation, OverlaySandbox merged dir reset after COMMITTED rollback, CopyOnWriteSandbox rollback from COMMITTED restores original, GitWorktreeSandbox rollback from COMMITTED undoes merge, TransactionSandbox rollback from COMMITTED raises SandboxRollbackError about irreversible commit. ISSUES CLOSED: #925 Post-review hardening (PR #1146 review findings): - OverlaySandbox rollback from COMMITTED with no backup (no-op) now skips the merged directory reset entirely, preventing unnecessary unmount/remount or re-copy that could fail and turn a harmless no-op rollback into a SandboxRollbackError during commit_all atomic recovery. - OverlaySandbox rollback no longer double-wraps SandboxRollbackError: the outer except Exception handler now has a preceding except SandboxRollbackError clause that re-raises directly, avoiding a confusing double-wrapped error chain. - CopyOnWriteSandbox.get_path() now accepts ROLLED_BACK status for consistency with OverlaySandbox and the protocol status transition table (ROLLED_BACK -> ACTIVE). - CopyOnWriteSandbox rollback from COMMITTED now resets the sandbox copy from the restored original via rmtree+copytree, preventing stale pre-rollback modifications from being exposed on re-activation. - rollback_all now catches Exception (not just SandboxError) so that unexpected rollback errors do not prevent remaining sandboxes from being rolled back, consistent with the pattern already used in _rollback_committed. - commit_all docstring now documents a thread-safety warning: the method is not safe for concurrent calls on the same plan_id since sandbox commit/rollback runs outside the lock. - Fixed CHANGELOG.md whitespace inconsistencies (double leading spaces on two lines). Post-review hardening round 2 (PR #1146 automated review): - safe_restore now uses os.rename (O(1) atomic rename) instead of shutil.copytree (O(n) recursive copy) for the main restore path, since backup and target are always on the same filesystem. This eliminates the ENOTEMPTY bug where a partial copytree failure left target_path partially populated, causing the recovery os.rename to fail and strand the original in the stale temp directory. - OverlaySandbox.get_path() now transitions ROLLED_BACK to ACTIVE, matching CopyOnWriteSandbox and the protocol transition table (ROLLED_BACK -> ACTIVE). - GitWorktreeSandbox.get_path() now accepts ROLLED_BACK status for consistency with all other sandbox implementations and the protocol transition table (ROLLED_BACK -> ACTIVE). - rollback_all now also handles sandboxes in COMMITTED status (not just ACTIVE), consistent with the state machine allowing COMMITTED -> ROLLED_BACK. - cleanup_all now catches Exception (not just SandboxError) so a single unexpected error does not abort cleanup of remaining sandboxes, consistent with _rollback_committed and rollback_all. - Restructured CHANGELOG entry from a single ~90-line paragraph into structured sub-bullets for readability. - Added BDD scenarios: no-op rollback from COMMITTED for CopyOnWriteSandbox and OverlaySandbox (zero-change commit), commit ordering verification (rollbackable before non-rollbackable). Post-review hardening round 3 (PR #1146 deep automated review): - cleanup_abandoned now catches Exception (not just SandboxError) so that unexpected errors (e.g. raw OSError, PermissionError) do not crash the loop and prevent remaining abandoned sandboxes from being cleaned up, consistent with cleanup_all, rollback_all, and _rollback_committed. - OverlaySandbox._mount_overlay() now catches subprocess.TimeoutExpired (in addition to CalledProcessError and OSError), preventing create() from leaving the sandbox in PENDING status when mount hangs beyond the timeout. - OverlaySandbox._unmount_overlay() now catches subprocess.TimeoutExpired (in addition to CalledProcessError and OSError), preventing cleanup() from leaving the sandbox in a zombie state when umount hangs beyond the timeout. - OverlaySandbox._mount_overlay() validates that overlay paths do not contain commas, which would corrupt the OverlayFS mount options string (comma is the mount option delimiter). - GitWorktreeSandbox.commit() now checks git diff return code so that a failed diff command raises CalledProcessError instead of silently concluding there are no changes and skipping the merge. - safe_restore cleanup of the temporary rollback container now runs in a finally block, preventing a temp directory leak when the rename fails and the exception is re-raised. - Fixed misleading BDD step name: "backup path that will cause copytree to fail" renamed to "backup path that will cause rename to fail" since safe_restore now uses os.rename.
228 lines
10 KiB
Gherkin
228 lines
10 KiB
Gherkin
Feature: Overlay filesystem sandbox lifecycle
|
|
As a developer
|
|
I want a sandbox that isolates plan changes using overlay filesystem semantics
|
|
So that changes are captured in an upper layer and merged on commit
|
|
|
|
# --- Creation ---
|
|
|
|
Scenario: Create an overlay sandbox
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
Then the ovl sandbox should be in the "created" state
|
|
And the ovl sandbox context should reference plan "plan-001"
|
|
And the ovl sandbox context should have strategy metadata "overlay"
|
|
And the ovl sandbox merged path should exist
|
|
|
|
Scenario: Creating an overlay sandbox with empty plan_id raises ValueError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created with empty plan_id
|
|
Then an ovl ValueError should be raised with message "plan_id cannot be empty"
|
|
|
|
Scenario: Creating an overlay sandbox with empty resource_id raises ValueError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is prepared with empty resource_id
|
|
Then an ovl ValueError should be raised with message "resource_id cannot be empty"
|
|
|
|
Scenario: Creating an overlay sandbox with empty original_path raises ValueError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is prepared with empty original_path
|
|
Then an ovl ValueError should be raised with message "original_path cannot be empty"
|
|
|
|
Scenario: Creating an overlay sandbox on a non-existent directory raises SandboxCreationError
|
|
Given an ovl test directory is initialised
|
|
Given an ovl non-existent directory
|
|
When an ovl sandbox is created on the non-existent directory for plan "plan-001"
|
|
Then an ovl SandboxCreationError should be raised
|
|
|
|
# --- Directory structure ---
|
|
|
|
Scenario: Overlay sandbox creates correct directory structure
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
Then the ovl sandbox should have upper, work, and merged directories
|
|
|
|
# --- Path resolution ---
|
|
|
|
Scenario: Resolve a path in the overlay merged directory
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl path "data/file.txt" is resolved
|
|
Then the ovl resolved path should be inside the merged directory
|
|
And the ovl sandbox should be in the "active" state
|
|
|
|
Scenario: Path traversal is rejected
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl path "../etc/passwd" is resolved
|
|
Then an ovl ValueError should be raised with message "Path traversal not allowed"
|
|
|
|
Scenario: Resolving a path on a cleaned-up sandbox raises SandboxStateError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox is cleaned up
|
|
And the ovl path "file.txt" is resolved on a cleaned-up sandbox
|
|
Then an ovl SandboxStateError should be raised
|
|
|
|
# --- Commit ---
|
|
|
|
Scenario: Commit with no changes produces empty result
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox is committed
|
|
Then the ovl commit result should indicate success
|
|
And the ovl commit result should have 0 changed files
|
|
And the ovl commit result should have 0 added files
|
|
And the ovl commit result should have 0 deleted files
|
|
|
|
Scenario: Commit with a new file syncs it to original
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And an ovl file "new_file.txt" is created in the sandbox with content "hello world"
|
|
And the ovl sandbox is committed
|
|
Then the ovl commit result should indicate success
|
|
And the ovl commit result should have 1 added files
|
|
And the ovl file "new_file.txt" should exist in the original directory with content "hello world"
|
|
|
|
Scenario: Commit with a modified file syncs the change
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl existing file "existing.txt" is modified in the sandbox with content "updated"
|
|
And the ovl sandbox is committed
|
|
Then the ovl commit result should indicate success
|
|
And the ovl commit result should have 1 changed files
|
|
And the ovl file "existing.txt" in the original should have content "updated"
|
|
|
|
Scenario: Commit with a deleted file removes it from original
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl existing file "existing.txt" is deleted from the sandbox
|
|
And the ovl sandbox is committed
|
|
Then the ovl commit result should indicate success
|
|
And the ovl commit result should have 1 deleted files
|
|
And the ovl file "existing.txt" should not exist in the original directory
|
|
|
|
Scenario: Commit on a cleaned-up sandbox raises SandboxStateError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox is cleaned up
|
|
And the ovl sandbox commit is attempted on cleaned-up sandbox
|
|
Then an ovl SandboxStateError should be raised
|
|
|
|
# --- Rollback ---
|
|
|
|
Scenario: Rollback restores the sandbox to original state
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And an ovl file "temp.txt" is created in the sandbox with content "temporary"
|
|
And the ovl path "temp.txt" is resolved
|
|
And the ovl sandbox is rolled back
|
|
Then the ovl sandbox should be in the "rolled_back" state
|
|
And the ovl file "temp.txt" should not exist in the sandbox
|
|
|
|
Scenario: Rollback on a non-active sandbox raises SandboxStateError
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox rollback is attempted on created sandbox
|
|
Then an ovl SandboxStateError should be raised
|
|
|
|
# --- Cleanup ---
|
|
|
|
Scenario: Cleanup removes the overlay directory
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox is cleaned up
|
|
Then the ovl sandbox should be in the "cleaned_up" state
|
|
And the ovl sandbox base path should not exist
|
|
|
|
Scenario: Cleanup is idempotent
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
And the ovl sandbox is cleaned up
|
|
And the ovl sandbox is cleaned up again
|
|
Then the ovl sandbox should be in the "cleaned_up" state
|
|
|
|
# --- Protocol properties ---
|
|
|
|
Scenario: Overlay sandbox has a unique ULID identifier
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is instantiated
|
|
Then the ovl sandbox_id should be a valid ULID
|
|
And the ovl sandbox status should be "pending"
|
|
And the ovl sandbox context should be None
|
|
|
|
# --- Fallback mode ---
|
|
|
|
Scenario: Fallback mode works when OverlayFS is unavailable
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-fallback"
|
|
Then the ovl sandbox should use userspace fallback
|
|
|
|
# --- Coverage boost: edge cases ---
|
|
|
|
Scenario: Commit with a message stores it in metadata
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-msg"
|
|
And an ovl file "new.txt" is created in the sandbox with content "data"
|
|
And the ovl sandbox is committed with message "snapshot before deploy"
|
|
Then the ovl commit result should indicate success
|
|
And the ovl commit metadata should contain message "snapshot before deploy"
|
|
|
|
Scenario: get_path raises SandboxStateError when merged_dir is None
|
|
Given an ovl sandbox with merged_dir forced to None
|
|
When ovl get_path is called with "file.txt" expecting an error
|
|
Then an ovl SandboxStateError should be raised with message "merged directory not set"
|
|
|
|
Scenario: Commit wraps OSError into SandboxCommitError
|
|
Given an ovl test directory is initialised
|
|
And an ovl sandbox is created and activated for plan "plan-commiterr"
|
|
And ovl shutil.copy2 is patched to raise OSError
|
|
When the ovl sandbox commit is attempted
|
|
Then an ovl SandboxCommitError should be raised
|
|
And the ovl sandbox should be in the "errored" state
|
|
|
|
Scenario: Rollback wraps OSError into SandboxRollbackError
|
|
Given an ovl test directory is initialised
|
|
And an ovl sandbox is created and activated for plan "plan-rberr"
|
|
And ovl shutil.rmtree is patched to raise OSError for rollback
|
|
When the ovl sandbox rollback is attempted
|
|
Then an ovl SandboxRollbackError should be raised
|
|
And the ovl sandbox should be in the "errored" state
|
|
|
|
Scenario: Create wraps OSError into SandboxCreationError
|
|
Given an ovl test directory is initialised
|
|
And ovl shutil.copytree is patched to raise OSError
|
|
When an ovl sandbox create is attempted for plan "plan-createerr"
|
|
Then an ovl SandboxCreationError should be raised
|
|
And the ovl sandbox should be in the "errored" state
|
|
|
|
# --- Atomic rollback from COMMITTED ---
|
|
|
|
Scenario: Rollback from COMMITTED resets the merged directory
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-rb-committed"
|
|
And an ovl file "changed.txt" is created in the sandbox with content "modified"
|
|
And the ovl sandbox is committed
|
|
Then the ovl sandbox should be in the "committed" state
|
|
When the ovl sandbox is rolled back from committed
|
|
Then the ovl sandbox should be in the "rolled_back" state
|
|
And the ovl original file "existing.txt" should contain "original content"
|
|
And the ovl merged directory should not contain stale file "changed.txt"
|
|
|
|
Scenario: Rollback from COMMITTED with no changes is a no-op
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-rb-noop"
|
|
And the ovl sandbox is committed
|
|
Then the ovl sandbox should be in the "committed" state
|
|
When the ovl sandbox is rolled back from committed
|
|
Then the ovl sandbox should be in the "rolled_back" state
|
|
And the ovl original file "existing.txt" should contain "original content"
|
|
|
|
# --- Status transitions ---
|
|
|
|
Scenario: Status transitions follow protocol
|
|
Given an ovl test directory is initialised
|
|
When an ovl sandbox is created for plan "plan-001"
|
|
Then the ovl sandbox should be in the "created" state
|
|
When the ovl path "existing.txt" is resolved
|
|
Then the ovl sandbox should be in the "active" state
|