Files
temp/features/steps/git_worktree_sandbox_steps.py
Luis Mendes 7f078f75a5 fix(sandbox): make commit_all atomic per specification
Changed SandboxManager.commit_all() from partial-commit semantics to
all-or-nothing atomic operation per specification requirement. On
partial failure, already-committed sandboxes are rolled back. Error
reporting indicates which sandbox failed and what was rolled back.
Added Behave scenarios verifying atomicity guarantee.

Hardened atomicity guarantees after code review:

- commit_all and _rollback_committed catch Exception (not just
  SandboxError) so unexpected errors cannot bypass rollback.
  Non-SandboxError exceptions are wrapped in a new AtomicCommitError
  (chaining the original as __cause__) that carries rolled_back_ids
  and failed_rollback_ids attributes so callers can programmatically
  determine rollback outcomes.

- _rollback_committed returns both rolled_back_ids and
  failed_rollback_ids; the error result metadata now carries
  both "rolled_back" and "rollback_failed" keys.

- _rollback_committed iterates in reverse (LIFO) order following
  the standard transaction-log undo pattern.  Clarified in
  docstring that this is distinct from the specification DAG-based
  "top-down" rollback ordering (line 24632).

- TransactionSandbox.rollback() from COMMITTED now raises
  SandboxRollbackError (database commits are irreversible)
  instead of silently transitioning to ROLLED_BACK.

- TransactionSandbox is now classified as non-rollbackable in
  commit_all alongside NoSandbox and committed last in the batch,
  since database COMMIT is irreversible.  Docstring corrected to
  match the raising behavior.

- CopyOnWriteSandbox and OverlaySandbox rollback-from-COMMITTED
  uses rename-based safe_restore() to prevent data loss when
  the copytree step fails after the original was removed.

- CopyOnWriteSandbox and OverlaySandbox rollback() now catches
  Exception (not just OSError), matching the broader catch used
  in _rollback_committed, so non-OSError exceptions from
  safe_restore set the status to ERRORED correctly.

- Extracted shared _fs_utils module (backup_directory,
  safe_restore, compute_diff) with symlink, permission, and
  timestamp preservation (including directory timestamps),
  replacing duplicated per-class _backup_directory and
  _compute_diff methods.

- backup_directory defers directory permissions and timestamps
  to a bottom-up post-walk pass, fixing incorrect mtime
  preservation (POSIX file creation inside a directory overwrites
  its mtime) and preventing restrictive source permissions from
  blocking backup writes.

- backup_directory skips non-regular files (FIFOs, sockets,
  device files) with a warning to prevent hangs on special files.

- CopyOnWriteSandbox and OverlaySandbox commit() now attempts
  to restore the original from the pre-commit backup when the
  file-copy phase fails midway, preventing partial corruption.
  If the restore itself fails the backup is preserved for manual
  recovery (cleanup() still removes it).

- CopyOnWriteSandbox and OverlaySandbox commit() error handler
  now catches Exception (not just OSError) so that unexpected
  errors during the file-copy phase also trigger pre-commit
  backup restoration, preventing partial corruption of the
  original directory.

- Pre-commit backup exception handler catches Exception (not
  just OSError) preventing temp directory leaks on non-OSError
  failures from backup_directory.

- Fixed _pre_commit_backup assignment timing: the backup
  reference is now assigned only AFTER backup_directory()
  succeeds, preventing safe_restore() from corrupting an intact
  original with a partial backup when backup_directory() fails
  (e.g. disk full).

- Rollback from COMMITTED with no pre-commit backup (no changes
  were applied) is now a no-op instead of raising
  SandboxRollbackError, preventing false rollback-failure
  reports in commit_all error metadata.

- commit_all logs a warning when NoSandbox or TransactionSandbox
  instances are present in the batch since their changes cannot
  be rolled back, which breaks the atomicity guarantee.

- Pre-commit backup is skipped when compute_diff returns no
  changes, avoiding a full directory copy for no-op commits.

- GitWorktreeSandbox clears _pre_merge_commit on commit failure
  so the stale value cannot be used by future code.

- commit_all docstring documents Raises clause for AtomicCommitError
  exception wrapping behavior.

- GitWorktreeSandbox.rollback() docstring warns about
  multi-worktree safety when rolling back from COMMITTED.

- Updated SandboxStatus transition diagram in protocol.py to
  clearly show the COMMITTED -> ROLLED_BACK path.

- Added spec-contradiction note (line 45938 vs 19193) in
  commit_all docstring.

- OverlaySandbox rollback from COMMITTED now properly remounts
  OverlayFS for real overlay (unmount, clean upper/work dirs,
  remount) and uses dirs_exist_ok=True for userspace fallback
  to prevent FileExistsError if rmtree silently fails.  The
  merged directory is reset from the restored original,
  preventing stale pre-rollback data from being exposed on
  re-activation via get_path() (which allows ROLLED_BACK status).

- OverlaySandbox rollback from COMMITTED now raises
  SandboxRollbackError if the OverlayFS unmount fails,
  preventing a double-mount attempt that would leave the
  sandbox in an inconsistent state.

- OverlaySandbox rollback from ACTIVE now uses dirs_exist_ok=True
  for userspace fallback to prevent FileExistsError when rmtree
  with ignore_errors=True silently fails.

- CopyOnWriteSandbox rollback from ACTIVE now uses
  dirs_exist_ok=True in copytree to prevent FileExistsError
  when rmtree with ignore_errors=True silently fails, matching
  the fix already applied to OverlaySandbox.

- Non-rollbackable sandboxes (NoSandbox, TransactionSandbox) are
  committed last in the batch so that all rollbackable sandboxes
  commit first; if any rollbackable sandbox fails, none of the
  non-rollbackable sandboxes will have committed yet.

- Moved NoSandbox and TransactionSandbox imports to module level
  in manager.py (no circular dependency exists).

- Pre-commit backups are now created on the same filesystem as
  the original directory (using dir= argument to mkdtemp),
  avoiding cross-device copy overhead and ensuring os.rename
  compatibility.

- safe_restore now renames the target into the mkdtemp directory
  instead of removing the mkdtemp dir first, eliminating the
  residual TOCTOU window between rmdir and rename.

- safe_restore catches BaseException (not just OSError) to
  ensure the original directory is always renamed back on
  unexpected errors, preventing the original from being left
  in the renamed-aside state.

- Added AtomicCommitError exception class to protocol.py
  carrying rolled_back_ids and failed_rollback_ids attributes.

- Exported AtomicCommitError from sandbox package __init__.py
  so callers can import it from the public API.

- Added BDD scenarios: LIFO rollback order, AtomicCommitError
  wrapping with RuntimeError cause and rollback metadata,
  _fs_utils backup/restore coverage, no-change commit rollback
  success, directory timestamp preservation, OverlaySandbox
  merged dir reset after COMMITTED rollback,
  CopyOnWriteSandbox rollback from COMMITTED restores original,
  GitWorktreeSandbox rollback from COMMITTED undoes merge,
  TransactionSandbox rollback from COMMITTED raises
  SandboxRollbackError about irreversible commit.

ISSUES CLOSED: #925

Post-review hardening (PR #1146 review findings):

- OverlaySandbox rollback from COMMITTED with no backup (no-op)
  now skips the merged directory reset entirely, preventing
  unnecessary unmount/remount or re-copy that could fail and
  turn a harmless no-op rollback into a SandboxRollbackError
  during commit_all atomic recovery.

- OverlaySandbox rollback no longer double-wraps
  SandboxRollbackError: the outer except Exception handler
  now has a preceding except SandboxRollbackError clause that
  re-raises directly, avoiding a confusing double-wrapped
  error chain.

- CopyOnWriteSandbox.get_path() now accepts ROLLED_BACK status
  for consistency with OverlaySandbox and the protocol status
  transition table (ROLLED_BACK -> ACTIVE).

- CopyOnWriteSandbox rollback from COMMITTED now resets the
  sandbox copy from the restored original via rmtree+copytree,
  preventing stale pre-rollback modifications from being
  exposed on re-activation.

- rollback_all now catches Exception (not just SandboxError)
  so that unexpected rollback errors do not prevent remaining
  sandboxes from being rolled back, consistent with the pattern
  already used in _rollback_committed.

- commit_all docstring now documents a thread-safety warning:
  the method is not safe for concurrent calls on the same
  plan_id since sandbox commit/rollback runs outside the lock.

- Fixed CHANGELOG.md whitespace inconsistencies (double leading
  spaces on two lines).

Post-review hardening round 2 (PR #1146 automated review):

- safe_restore now uses os.rename (O(1) atomic rename) instead
  of shutil.copytree (O(n) recursive copy) for the main restore
  path, since backup and target are always on the same
  filesystem.  This eliminates the ENOTEMPTY bug where a partial
  copytree failure left target_path partially populated, causing
  the recovery os.rename to fail and strand the original in the
  stale temp directory.

- OverlaySandbox.get_path() now transitions ROLLED_BACK to
  ACTIVE, matching CopyOnWriteSandbox and the protocol
  transition table (ROLLED_BACK -> ACTIVE).

- GitWorktreeSandbox.get_path() now accepts ROLLED_BACK status
  for consistency with all other sandbox implementations and
  the protocol transition table (ROLLED_BACK -> ACTIVE).

- rollback_all now also handles sandboxes in COMMITTED status
  (not just ACTIVE), consistent with the state machine allowing
  COMMITTED -> ROLLED_BACK.

- cleanup_all now catches Exception (not just SandboxError) so
  a single unexpected error does not abort cleanup of remaining
  sandboxes, consistent with _rollback_committed and
  rollback_all.

- Restructured CHANGELOG entry from a single ~90-line paragraph
  into structured sub-bullets for readability.

- Added BDD scenarios: no-op rollback from COMMITTED for
  CopyOnWriteSandbox and OverlaySandbox (zero-change commit),
  commit ordering verification (rollbackable before
  non-rollbackable).

Post-review hardening round 3 (PR #1146 deep automated review):

- cleanup_abandoned now catches Exception (not just SandboxError)
  so that unexpected errors (e.g. raw OSError, PermissionError)
  do not crash the loop and prevent remaining abandoned sandboxes
  from being cleaned up, consistent with cleanup_all,
  rollback_all, and _rollback_committed.

- OverlaySandbox._mount_overlay() now catches
  subprocess.TimeoutExpired (in addition to CalledProcessError
  and OSError), preventing create() from leaving the sandbox in
  PENDING status when mount hangs beyond the timeout.

- OverlaySandbox._unmount_overlay() now catches
  subprocess.TimeoutExpired (in addition to CalledProcessError
  and OSError), preventing cleanup() from leaving the sandbox
  in a zombie state when umount hangs beyond the timeout.

- OverlaySandbox._mount_overlay() validates that overlay paths
  do not contain commas, which would corrupt the OverlayFS mount
  options string (comma is the mount option delimiter).

- GitWorktreeSandbox.commit() now checks git diff return code
  so that a failed diff command raises CalledProcessError instead
  of silently concluding there are no changes and skipping the
  merge.

- safe_restore cleanup of the temporary rollback container now
  runs in a finally block, preventing a temp directory leak
  when the rename fails and the exception is re-raised.

- Fixed misleading BDD step name: "backup path that will cause
  copytree to fail" renamed to "backup path that will cause
  rename to fail" since safe_restore now uses os.rename.
2026-03-29 18:57:15 +01:00

413 lines
13 KiB
Python

"""Step definitions for git worktree sandbox feature.
All steps use the ``gwt`` prefix to avoid collisions with the 128+ existing
step files loaded globally by Behave.
"""
from __future__ import annotations
import os
import subprocess
import tempfile
from behave import given, then, when
from behave.runner import Context
from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox
from cleveragents.infrastructure.sandbox.protocol import (
SandboxCreationError,
SandboxStateError,
SandboxStatus,
)
def _init_test_repo(ctx: Context) -> str:
"""Create a temporary git repo with an initial commit."""
repo_dir = tempfile.mkdtemp(prefix="gwt-test-repo-")
subprocess.run(["git", "init"], cwd=repo_dir, capture_output=True, check=True)
subprocess.run(
["git", "config", "user.email", "test@test.com"],
cwd=repo_dir,
capture_output=True,
check=True,
)
subprocess.run(
["git", "config", "user.name", "Test"],
cwd=repo_dir,
capture_output=True,
check=True,
)
subprocess.run(
["git", "config", "commit.gpgSign", "false"],
cwd=repo_dir,
capture_output=True,
check=True,
)
# Create initial files
readme = os.path.join(repo_dir, "README.md")
with open(readme, "w") as f:
f.write("# Test Repo\n")
subprocess.run(["git", "add", "."], cwd=repo_dir, capture_output=True, check=True)
subprocess.run(
["git", "commit", "-m", "Initial commit"],
cwd=repo_dir,
capture_output=True,
check=True,
)
return repo_dir
# ---------------------------------------------------------------------------
# Background / Given
# ---------------------------------------------------------------------------
@given("a gwt test git repository is initialised")
def step_gwt_init_repo(ctx: Context) -> None:
ctx.gwt_repo_dir = _init_test_repo(ctx)
ctx.gwt_sandbox = None
ctx.gwt_error = None
ctx.gwt_commit_result = None
ctx.gwt_resolved_path = None
@given("a gwt non-git directory")
def step_gwt_non_git_dir(ctx: Context) -> None:
ctx.gwt_non_git_dir = tempfile.mkdtemp(prefix="gwt-non-git-")
# ---------------------------------------------------------------------------
# When - instantiation
# ---------------------------------------------------------------------------
@when("a gwt sandbox is instantiated")
def step_gwt_instantiate(ctx: Context) -> None:
ctx.gwt_sandbox = GitWorktreeSandbox(
resource_id="res-001",
original_path=ctx.gwt_repo_dir,
)
@when("a gwt sandbox is prepared with empty resource_id")
def step_gwt_empty_resource_id(ctx: Context) -> None:
try:
GitWorktreeSandbox(resource_id="", original_path=ctx.gwt_repo_dir)
except ValueError as exc:
ctx.gwt_error = exc
@when("a gwt sandbox is prepared with empty original_path")
def step_gwt_empty_original_path(ctx: Context) -> None:
try:
GitWorktreeSandbox(resource_id="res-001", original_path="")
except ValueError as exc:
ctx.gwt_error = exc
@when("a gwt sandbox is prepared with zero timeout")
def step_gwt_zero_timeout(ctx: Context) -> None:
try:
GitWorktreeSandbox(
resource_id="res-001",
original_path=ctx.gwt_repo_dir,
git_timeout=0,
)
except ValueError as exc:
ctx.gwt_error = exc
# ---------------------------------------------------------------------------
# When - create
# ---------------------------------------------------------------------------
@when('a gwt sandbox is created for plan "{plan_id}"')
def step_gwt_create(ctx: Context, plan_id: str) -> None:
ctx.gwt_sandbox = GitWorktreeSandbox(
resource_id="res-001",
original_path=ctx.gwt_repo_dir,
)
ctx.gwt_sandbox.create(plan_id)
@when("a gwt sandbox is created with empty plan_id")
def step_gwt_create_empty_plan(ctx: Context) -> None:
ctx.gwt_sandbox = GitWorktreeSandbox(
resource_id="res-001",
original_path=ctx.gwt_repo_dir,
)
try:
ctx.gwt_sandbox.create("")
except ValueError as exc:
ctx.gwt_error = exc
@when('a gwt sandbox is created on the non-git directory for plan "{plan_id}"')
def step_gwt_create_non_git(ctx: Context, plan_id: str) -> None:
ctx.gwt_sandbox = GitWorktreeSandbox(
resource_id="res-001",
original_path=ctx.gwt_non_git_dir,
)
try:
ctx.gwt_sandbox.create(plan_id)
except SandboxCreationError as exc:
ctx.gwt_error = exc
# ---------------------------------------------------------------------------
# When - path resolution
# ---------------------------------------------------------------------------
@when('the gwt path "{path}" is resolved')
def step_gwt_resolve_path(ctx: Context, path: str) -> None:
try:
ctx.gwt_resolved_path = ctx.gwt_sandbox.get_path(path)
except (ValueError, SandboxStateError) as exc:
ctx.gwt_error = exc
@when('the gwt path "{path}" is resolved on a cleaned-up sandbox')
def step_gwt_resolve_path_cleaned(ctx: Context, path: str) -> None:
try:
ctx.gwt_sandbox.get_path(path)
except SandboxStateError as exc:
ctx.gwt_error = exc
# ---------------------------------------------------------------------------
# When - file operations in worktree
# ---------------------------------------------------------------------------
@when('a gwt file "{filename}" is created in the worktree with content "{content}"')
def step_gwt_create_file(ctx: Context, filename: str, content: str) -> None:
worktree_path = ctx.gwt_sandbox.context.sandbox_path
file_path = os.path.join(worktree_path, filename)
os.makedirs(os.path.dirname(file_path), exist_ok=True)
with open(file_path, "w") as f:
f.write(content)
@when('the gwt existing file "{filename}" is modified in the worktree')
def step_gwt_modify_file(ctx: Context, filename: str) -> None:
worktree_path = ctx.gwt_sandbox.context.sandbox_path
file_path = os.path.join(worktree_path, filename)
with open(file_path, "a") as f:
f.write("\n# Modified by sandbox\n")
# ---------------------------------------------------------------------------
# When - commit / rollback / cleanup
# ---------------------------------------------------------------------------
@when('the gwt sandbox is committed with message "{message}"')
def step_gwt_commit(ctx: Context, message: str) -> None:
try:
ctx.gwt_commit_result = ctx.gwt_sandbox.commit(message)
except (SandboxStateError, Exception) as exc:
ctx.gwt_error = exc
@when("the gwt sandbox commit is attempted on cleaned-up sandbox")
def step_gwt_commit_cleaned(ctx: Context) -> None:
try:
ctx.gwt_sandbox.commit("should fail")
except SandboxStateError as exc:
ctx.gwt_error = exc
@when("the gwt sandbox is rolled back")
def step_gwt_rollback(ctx: Context) -> None:
try:
ctx.gwt_sandbox.rollback()
except (SandboxStateError, Exception) as exc:
ctx.gwt_error = exc
@when("the gwt sandbox rollback is attempted on created sandbox")
def step_gwt_rollback_created(ctx: Context) -> None:
try:
ctx.gwt_sandbox.rollback()
except SandboxStateError as exc:
ctx.gwt_error = exc
@when("the gwt sandbox is cleaned up")
def step_gwt_cleanup(ctx: Context) -> None:
ctx.gwt_sandbox.cleanup()
@when("the gwt sandbox is cleaned up again")
def step_gwt_cleanup_again(ctx: Context) -> None:
ctx.gwt_sandbox.cleanup()
# ---------------------------------------------------------------------------
# Then - status / state
# ---------------------------------------------------------------------------
@then('the gwt sandbox should be in the "{status}" state')
def step_gwt_check_status(ctx: Context, status: str) -> None:
expected = SandboxStatus(status)
assert ctx.gwt_sandbox.status == expected, (
f"Expected status {expected}, got {ctx.gwt_sandbox.status}"
)
@then('the gwt sandbox context should reference plan "{plan_id}"')
def step_gwt_check_plan(ctx: Context, plan_id: str) -> None:
assert ctx.gwt_sandbox.context is not None
assert ctx.gwt_sandbox.context.plan_id == plan_id
@then('the gwt sandbox context should have strategy metadata "{strategy}"')
def step_gwt_check_strategy(ctx: Context, strategy: str) -> None:
assert ctx.gwt_sandbox.context is not None
assert ctx.gwt_sandbox.context.metadata.get("strategy") == strategy
@then("the gwt sandbox worktree path should exist")
def step_gwt_worktree_exists(ctx: Context) -> None:
assert ctx.gwt_sandbox.context is not None
assert os.path.isdir(ctx.gwt_sandbox.context.sandbox_path)
@then("the gwt sandbox worktree path should not exist")
def step_gwt_worktree_not_exists(ctx: Context) -> None:
if ctx.gwt_sandbox.context is not None:
assert not os.path.exists(ctx.gwt_sandbox.context.sandbox_path)
@then("the gwt sandbox branch should not exist")
def step_gwt_branch_not_exists(ctx: Context) -> None:
result = subprocess.run(
["git", "branch", "--list"],
cwd=ctx.gwt_repo_dir,
capture_output=True,
text=True,
check=False,
)
branch_name = ctx.gwt_sandbox._branch_name
if branch_name:
assert branch_name not in result.stdout
# ---------------------------------------------------------------------------
# Then - errors
# ---------------------------------------------------------------------------
@then('a gwt ValueError should be raised with message "{msg}"')
def step_gwt_valueerror(ctx: Context, msg: str) -> None:
assert ctx.gwt_error is not None, "Expected an error but none occurred"
assert isinstance(ctx.gwt_error, ValueError), (
f"Expected ValueError, got {type(ctx.gwt_error).__name__}"
)
assert msg in str(ctx.gwt_error), (
f"Expected '{msg}' in error message, got: {ctx.gwt_error}"
)
@then("a gwt SandboxCreationError should be raised")
def step_gwt_creation_error(ctx: Context) -> None:
assert ctx.gwt_error is not None
assert isinstance(ctx.gwt_error, SandboxCreationError)
@then("a gwt SandboxStateError should be raised")
def step_gwt_state_error(ctx: Context) -> None:
assert ctx.gwt_error is not None
assert isinstance(ctx.gwt_error, SandboxStateError)
# ---------------------------------------------------------------------------
# Then - path resolution
# ---------------------------------------------------------------------------
@then("the gwt resolved path should be inside the worktree")
def step_gwt_path_in_worktree(ctx: Context) -> None:
assert ctx.gwt_resolved_path is not None
sandbox_path = ctx.gwt_sandbox.context.sandbox_path
assert ctx.gwt_resolved_path.startswith(sandbox_path)
# ---------------------------------------------------------------------------
# Then - commit results
# ---------------------------------------------------------------------------
@then("the gwt commit result should indicate success")
def step_gwt_commit_success(ctx: Context) -> None:
assert ctx.gwt_commit_result is not None
assert ctx.gwt_commit_result.success is True
@then("the gwt commit result should have {count:d} changed files")
def step_gwt_commit_changed(ctx: Context, count: int) -> None:
assert ctx.gwt_commit_result is not None
assert len(ctx.gwt_commit_result.changed_files) == count
@then("the gwt commit result should have {count:d} added files")
def step_gwt_commit_added(ctx: Context, count: int) -> None:
assert ctx.gwt_commit_result is not None
assert len(ctx.gwt_commit_result.added_files) == count
@then('the gwt file "{filename}" should exist in the original repo')
def step_gwt_file_in_original(ctx: Context, filename: str) -> None:
file_path = os.path.join(ctx.gwt_repo_dir, filename)
assert os.path.exists(file_path), f"File {filename} not found in original repo"
@then('the gwt file "{filename}" should not exist in the original repo')
def step_gwt_file_not_in_original(ctx: Context, filename: str) -> None:
file_path = os.path.join(ctx.gwt_repo_dir, filename)
assert not os.path.exists(file_path), (
f"File {filename} should not exist in original repo after rollback"
)
@then('the gwt file "{filename}" should not exist in the worktree')
def step_gwt_file_not_in_worktree(ctx: Context, filename: str) -> None:
if ctx.gwt_sandbox.context:
file_path = os.path.join(ctx.gwt_sandbox.context.sandbox_path, filename)
assert not os.path.exists(file_path)
# ---------------------------------------------------------------------------
# Then - protocol properties
# ---------------------------------------------------------------------------
@then("the gwt sandbox_id should be a valid ULID")
def step_gwt_valid_ulid(ctx: Context) -> None:
sid = ctx.gwt_sandbox.sandbox_id
assert len(sid) == 26, f"Expected ULID length 26, got {len(sid)}"
@then('the gwt sandbox status should be "{status}"')
def step_gwt_status_is(ctx: Context, status: str) -> None:
assert ctx.gwt_sandbox.status == SandboxStatus(status)
@then("the gwt sandbox context should be None")
def step_gwt_context_none(ctx: Context) -> None:
assert ctx.gwt_sandbox.context is None
@then("the gwt sandbox branch name should be safe for git")
def step_gwt_branch_safe(ctx: Context) -> None:
branch = ctx.gwt_sandbox._branch_name
assert branch is not None
# Branch name should not contain spaces or special chars
assert " " not in branch
assert "!" not in branch
assert "@" not in branch
assert "#" not in branch