Files
cleveragents-core/features/checkpoint_rollback.feature
CoreRasurae 86e245c585
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 37s
CI / unit_tests (pull_request) Successful in 2m22s
CI / integration_tests (pull_request) Successful in 2m51s
CI / docker (pull_request) Successful in 38s
CI / coverage (pull_request) Successful in 3m34s
CI / benchmark-regression (pull_request) Successful in 22m10s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 18s
CI / security (push) Successful in 32s
CI / typecheck (push) Successful in 34s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 1m55s
CI / docker (push) Successful in 38s
CI / integration_tests (push) Successful in 2m52s
CI / coverage (push) Successful in 4m4s
CI / benchmark-publish (push) Successful in 13m13s
feat(checkpoint): add checkpointing and rollback
fix Alembic migration down_revision to chain after
  m4_002_skill_flattened_tools (was pointing to stale
  c3_001_actor_registry)

BDD coverage: 33 scenarios (7 new), 129 steps, all passing.

ISSUES CLOSED: #206
2026-03-02 10:18:14 +00:00

297 lines
17 KiB
Gherkin

@phase2 @checkpoint @rollback
Feature: Checkpoint and rollback
As an operator
I want to create checkpoints during plan execution and rollback to them
So that I can recover from mistakes without losing all progress
#
# Domain model scenarios
#
Scenario: Checkpoint model validates ULID fields
Given a valid checkpoint with plan_id "01ARZ3NDEKTSV4RRFFQ69G5FAV" and sandbox_ref "abc123"
Then the checkpoint plan_id should be "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And the checkpoint sandbox_ref should be "abc123"
Scenario: Checkpoint metadata stores reason and source tool
Given a checkpoint with reason "pre-execution" and source_tool "lint-check" and phase "execute"
Then the checkpoint metadata reason should be "pre-execution"
And the checkpoint metadata source_tool should be "lint-check"
And the checkpoint metadata phase should be "execute"
Scenario: Checkpoint retention policy defaults
Given a default retention policy
Then the max_checkpoints should be 50
And auto_prune should be true
Scenario: Checkpoint retention policy custom values
Given a retention policy with max_checkpoints 5 and auto_prune false
Then the max_checkpoints should be 5
And auto_prune should be false
Scenario: RollbackResult model stores restored file count
Given a rollback result with 3 restored files and checkpoint "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Then the restored_files_count should be 3
And the from_checkpoint_id should be "01ARZ3NDEKTSV4RRFFQ69G5FAV"
# ───────────────────────────────────────────────────────────
# CheckpointService scenarios
# ───────────────────────────────────────────────────────────
Scenario: Create a checkpoint
Given a checkpoint service
When I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
Then the checkpoint should be created successfully
And the checkpoint plan_id should match "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Scenario: List checkpoints for a plan
Given a checkpoint service
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-1"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-2"
When I list checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Then I should see 2 checkpoints
Scenario: Rollback to checkpoint succeeds
Given a checkpoint service with sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should succeed
And the rollback result should show restored files
Scenario: Rollback rejected when plan is applied
Given a checkpoint service
And plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" is marked as applied
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "plan is already applied"
Scenario: Rollback rejected when sandbox is missing
Given a checkpoint service
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "sandbox is missing"
Scenario: Rollback with non-existent checkpoint
Given a checkpoint service with sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to checkpoint "01NONEXISTENT0000000000000"
Then the rollback should fail with not found error
Scenario: Delete a checkpoint
Given a checkpoint service
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I delete the created checkpoint
Then the checkpoint should be deleted
Scenario: Prune enforces retention policy
Given a checkpoint service
And I create 5 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I prune checkpoints with max 3
Then 2 checkpoints should be pruned
And 3 checkpoint snapshots should remain
Scenario: Auto-prune on create with retention policy
Given a checkpoint service
And I create 10 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I create a checkpoint with retention policy max 5
Then 6 checkpoints should have been pruned on creation
And 5 checkpoint snapshots should remain
Scenario: Create checkpoint with empty plan_id raises error
Given a checkpoint service
When I attempt to create a checkpoint with empty plan_id
Then a checkpoint validation error should be raised
Scenario: Create checkpoint with empty sandbox_ref raises error
Given a checkpoint service
When I attempt to create a checkpoint with empty sandbox_ref
Then a checkpoint validation error should be raised
Scenario: Get checkpoint by ID
Given a checkpoint service
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-xyz"
When I get the checkpoint by its ID
Then the checkpoint should be returned successfully
Scenario: Get non-existent checkpoint raises error
Given a checkpoint service
When I attempt to get checkpoint "01NONEXISTENT0000000000000"
Then a checkpoint not found error should be raised
# ───────────────────────────────────────────────────────────
# Correction service rollback integration
# ───────────────────────────────────────────────────────────
Scenario: Correction service accepts checkpoint_service parameter
Given a correction service with checkpoint support
Then the correction service should have checkpoint support
Scenario: Checkpoint metadata reason stored for auditability
Given a checkpoint service
When I create a checkpoint with reason "pre-tool-run" and source_tool "format-code" and phase "execute"
Then the stored checkpoint metadata should have reason "pre-tool-run"
And the stored checkpoint metadata should have source_tool "format-code"
And the stored checkpoint metadata should have phase "execute"
# ───────────────────────────────────────────────────────────
# Spec-aligned field scenarios (decision_id, checkpoint_type, etc.)
# ───────────────────────────────────────────────────────────
Scenario: Checkpoint stores decision_id when aligned to a decision
Given a checkpoint service
When I create a checkpoint aligned to decision "01DEC1S10N0000000000000000"
Then the stored checkpoint decision_id should be "01DEC1S10N0000000000000000"
Scenario: Checkpoint stores checkpoint_type
Given a checkpoint service
When I create a pre_write checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Then the stored checkpoint_type should be "pre_write"
Scenario: Checkpoint type validation rejects invalid values
Given a checkpoint service
When I attempt to create a checkpoint with type "invalid_type"
Then a checkpoint validation error should be raised
Scenario: Checkpoint stores resource_id
Given a checkpoint service
When I create a checkpoint with resource "01RES0URCE0000000000000000"
Then the stored checkpoint resource_id should be "01RES0URCE0000000000000000"
Scenario: Checkpoint stores filesystem_path and size_bytes
Given a checkpoint service
When I create a checkpoint with path "snapshots/cp001" and size 4096
Then the stored checkpoint filesystem_path should be "snapshots/cp001"
And the stored checkpoint size_bytes should be 4096
Scenario: Prune preserves first and most recent checkpoints
Given a checkpoint service
And I create 5 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I prune checkpoints with max 2
Then 3 checkpoints should be pruned
And 2 checkpoint snapshots should remain
And the first checkpoint should survive pruning
And the most recent checkpoint should survive pruning
# ───────────────────────────────────────────────────────────
# Persistent guard scenarios (via PlanLifecycleService)
# ───────────────────────────────────────────────────────────
Scenario: Rollback rejected via lifecycle service when plan is applied
Given a checkpoint service backed by a lifecycle service with an applied plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "plan is already applied"
Scenario: Rollback rejected via lifecycle service when sandbox refs are empty
Given a checkpoint service backed by a lifecycle service with no sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "sandbox is missing"
Scenario: Rollback succeeds via lifecycle service when plan has sandbox
Given a checkpoint service backed by a lifecycle service with sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
When I rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should succeed
And the rollback result should show restored files
# ───────────────────────────────────────────────────────────
# Default retention policy scenario
# ───────────────────────────────────────────────────────────
Scenario: Default retention policy applied when none specified
Given a checkpoint service
And I create 55 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-final"
Then at most 50 checkpoint snapshots should remain for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Scenario: Rollback rejected when checkpoint belongs to different plan
Given a checkpoint service with sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01KJGSHD2JKTFZGGBT6FRQD2W2" with sandbox_ref "commit-other"
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "does not belong to plan"
Scenario: Unregister sandbox prevents rollback
Given a checkpoint service with sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-abc"
And the sandbox for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" is unregistered
When I attempt rollback plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the created checkpoint
Then the rollback should be rejected with "sandbox is missing"
Scenario: Prune with auto_prune disabled returns empty
Given a checkpoint service
And I create 5 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I prune checkpoints with auto_prune disabled
Then 0 checkpoints should be pruned
# ───────────────────────────────────────────────────────────
# Repository-backed service scenarios
# ───────────────────────────────────────────────────────────
Scenario: Repository-backed create persists via repository
Given a checkpoint service backed by a stub repository
When I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-repo"
Then the checkpoint should be created successfully
And the stub repository should have 1 checkpoint stored
Scenario: Repository-backed list delegates to repository
Given a checkpoint service backed by a stub repository
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-r1"
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-r2"
When I list checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Then I should see 2 checkpoints
Scenario: Repository-backed delete removes via repository
Given a checkpoint service backed by a stub repository
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-del"
When I delete the created checkpoint
Then the checkpoint should be deleted
Scenario: Repository-backed delete raises error when not found
Given a checkpoint service backed by a stub repository
When I attempt to delete checkpoint "01NONEXISTENT0000000000000"
Then a checkpoint not found error should be raised
Scenario: Repository-backed prune delegates to repository
Given a checkpoint service backed by a stub repository
And I create 5 checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
When I prune checkpoints with max 3
Then 2 checkpoints should be pruned
Scenario: Repository-backed get retrieves via repository
Given a checkpoint service backed by a stub repository
And I create a checkpoint for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" with sandbox_ref "commit-get"
When I get the checkpoint by its ID
Then the checkpoint should be returned successfully
Scenario: Repository-backed get raises not found for missing checkpoint
Given a checkpoint service backed by a stub repository
When I attempt to get checkpoint "01NONEXISTENT0000000000000"
Then a checkpoint not found error should be raised
# ───────────────────────────────────────────────────────────
# In-memory edge case scenarios
# ───────────────────────────────────────────────────────────
Scenario: Delete non-existent checkpoint from in-memory store raises error
Given a checkpoint service
When I attempt to delete checkpoint "01NONEXISTENT0000000000000"
Then a checkpoint not found error should be raised
Scenario: List checkpoints tolerates orphaned index entries
Given a checkpoint service with an orphaned plan index entry
When I list checkpoints for plan "01ARZ3NDEKTSV4RRFFQ69G5FAV"
Then I should see 1 checkpoints
Scenario: Delete checkpoint whose plan index is missing skips cleanup
Given a checkpoint service with a checkpoint missing from plan index
When I delete the created checkpoint
Then the checkpoint should be deleted
# ───────────────────────────────────────────────────────────
# Domain model validator edge cases
# ───────────────────────────────────────────────────────────
Scenario: Checkpoint ULID validator rejects empty string
When I invoke the checkpoint ULID validator with an empty string
Then the checkpoint validator should raise ValueError "ULID field must not be empty"