forked from cleveragents/cleveragents-core
CoreRasurae
f2f7aa5dc9
Implement multiple Stage A/B/E/SEC milestones for the v3 lifecycle system: - Stage A5.3+A5.4: Add LifecycleActionModel and LifecyclePlanModel SQLAlchemy models with to_domain()/from_domain() conversion methods - Stage A5.6: Implement ActionRepository with full CRUD, namespace/state queries, referential integrity checks, and retry decorator - Stage E1: Add subplan domain models (ExecutionMode, SubplanMergeStrategy, SubplanConfig, SubplanStatus, SubplanAttempt, SubplanFailureHandler) with computed properties on Plan (is_subplan, is_root_plan, depth, has_subplans) - Stage A6: Add AutomationLevel enum (MANUAL, REVIEW_BEFORE_APPLY, FULL_AUTOMATION), settings integration, PlanLifecycleService auto-progression, pause/resume, and CLI commands (--automation-level, set-automation-level) - Stage SEC1: Remove eval()/exec() from stream_router.py, replace with named operation and transform registries; code blocks and unregistered transforms now raise StreamRoutingError - Add langchain-anthropic dependency - Update BDD tests for security changes and relax ADR directory requirement
281 lines
15 KiB
Gherkin
281 lines
15 KiB
Gherkin
Feature: Sandbox Manager Lifecycle
|
|
As an execution engine
|
|
I want a manager that tracks sandbox instances across plan executions
|
|
So that resources are isolated during execution and cleaned up reliably
|
|
|
|
Background:
|
|
Given a sandbox factory instance
|
|
And a sandbox manager with the factory
|
|
|
|
# Initialisation
|
|
|
|
Scenario: A new manager starts with no tracked sandboxes
|
|
Then the manager should have no active sandboxes
|
|
And the manager cleanup_on_exit flag should be true
|
|
|
|
Scenario: A manager can be created without automatic exit cleanup
|
|
Given a sandbox manager with cleanup_on_exit disabled
|
|
Then the manager cleanup_on_exit flag should be false
|
|
|
|
Scenario: Creating a manager without a factory is rejected
|
|
When I create a sandbox manager with None factory
|
|
Then a ValueError should be raised with message "factory cannot be None"
|
|
|
|
# Lazy sandbox creation
|
|
|
|
Scenario: Requesting a sandbox for a plan creates and tracks a new sandbox
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then a sandbox should be returned
|
|
And the sandbox should have status "created"
|
|
And the sandbox should be tracked for plan "plan-001"
|
|
|
|
Scenario: Requesting the same plan-resource pair returns the existing sandbox
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then the same sandbox instance should be returned
|
|
|
|
Scenario: An active sandbox is reused when requested again
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then the same sandbox instance should be returned
|
|
|
|
Scenario: A committed sandbox is replaced by a fresh instance on next request
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is committed
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then a different sandbox instance should be returned
|
|
|
|
Scenario: A cleaned-up sandbox is replaced by a fresh instance on next request
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is cleaned up
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then a different sandbox instance should be returned
|
|
|
|
Scenario: A rolled-back sandbox is still usable and returned on next request
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is rolled back
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then the same sandbox instance should be returned
|
|
|
|
Scenario: An errored sandbox is replaced by a fresh instance on next request
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is in errored state
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
Then a different sandbox instance should be returned
|
|
|
|
# Argument validation on sandbox creation
|
|
|
|
Scenario: A sandbox request without a plan identifier is rejected
|
|
When I get or create a sandbox with empty plan_id
|
|
Then a ValueError should be raised with message "plan_id cannot be empty"
|
|
|
|
Scenario: A sandbox request without a resource identifier is rejected
|
|
When I get or create a sandbox with empty resource_id
|
|
Then a ValueError should be raised with message "resource_id cannot be empty"
|
|
|
|
Scenario: A sandbox request without an original path is rejected
|
|
When I get or create a sandbox with empty original_path
|
|
Then a ValueError should be raised with message "original_path cannot be empty"
|
|
|
|
# Multi-resource and multi-plan tracking
|
|
|
|
Scenario: Multiple resources under the same plan are tracked independently
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And I get or create a sandbox for plan "plan-001" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
Then 2 sandboxes should be tracked for plan "plan-001"
|
|
|
|
Scenario: Sandboxes from different plans do not interfere with each other
|
|
When I get or create a sandbox for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And I get or create a sandbox for plan "plan-002" resource "res-001" path "/tmp/repo2" strategy "none"
|
|
Then 1 sandboxes should be tracked for plan "plan-001"
|
|
And 1 sandboxes should be tracked for plan "plan-002"
|
|
|
|
# Looking up an existing sandbox
|
|
|
|
Scenario: An existing sandbox can be retrieved by plan and resource
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
When I get sandbox for plan "plan-001" resource "res-001"
|
|
Then a sandbox should be returned
|
|
|
|
Scenario: Looking up a sandbox for an unknown plan returns nothing
|
|
When I get sandbox for plan "plan-999" resource "res-999"
|
|
Then the sandbox result should be None
|
|
|
|
Scenario: Looking up a sandbox for a known plan but unknown resource returns nothing
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo" strategy "none"
|
|
When I get sandbox for plan "plan-001" resource "res-999"
|
|
Then the sandbox result should be None
|
|
|
|
# Listing sandboxes for a plan
|
|
|
|
Scenario: All sandboxes for a plan can be listed
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And a sandbox exists for plan "plan-001" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
When I list sandboxes for plan "plan-001"
|
|
Then the sandbox list should contain 2 sandboxes
|
|
|
|
Scenario: Listing sandboxes for a plan with none returns an empty list
|
|
When I list sandboxes for plan "plan-999"
|
|
Then the sandbox list should contain 0 sandboxes
|
|
|
|
# Batch commit
|
|
|
|
Scenario: All active sandboxes for a plan are committed together
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
And a sandbox exists for plan "plan-001" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-002" is activated
|
|
When I commit all sandboxes for plan "plan-001"
|
|
Then 2 commit results should be returned
|
|
And all commit results should be successful
|
|
|
|
Scenario: Batch commit skips sandboxes that are already committed
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is committed
|
|
When I commit all sandboxes for plan "plan-001"
|
|
Then 0 commit results should be returned
|
|
|
|
Scenario: Batch commit on a plan with no sandboxes returns an empty result set
|
|
When I commit all sandboxes for plan "plan-999"
|
|
Then 0 commit results should be returned
|
|
|
|
Scenario: Batch commit without a plan identifier is rejected
|
|
When I commit all sandboxes with empty plan_id
|
|
Then a ValueError should be raised with message "plan_id cannot be empty"
|
|
|
|
Scenario: A single sandbox failure during batch commit does not abort the others
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
And the sandbox for plan "plan-001" resource "res-001" will fail on commit
|
|
When I commit all sandboxes for plan "plan-001"
|
|
Then 1 commit results should be returned
|
|
And the first commit result should have success false
|
|
|
|
Scenario: Newly created sandboxes can be committed before activation
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
When I commit all sandboxes for plan "plan-001"
|
|
Then 1 commit results should be returned
|
|
And all commit results should be successful
|
|
|
|
# Batch rollback
|
|
|
|
Scenario: All active sandboxes for a plan are rolled back together
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
When I rollback all sandboxes for plan "plan-001"
|
|
Then no sandbox error should be raised
|
|
|
|
Scenario: Batch rollback skips sandboxes that are not active
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
When I rollback all sandboxes for plan "plan-001"
|
|
Then no sandbox error should be raised
|
|
|
|
Scenario: Batch rollback on a plan with no sandboxes succeeds silently
|
|
When I rollback all sandboxes for plan "plan-999"
|
|
Then no sandbox error should be raised
|
|
|
|
Scenario: Batch rollback without a plan identifier is rejected
|
|
When I rollback all sandboxes with empty plan_id
|
|
Then a ValueError should be raised with message "plan_id cannot be empty"
|
|
|
|
Scenario: A single sandbox failure during batch rollback does not abort the others
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
And the sandbox for plan "plan-001" resource "res-001" will fail on rollback
|
|
When I rollback all sandboxes for plan "plan-001"
|
|
Then no sandbox error should be raised
|
|
|
|
# Batch cleanup
|
|
|
|
Scenario: All sandboxes for a plan are cleaned up and the plan is untracked
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And a sandbox exists for plan "plan-001" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
When I cleanup all sandboxes for plan "plan-001"
|
|
Then no sandbox error should be raised
|
|
And plan "plan-001" should no longer be tracked
|
|
|
|
Scenario: Batch cleanup without a plan identifier is rejected
|
|
When I cleanup all sandboxes with empty plan_id
|
|
Then a ValueError should be raised with message "plan_id cannot be empty"
|
|
|
|
Scenario: Batch cleanup on a plan with no sandboxes succeeds silently
|
|
When I cleanup all sandboxes for plan "plan-999"
|
|
Then no sandbox error should be raised
|
|
|
|
Scenario: A single sandbox failure during batch cleanup does not prevent untracking
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" will fail on cleanup
|
|
When I cleanup all sandboxes for plan "plan-001"
|
|
Then no sandbox error should be raised
|
|
And plan "plan-001" should no longer be tracked
|
|
|
|
# Abandoned sandbox cleanup
|
|
|
|
Scenario: Errored sandboxes are cleaned up as abandoned
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is in errored state
|
|
When I cleanup abandoned sandboxes
|
|
Then 1 abandoned sandboxes should be cleaned up
|
|
|
|
Scenario: Committed sandboxes are cleaned up as abandoned
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is committed
|
|
When I cleanup abandoned sandboxes
|
|
Then 1 abandoned sandboxes should be cleaned up
|
|
|
|
Scenario: Rolled-back sandboxes are cleaned up as abandoned
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is rolled back
|
|
When I cleanup abandoned sandboxes
|
|
Then 1 abandoned sandboxes should be cleaned up
|
|
|
|
Scenario: Active sandboxes are not considered abandoned
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is activated
|
|
When I cleanup abandoned sandboxes
|
|
Then 0 abandoned sandboxes should be cleaned up
|
|
|
|
Scenario: A plan entry is removed once all of its sandboxes have been cleaned up
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is committed
|
|
When I cleanup abandoned sandboxes
|
|
Then 1 abandoned sandboxes should be cleaned up
|
|
And plan "plan-001" should no longer be tracked
|
|
|
|
Scenario: Cleanup errors on individual abandoned sandboxes are contained
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is in errored state
|
|
And the sandbox for plan "plan-001" resource "res-001" will fail on cleanup
|
|
When I cleanup abandoned sandboxes
|
|
Then 0 abandoned sandboxes should be cleaned up
|
|
|
|
Scenario: Abandoned sandbox cleanup operates across all tracked plans
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" is committed
|
|
And a sandbox exists for plan "plan-002" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
And the sandbox for plan "plan-002" resource "res-002" is in errored state
|
|
When I cleanup abandoned sandboxes
|
|
Then 2 abandoned sandboxes should be cleaned up
|
|
|
|
# Graceful process exit
|
|
|
|
Scenario: The exit handler cleans up all remaining sandboxes
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And a sandbox exists for plan "plan-002" resource "res-002" path "/tmp/repo2" strategy "none"
|
|
When the atexit cleanup handler runs
|
|
Then plan "plan-001" should no longer be tracked
|
|
And plan "plan-002" should no longer be tracked
|
|
|
|
Scenario: The exit handler tolerates individual sandbox cleanup failures
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And the sandbox for plan "plan-001" resource "res-001" will fail on cleanup
|
|
When the atexit cleanup handler runs
|
|
Then no sandbox error should be raised
|
|
|
|
Scenario: The exit handler swallows exceptions from the cleanup subsystem
|
|
Given a sandbox exists for plan "plan-001" resource "res-001" path "/tmp/repo1" strategy "none"
|
|
And cleanup_all is patched to raise a RuntimeError
|
|
When the atexit cleanup handler runs
|
|
Then no sandbox error should be raised
|