CoreRasurae
f2f7aa5dc9
Implement multiple Stage A/B/E/SEC milestones for the v3 lifecycle system: - Stage A5.3+A5.4: Add LifecycleActionModel and LifecyclePlanModel SQLAlchemy models with to_domain()/from_domain() conversion methods - Stage A5.6: Implement ActionRepository with full CRUD, namespace/state queries, referential integrity checks, and retry decorator - Stage E1: Add subplan domain models (ExecutionMode, SubplanMergeStrategy, SubplanConfig, SubplanStatus, SubplanAttempt, SubplanFailureHandler) with computed properties on Plan (is_subplan, is_root_plan, depth, has_subplans) - Stage A6: Add AutomationLevel enum (MANUAL, REVIEW_BEFORE_APPLY, FULL_AUTOMATION), settings integration, PlanLifecycleService auto-progression, pause/resume, and CLI commands (--automation-level, set-automation-level) - Stage SEC1: Remove eval()/exec() from stream_router.py, replace with named operation and transform registries; code blocks and unregistered transforms now raise StreamRoutingError - Add langchain-anthropic dependency - Update BDD tests for security changes and relax ADR directory requirement
145 lines
6.8 KiB
Gherkin
145 lines
6.8 KiB
Gherkin
Feature: Automation Levels
|
|
As a developer using CleverAgents
|
|
I want to control how automated phase transitions are
|
|
So that I can choose between manual control, review-before-apply, and full automation
|
|
|
|
Background:
|
|
Given I have a plan lifecycle service with automation level support
|
|
|
|
# Manual Mode Tests
|
|
|
|
Scenario: Manual mode requires explicit execute command
|
|
Given I have a plan with automation level "manual" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "strategize"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
Scenario: Manual mode requires explicit apply command
|
|
Given I have a plan with automation level "manual" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
# Review-Before-Apply Tests
|
|
|
|
Scenario: Review-before-apply auto-executes after strategize completes
|
|
Given I have a plan with automation level "review_before_apply" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
Scenario: Review-before-apply pauses at apply
|
|
Given I have a plan with automation level "review_before_apply" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
# Full Automation Tests
|
|
|
|
Scenario: Full automation auto-executes after strategize completes
|
|
Given I have a plan with automation level "full_automation" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
Scenario: Full automation auto-applies after execute completes
|
|
Given I have a plan with automation level "full_automation" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "apply"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
# Plan-Level Override Tests
|
|
|
|
Scenario: Plan-level automation overrides global setting
|
|
Given the global automation level is "manual"
|
|
And I have a plan created with explicit automation level "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Use action with explicit automation level
|
|
Given I have an available action for automation tests
|
|
When I use the action with automation level "review_before_apply" on project "project-auto-1"
|
|
Then the plan automation level should be "review_before_apply"
|
|
|
|
Scenario: Use action without explicit automation level uses global default
|
|
Given the global automation level is "manual"
|
|
And I have an available action for automation tests
|
|
When I use the action without explicit automation level on project "project-auto-2"
|
|
Then the plan automation level should be "manual"
|
|
|
|
# Change Automation Level Mid-Plan
|
|
|
|
Scenario: Change automation level mid-plan works correctly
|
|
Given I have a plan with automation level "manual" in strategize phase with queued state
|
|
When I set the plan automation level to "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Cannot change automation level on terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to set the plan automation level to "full_automation"
|
|
Then a plan error should be raised for automation
|
|
|
|
# should_auto_progress Query Tests
|
|
|
|
Scenario: should_auto_progress returns false for manual plan in strategize complete
|
|
Given I have a plan with automation level "manual" in strategize phase with complete state for query
|
|
Then should_auto_progress should return false
|
|
|
|
Scenario: should_auto_progress returns true for review-before-apply plan in strategize complete
|
|
Given I have a plan with automation level "review_before_apply" in strategize phase with complete state for query
|
|
Then should_auto_progress should return true
|
|
|
|
Scenario: should_auto_progress returns false for review-before-apply plan in execute complete
|
|
Given I have a plan with automation level "review_before_apply" in execute phase with complete state for query
|
|
Then should_auto_progress should return false
|
|
|
|
Scenario: should_auto_progress returns true for full-automation plan in execute complete
|
|
Given I have a plan with automation level "full_automation" in execute phase with complete state for query
|
|
Then should_auto_progress should return true
|
|
|
|
Scenario: should_auto_progress returns false for terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
Then should_auto_progress on terminal plan should return false
|
|
|
|
# Pause and Resume Tests
|
|
|
|
Scenario: Pause plan sets automation to manual
|
|
Given I have a plan with automation level "full_automation" in strategize phase with queued state
|
|
When I pause the plan
|
|
Then the plan automation level should be "manual"
|
|
|
|
Scenario: Cannot pause terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to pause the terminal plan
|
|
Then a plan error should be raised for automation
|
|
|
|
Scenario: Resume plan restores automation level
|
|
Given I have a paused plan that was previously "full_automation" in strategize phase
|
|
When I resume the plan with level "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Resume plan without explicit level defaults to review-before-apply
|
|
Given I have a paused plan that was previously "full_automation" in strategize phase
|
|
When I resume the plan without explicit level
|
|
Then the plan automation level should be "review_before_apply"
|
|
|
|
Scenario: Cannot resume terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to resume the terminal plan
|
|
Then a plan error should be raised for automation
|
|
|
|
Scenario: Resume plan triggers auto-progress when ready
|
|
Given I have a paused plan in strategize phase with complete state
|
|
When I resume the plan with level "review_before_apply"
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
# Settings Resolution Tests
|
|
|
|
Scenario: Resolve automation level from settings
|
|
Given the global automation level is "review_before_apply"
|
|
Then the resolved default automation level should be "review_before_apply"
|
|
|
|
Scenario: Invalid automation level in settings falls back to manual
|
|
Given the global automation level is "invalid_value"
|
|
Then the resolved default automation level should be "manual"
|