Luis Mendes
01efa2922c
Updated automation_levels.feature to expect "applied" processing state instead of "queued" after completing execute on a full_automation plan, matching the new auto_progress() behavior that drives Apply to terminal state. Updated plan_cli_coverage_boost_steps.py to properly mock the full apply lifecycle (get_plan → apply_plan → start_apply → complete_apply) instead of returning a single plan from apply_plan, matching the updated lifecycle_apply_plan command handler that now drives plans through to the terminal applied state. Refs: #753
145 lines
6.8 KiB
Gherkin
145 lines
6.8 KiB
Gherkin
Feature: Automation Levels
|
|
As a developer using CleverAgents
|
|
I want to control how automated phase transitions are
|
|
So that I can choose between manual control, review-before-apply, and full automation
|
|
|
|
Background:
|
|
Given I have a plan lifecycle service with automation level support
|
|
|
|
# Manual Mode Tests
|
|
|
|
Scenario: Manual mode requires explicit execute command
|
|
Given I have a plan with automation level "manual" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "strategize"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
Scenario: Manual mode requires explicit apply command
|
|
Given I have a plan with automation level "manual" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
# Review-Before-Apply Tests
|
|
|
|
Scenario: Review-before-apply auto-executes after strategize completes
|
|
Given I have a plan with automation level "review_before_apply" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
Scenario: Review-before-apply pauses at apply
|
|
Given I have a plan with automation level "review_before_apply" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "complete"
|
|
|
|
# Full Automation Tests
|
|
|
|
Scenario: Full automation auto-executes after strategize completes
|
|
Given I have a plan with automation level "full_automation" in strategize phase with processing state
|
|
When I complete strategize on the automated plan
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
Scenario: Full automation auto-applies after execute completes
|
|
Given I have a plan with automation level "full_automation" in execute phase with processing state
|
|
When I complete execute on the automated plan
|
|
Then the automated plan phase should be "apply"
|
|
And the automated plan processing state should be "applied"
|
|
|
|
# Plan-Level Override Tests
|
|
|
|
Scenario: Plan-level automation overrides global setting
|
|
Given the global automation level is "manual"
|
|
And I have a plan created with explicit automation level "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Use action with explicit automation level
|
|
Given I have an available action for automation tests
|
|
When I use the action with automation level "review_before_apply" on project "project-auto-1"
|
|
Then the plan automation level should be "review_before_apply"
|
|
|
|
Scenario: Use action without explicit automation level uses global default
|
|
Given the global automation level is "manual"
|
|
And I have an available action for automation tests
|
|
When I use the action without explicit automation level on project "project-auto-2"
|
|
Then the plan automation level should be "manual"
|
|
|
|
# Change Automation Level Mid-Plan
|
|
|
|
Scenario: Change automation level mid-plan works correctly
|
|
Given I have a plan with automation level "manual" in strategize phase with queued state
|
|
When I set the plan automation level to "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Cannot change automation level on terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to set the plan automation level to "full_automation"
|
|
Then a plan error should be raised for automation
|
|
|
|
# should_auto_progress Query Tests
|
|
|
|
Scenario: should_auto_progress returns false for manual plan in strategize complete
|
|
Given I have a plan with automation level "manual" in strategize phase with complete state for query
|
|
Then should_auto_progress should return false
|
|
|
|
Scenario: should_auto_progress returns true for review-before-apply plan in strategize complete
|
|
Given I have a plan with automation level "review_before_apply" in strategize phase with complete state for query
|
|
Then should_auto_progress should return true
|
|
|
|
Scenario: should_auto_progress returns false for review-before-apply plan in execute complete
|
|
Given I have a plan with automation level "review_before_apply" in execute phase with complete state for query
|
|
Then should_auto_progress should return false
|
|
|
|
Scenario: should_auto_progress returns true for full-automation plan in execute complete
|
|
Given I have a plan with automation level "full_automation" in execute phase with complete state for query
|
|
Then should_auto_progress should return true
|
|
|
|
Scenario: should_auto_progress returns false for terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
Then should_auto_progress on terminal plan should return false
|
|
|
|
# Pause and Resume Tests
|
|
|
|
Scenario: Pause plan sets automation to manual
|
|
Given I have a plan with automation level "full_automation" in strategize phase with queued state
|
|
When I pause the plan
|
|
Then the plan automation level should be "manual"
|
|
|
|
Scenario: Cannot pause terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to pause the terminal plan
|
|
Then a plan error should be raised for automation
|
|
|
|
Scenario: Resume plan restores automation level
|
|
Given I have a paused plan that was previously "full_automation" in strategize phase
|
|
When I resume the plan with level "full_automation"
|
|
Then the plan automation level should be "full_automation"
|
|
|
|
Scenario: Resume plan without explicit level defaults to review-before-apply
|
|
Given I have a paused plan that was previously "full_automation" in strategize phase
|
|
When I resume the plan without explicit level
|
|
Then the plan automation level should be "review_before_apply"
|
|
|
|
Scenario: Cannot resume terminal plan
|
|
Given I have a terminal plan for automation tests
|
|
When I try to resume the terminal plan
|
|
Then a plan error should be raised for automation
|
|
|
|
Scenario: Resume plan triggers auto-progress when ready
|
|
Given I have a paused plan in strategize phase with complete state
|
|
When I resume the plan with level "review_before_apply"
|
|
Then the automated plan phase should be "execute"
|
|
And the automated plan processing state should be "queued"
|
|
|
|
# Settings Resolution Tests
|
|
|
|
Scenario: Resolve automation level from settings
|
|
Given the global automation level is "review_before_apply"
|
|
Then the resolved default automation level should be "review_before_apply"
|
|
|
|
Scenario: Invalid automation level in settings falls back to manual
|
|
Given the global automation level is "invalid_value"
|
|
Then the resolved default automation level should be "manual"
|