fix(automation): respect automation profile gates in lifecycle service and async jobs
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 1m5s
CI / benchmark-regression (pull_request) Failing after 1m8s
CI / typecheck (pull_request) Successful in 1m16s
CI / security (pull_request) Successful in 1m26s
CI / helm (pull_request) Successful in 32s
CI / push-validation (pull_request) Successful in 35s
CI / build (pull_request) Successful in 50s
CI / unit_tests (pull_request) Successful in 4m40s
CI / integration_tests (pull_request) Successful in 3m18s
CI / docker (pull_request) Successful in 1m29s
CI / e2e_tests (pull_request) Successful in 3m12s
CI / coverage (pull_request) Successful in 10m47s
CI / status-check (pull_request) Successful in 4s

Add BDD regression tests for issue #4328 automation profile gates

Tests verify that complete_strategize() and complete_execute() respect
automation profile thresholds and do NOT unconditionally call auto_progress().

Covers 8 built-in profiles: manual, full-auto, supervised, auto,
review_before_apply, ci, trusted

Update CHANGELOG.md with issue #4328 fix entry

ISSUES CLOSED: #4328
This commit is contained in:
2026-05-11 13:38:44 +00:00
parent b692894c88
commit 82205d596c
3 changed files with 127 additions and 4 deletions
+9
View File
@@ -36,6 +36,15 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
removal. Mocked existing steps to allow remaining V2 features to be
covered/tested.
- **Manual automation profile respected during phase completion** (#4328): Fixed
``complete_strategize()`` and ``complete_execute()`` in ``PlanLifecycleService`` to
gate ``auto_progress()`` calls with ``should_auto_progress()`` checks. Previously
these methods called ``auto_progress()`` unconditionally, causing plans with
``--automation-profile "manual"`` (all thresholds=1.0) to auto-progress through
phases without human approval. Added BDD regression tests in
``features/tdd_automation_profile_gates_4328.feature`` covering all 8 built-in
profiles.
- **TUI Prompt Symbol Mode Awareness** (#6431): The prompt widget now displays a
mode-dependent symbol (`` normal, `/` command, `$` shell, `` multi-line),
implemented via `_PromptSymbolMixin` and `InputMode.MULTILINE`. The widget uses
@@ -0,0 +1,112 @@
@tdd_issue @tdd_issue_4328
Feature: Automation Profile Gates - Issue #4328 Regression Tests
Regression tests for manual automation profile being disrespected during phase
completion. Verifies that complete_strategize() and complete_execute() respect
automation profile gates and do NOT auto-progress when thresholds are 1.0.
Background:
Given I have a plan lifecycle service with automation level support
# ========================================================================
# Issue #4328: Manual profile (all thresholds = 1.0) should NOT auto-progress
# These scenarios verify the fix - that complete_strategize/execute do NOT
# unconditionally call auto_progress()
# ========================================================================
Scenario: Manual profile create_tool=1.0 blocks strategize to execute transition
Given I have a plan with automation level "manual" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "strategize"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
Scenario: Manual profile select_tool=1.0 blocks execute to apply transition
Given I have a plan with automation level "manual" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
# ========================================================================
# Verify auto_progress() IS called when should_auto_progress() returns true
# ========================================================================
Scenario: Full-auto profile create_tool=0.0 allows strategize to execute auto-progress
Given I have a plan with automation level "full_automation" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "queued"
Scenario: Full-auto profile select_tool=0.0 allows execute to apply auto-progress
Given I have a plan with automation level "full_automation" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "apply"
And the automated plan processing state should be "applied"
# ========================================================================
# Supervised profile: create_tool=1.0 (blocks execute), select_tool=1.0 (blocks apply)
# ========================================================================
Scenario: Supervised profile create_tool=1.0 blocks strategize to execute auto-progress
Given I have a plan with automation level "supervised" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "strategize"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
Scenario: Supervised profile select_tool=1.0 blocks execute to apply transition
Given I have a plan with automation level "supervised" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
# ========================================================================
# Review-before-apply (auto) profile: create_tool=0.0, select_tool=1.0
# ========================================================================
Scenario: Auto profile create_tool=0.0 allows strategize to execute auto-progress
Given I have a plan with automation level "review_before_apply" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "queued"
Scenario: Auto profile select_tool=1.0 blocks execute to apply transition
Given I have a plan with automation level "review_before_apply" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
# ========================================================================
# CI profile: all thresholds=0.0 (auto-progresses everything)
# ========================================================================
Scenario: CI profile blocks neither strategize nor execute transitions
Given I have a plan with automation level "ci" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "queued"
Scenario: CI profile execute phase auto-progresses to apply
Given I have a plan with automation level "ci" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "apply"
And the automated plan processing state should be "applied"
# ========================================================================
# Trusted profile: create_tool=0.0, select_tool=1.0 (same as auto for strategize/execute)
# ========================================================================
Scenario: Trusted profile allows strategize to execute auto-progress
Given I have a plan with automation level "trusted" in strategize phase with processing state
When I complete strategize on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "queued"
Scenario: Trusted profile select_tool=1.0 blocks execute to apply transition
Given I have a plan with automation level "trusted" in execute phase with processing state
When I complete execute on the automated plan
Then the automated plan phase should be "execute"
And the automated plan processing state should be "complete"
And should_auto_progress should return false
@@ -1533,8 +1533,9 @@ class PlanLifecycleService:
exc_info=True,
)
# Auto-progress if automation level permits
return self.auto_progress(plan_id)
if self.should_auto_progress(plan):
return self.auto_progress(plan_id)
return plan
def fail_strategize(self, plan_id: str, error_message: str) -> Plan:
"""Mark Strategize phase as failed.
@@ -1751,8 +1752,9 @@ class PlanLifecycleService:
exc_info=True,
)
# Auto-progress if automation level permits
return self.auto_progress(plan_id)
if self.should_auto_progress(plan):
return self.auto_progress(plan_id)
return plan
def fail_execute(self, plan_id: str, error_message: str) -> Plan:
"""Mark Execute phase as failed."""