UAT: m6_acceptance.robot missing action > global automation profile precedence test — documented gap in test file #6026

New issue

Open

opened 2026-04-09 13:48:54 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 13:48:54 +00:00

Owner

Bug Report

Feature Area: E2E Workflow Specification Tests — M6 Acceptance (Automation Profile Precedence)
Milestone: v3.6.0 (M7) — E2E workflow specification tests
Severity: Priority/Backlog — test quality gap, not blocking runtime

What Was Tested

Code-level analysis of robot/e2e/m6_acceptance.robot against the M6 milestone acceptance criteria for automation profile resolution precedence.

Expected Behavior (from spec)

The M6 (v3.5.0) milestone acceptance criteria include:

Automation profile resolution precedence correct (plan > action > global)

The spec defines a four-level precedence chain: plan > action > project > global

The test M6 E2E Profile Precedence Plan Overrides Global covers plan > global precedence, but the action > global precedence is explicitly documented as missing.

Actual Behavior

The test file documents the gap explicitly:

M6 E2E Profile Precedence Plan Overrides Global
  [Documentation]    ...
  ...    Note: action > global precedence requires the action's
  ...    automation_profile to propagate to the Plan during plan-use,
  ...    which is not yet wired in PlanLifecycleService.use_action;
  ...    this test covers plan > global.

The test only covers plan > global precedence. The following precedence levels are untested:

action > global: When an action has automation_profile: trusted and global is manual, the plan should use trusted. This is not tested because PlanLifecycleService.use_action does not propagate the action's automation_profile to the Plan.
project > global: When a project has a configured automation profile and global is different, the project profile should win.

Code Location

File: robot/e2e/m6_acceptance.robot
Lines 279-314: M6 E2E Profile Precedence Plan Overrides Global — only tests plan > global
Lines 287-290: Explicit documentation of the action > global gap

Impact

The M6 acceptance criteria state "automation profile resolution precedence correct (plan > action > global)". The current test only verifies plan > global. The action > global and project > global precedence levels are untested.

This means a regression in action-level or project-level precedence resolution would not be caught by the E2E test suite.

Root Cause

PlanLifecycleService.use_action does not propagate the action's automation_profile field to the created Plan. This is a separate implementation bug, but the E2E test should at minimum document the gap with a tdd_expected_fail test that will pass once the implementation is fixed.

Definition of Done

Add M6 E2E Profile Precedence Action Overrides Global test:
- Create action with automation_profile: trusted
- Set global profile to manual
- Run plan use without --automation-profile flag
- Assert plan output shows trusted (not manual)
- Tag tdd_expected_fail until PlanLifecycleService.use_action is fixed to propagate action's profile
Add M6 E2E Profile Precedence Project Overrides Global test:
- Configure project with automation_profile: supervised
- Set global profile to manual
- Run plan use without --automation-profile flag
- Assert plan output shows supervised (not manual)

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: E2E Workflow Specification Tests — M6 Acceptance (Automation Profile Precedence) **Milestone**: v3.6.0 (M7) — E2E workflow specification tests **Severity**: Priority/Backlog — test quality gap, not blocking runtime ### What Was Tested Code-level analysis of `robot/e2e/m6_acceptance.robot` against the M6 milestone acceptance criteria for automation profile resolution precedence. ### Expected Behavior (from spec) The M6 (v3.5.0) milestone acceptance criteria include: > Automation profile resolution precedence correct (plan > action > global) The spec defines a four-level precedence chain: **plan > action > project > global** The test `M6 E2E Profile Precedence Plan Overrides Global` covers plan > global precedence, but the action > global precedence is explicitly documented as missing. ### Actual Behavior The test file documents the gap explicitly: ```robot M6 E2E Profile Precedence Plan Overrides Global [Documentation] ... ... Note: action > global precedence requires the action's ... automation_profile to propagate to the Plan during plan-use, ... which is not yet wired in PlanLifecycleService.use_action; ... this test covers plan > global. ``` The test only covers plan > global precedence. The following precedence levels are untested: 1. **action > global**: When an action has `automation_profile: trusted` and global is `manual`, the plan should use `trusted`. This is not tested because `PlanLifecycleService.use_action` does not propagate the action's automation_profile to the Plan. 2. **project > global**: When a project has a configured automation profile and global is different, the project profile should win. ### Code Location - File: `robot/e2e/m6_acceptance.robot` - Lines 279-314: `M6 E2E Profile Precedence Plan Overrides Global` — only tests plan > global - Lines 287-290: Explicit documentation of the action > global gap ### Impact The M6 acceptance criteria state "automation profile resolution precedence correct (plan > action > global)". The current test only verifies plan > global. The action > global and project > global precedence levels are untested. This means a regression in action-level or project-level precedence resolution would not be caught by the E2E test suite. ### Root Cause `PlanLifecycleService.use_action` does not propagate the action's `automation_profile` field to the created Plan. This is a separate implementation bug, but the E2E test should at minimum document the gap with a `tdd_expected_fail` test that will pass once the implementation is fixed. ### Definition of Done 1. Add `M6 E2E Profile Precedence Action Overrides Global` test: - Create action with `automation_profile: trusted` - Set global profile to `manual` - Run `plan use` without `--automation-profile` flag - Assert plan output shows `trusted` (not `manual`) - Tag `tdd_expected_fail` until `PlanLifecycleService.use_action` is fixed to propagate action's profile 2. Add `M6 E2E Profile Precedence Project Overrides Global` test: - Configure project with `automation_profile: supervised` - Set global profile to `manual` - Run `plan use` without `--automation-profile` flag - Assert plan output shows `supervised` (not `manual`) --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

Rows
Columns