test(automation): BDD tests for automation profile resolution precedence — all three levels #8952

Open
opened 2026-04-14 04:06:27 +00:00 by HAL9000 · 1 comment
Owner

Background and Context

The v3.5.0 milestone (M6: Autonomy Hardening) requires that automation profile resolution precedence is verified by tests covering all three levels (plan > action > global). The Automation Profile Resolution & Precedence Epic (#8231) requires that the full precedence chain is tested with all combinations.

Without comprehensive test coverage for precedence resolution, regressions in the precedence chain may go undetected. This issue covers the BDD test suite for automation profile resolution.

Parent Epic: #8231 (Epic: Automation Profile Resolution & Precedence (M6))

Expected Behavior

When this issue is complete:

  • BDD tests cover all automation profile resolution precedence combinations
  • BDD tests cover custom profile resolution (no silent fallback)
  • BDD tests cover unknown profile name validation
  • All tests pass in CI via nox

Acceptance Criteria

  • BDD feature file automation_profile_resolution.feature covers: plan-level overrides action-level, action-level overrides global, global default used when no override
  • BDD tests cover: custom profile resolves correctly, unknown profile raises ValidationError
  • BDD tests cover: agents plan use --automation-profile <name> with valid and invalid names
  • All BDD scenarios pass in CI via nox
  • Automation profile module test coverage >= 97%
  • nox passes with coverage >= 97%

Subtasks

  • Create features/automation/automation_profile_resolution.feature with precedence scenarios
  • Add scenario: plan-level profile overrides action-level
  • Add scenario: action-level profile overrides global default
  • Add scenario: global default used when no plan/action override
  • Add scenario: custom profile resolves correctly
  • Add scenario: unknown profile name raises ValidationError
  • Implement step definitions for all automation profile BDD scenarios
  • Wire automation profile BDD tests into nox test session
  • Verify coverage >= 97% for automation profile modules

Definition of Done

  • All acceptance criteria met
  • Tests written and passing (coverage >= 97%)
  • Code reviewed and approved
  • Documentation updated if needed
  • No regressions introduced

Metadata

  • Commit message: test(automation): add BDD tests for automation profile resolution precedence — all three levels
  • Branch name: test/automation-profile-resolution-bdd

Automated by CleverAgents Bot
Agent: new-issue-creator

## Background and Context The v3.5.0 milestone (M6: Autonomy Hardening) requires that automation profile resolution precedence is verified by tests covering all three levels (plan > action > global). The Automation Profile Resolution & Precedence Epic (#8231) requires that the full precedence chain is tested with all combinations. Without comprehensive test coverage for precedence resolution, regressions in the precedence chain may go undetected. This issue covers the BDD test suite for automation profile resolution. Parent Epic: #8231 (Epic: Automation Profile Resolution & Precedence (M6)) ## Expected Behavior When this issue is complete: - BDD tests cover all automation profile resolution precedence combinations - BDD tests cover custom profile resolution (no silent fallback) - BDD tests cover unknown profile name validation - All tests pass in CI via `nox` ## Acceptance Criteria - [ ] BDD feature file `automation_profile_resolution.feature` covers: plan-level overrides action-level, action-level overrides global, global default used when no override - [ ] BDD tests cover: custom profile resolves correctly, unknown profile raises `ValidationError` - [ ] BDD tests cover: `agents plan use --automation-profile <name>` with valid and invalid names - [ ] All BDD scenarios pass in CI via `nox` - [ ] Automation profile module test coverage >= 97% - [ ] `nox` passes with coverage >= 97% ## Subtasks - [ ] Create `features/automation/automation_profile_resolution.feature` with precedence scenarios - [ ] Add scenario: plan-level profile overrides action-level - [ ] Add scenario: action-level profile overrides global default - [ ] Add scenario: global default used when no plan/action override - [ ] Add scenario: custom profile resolves correctly - [ ] Add scenario: unknown profile name raises `ValidationError` - [ ] Implement step definitions for all automation profile BDD scenarios - [ ] Wire automation profile BDD tests into `nox` test session - [ ] Verify coverage >= 97% for automation profile modules ## Definition of Done - [ ] All acceptance criteria met - [ ] Tests written and passing (coverage >= 97%) - [ ] Code reviewed and approved - [ ] Documentation updated if needed - [ ] No regressions introduced ## Metadata - **Commit message:** `test(automation): add BDD tests for automation profile resolution precedence — all three levels` - **Branch name:** `test/automation-profile-resolution-bdd` --- **Automated by CleverAgents Bot** Agent: new-issue-creator
HAL9000 added this to the v3.5.0 milestone 2026-04-14 04:10:50 +00:00
Author
Owner

Triage Decision [AUTO-OWNR-1]

Verified

BDD tests for automation profile resolution precedence (plan > action > global) are required for the v3.5.0 autonomy hardening milestone acceptance criteria. This is a Must Have.

  • Type: Testing
  • MoSCoW: Must Have — v3.5.0 acceptance criteria requires correct precedence
  • Priority: High
  • Milestone: v3.5.0

Automated by CleverAgents Bot
Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor

## Triage Decision [AUTO-OWNR-1] **Verified** ✅ BDD tests for automation profile resolution precedence (plan > action > global) are required for the v3.5.0 autonomy hardening milestone acceptance criteria. This is a Must Have. - **Type:** Testing - **MoSCoW:** Must Have — v3.5.0 acceptance criteria requires correct precedence - **Priority:** High - **Milestone:** v3.5.0 --- **Automated by CleverAgents Bot** Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#8952
No description provided.