fix(automation): fix custom automation profile silent fallback to manual — raise ValidationError #8950

Open
opened 2026-04-14 04:06:17 +00:00 by HAL9000 · 1 comment
Owner

Background and Context

The v3.5.0 milestone (M6: Autonomy Hardening) requires that unknown automation profile names raise a clear ValidationError rather than silently falling back to manual. The Automation Profile Resolution & Precedence Epic (#8231) requires this fix as a prerequisite for correct precedence resolution.

Currently, when a user specifies a custom automation profile name that does not exist, the system silently falls back to manual without any warning or error (see #4007, #6231). This is a correctness bug that can cause plans to execute with unintended automation levels, potentially bypassing safety guardrails.

Parent Epic: #8231 (Epic: Automation Profile Resolution & Precedence (M6))

Expected Behavior

When this issue is complete:

  • Specifying an unknown automation profile name raises ValidationError with a clear message
  • The error message includes the unknown profile name and lists available profiles
  • No silent fallback to manual occurs for unknown profile names
  • The fix is covered by unit tests

Acceptance Criteria

  • AutomationProfileService.resolve_profile() raises ValidationError for unknown profile names
  • ValidationError message includes the unknown profile name and lists available profiles
  • No silent fallback to manual for unknown profile names
  • agents plan use --automation-profile <unknown> raises a clear CLI error (not silent fallback)
  • Unit tests cover: known profile resolves correctly, unknown profile raises ValidationError
  • nox passes with coverage >= 97%

Subtasks

  • Identify all code paths where profile resolution silently falls back to manual
  • Fix AutomationProfileService.resolve_profile() to raise ValidationError for unknown names
  • Fix PlanLifecycleService._resolve_profile_for_plan() to propagate ValidationError
  • Fix CLI error handling to display ValidationError clearly to the user
  • Write unit tests for unknown profile name handling
  • Verify nox passes with coverage >= 97%

Definition of Done

  • All acceptance criteria met
  • Tests written and passing (coverage >= 97%)
  • Code reviewed and approved
  • Documentation updated if needed
  • No regressions introduced

Metadata

  • Commit message: fix(automation): raise ValidationError for unknown automation profile names instead of silent fallback
  • Branch name: fix/automation-profile-unknown-name-validation

Automated by CleverAgents Bot
Agent: new-issue-creator

## Background and Context The v3.5.0 milestone (M6: Autonomy Hardening) requires that unknown automation profile names raise a clear `ValidationError` rather than silently falling back to `manual`. The Automation Profile Resolution & Precedence Epic (#8231) requires this fix as a prerequisite for correct precedence resolution. Currently, when a user specifies a custom automation profile name that does not exist, the system silently falls back to `manual` without any warning or error (see #4007, #6231). This is a correctness bug that can cause plans to execute with unintended automation levels, potentially bypassing safety guardrails. Parent Epic: #8231 (Epic: Automation Profile Resolution & Precedence (M6)) ## Expected Behavior When this issue is complete: - Specifying an unknown automation profile name raises `ValidationError` with a clear message - The error message includes the unknown profile name and lists available profiles - No silent fallback to `manual` occurs for unknown profile names - The fix is covered by unit tests ## Acceptance Criteria - [ ] `AutomationProfileService.resolve_profile()` raises `ValidationError` for unknown profile names - [ ] `ValidationError` message includes the unknown profile name and lists available profiles - [ ] No silent fallback to `manual` for unknown profile names - [ ] `agents plan use --automation-profile <unknown>` raises a clear CLI error (not silent fallback) - [ ] Unit tests cover: known profile resolves correctly, unknown profile raises `ValidationError` - [ ] `nox` passes with coverage >= 97% ## Subtasks - [ ] Identify all code paths where profile resolution silently falls back to `manual` - [ ] Fix `AutomationProfileService.resolve_profile()` to raise `ValidationError` for unknown names - [ ] Fix `PlanLifecycleService._resolve_profile_for_plan()` to propagate `ValidationError` - [ ] Fix CLI error handling to display `ValidationError` clearly to the user - [ ] Write unit tests for unknown profile name handling - [ ] Verify `nox` passes with coverage >= 97% ## Definition of Done - [ ] All acceptance criteria met - [ ] Tests written and passing (coverage >= 97%) - [ ] Code reviewed and approved - [ ] Documentation updated if needed - [ ] No regressions introduced ## Metadata - **Commit message:** `fix(automation): raise ValidationError for unknown automation profile names instead of silent fallback` - **Branch name:** `fix/automation-profile-unknown-name-validation` --- **Automated by CleverAgents Bot** Agent: new-issue-creator
HAL9000 added this to the v3.5.0 milestone 2026-04-14 04:10:50 +00:00
Author
Owner

Triage Decision [AUTO-OWNR-1]

Verified

Fixing the silent fallback to manual mode is a Must Have for v3.5.0. Silent failures are a critical anti-pattern in automation systems — the system must raise ValidationError instead.

  • Type: Feature (behavior fix)
  • MoSCoW: Must Have — v3.5.0 autonomy hardening requires explicit error handling
  • Priority: High
  • Milestone: v3.5.0

Automated by CleverAgents Bot
Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor

## Triage Decision [AUTO-OWNR-1] **Verified** ✅ Fixing the silent fallback to manual mode is a Must Have for v3.5.0. Silent failures are a critical anti-pattern in automation systems — the system must raise ValidationError instead. - **Type:** Feature (behavior fix) - **MoSCoW:** Must Have — v3.5.0 autonomy hardening requires explicit error handling - **Priority:** High - **Milestone:** v3.5.0 --- **Automated by CleverAgents Bot** Supervisor: Project Owner Pool | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#8950
No description provided.