UAT: `PlanLifecycleService._resolve_profile_for_plan()` silently falls back to 'manual' for custom profiles — custom profiles are ignored during execution #4007

New issue

Open

opened 2026-04-06 08:33:14 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 08:33:14 +00:00

Owner

Metadata

Branch: fix/plan-lifecycle-custom-profile-resolution
Commit Message: fix(plan-lifecycle): look up custom automation profiles from repository in _resolve_profile_for_plan
Milestone: (none — see backlog note below)
Parent Epic: #3370

Bug Report

What Was Tested

The PlanLifecycleService._resolve_profile_for_plan() method was analyzed to understand how automation profiles are resolved during plan execution (used by should_auto_progress() and other execution-time decisions).

Expected Behavior (from spec)

Per docs/specification.md lines 28456–28465:

The effective automation profile for a plan is resolved at plan use time and locked to the plan.
During execution, the plan's stored automation_profile reference (profile name + provenance) should be used to look up the full profile including custom profiles.
Custom profiles registered via agents automation-profile add should be retrievable by name during plan execution.

Actual Behavior

Service layer (src/cleveragents/application/services/plan_lifecycle_service.py, lines 2131–2135):

def _resolve_profile_for_plan(self, plan: Plan) -> AutomationProfile:
    if plan.automation_profile is not None:
        profile_name = plan.automation_profile.profile_name
    else:
        profile_name = "manual"
    return BUILTIN_PROFILES.get(profile_name, BUILTIN_PROFILES["manual"])

The method uses BUILTIN_PROFILES.get(profile_name, BUILTIN_PROFILES["manual"]). If a plan was created with a custom profile (e.g., acme/strict), this method silently falls back to the "manual" built-in profile instead of looking up the custom profile from the database. This means:

Plans with custom profiles execute with "manual" behavior (all thresholds at 1.0 — maximum human approval required)
No error is raised — the silent fallback masks the problem
The should_auto_progress() method will always return False for plans with custom profiles (since "manual" has all thresholds at 1.0)

Steps to Reproduce

Register a custom profile with decompose_task=0.0, select_tool=0.0 (fully automatic)
Create a plan using that custom profile
Observe that should_auto_progress() returns False (plan does not auto-advance) because the profile resolved to "manual" instead of the custom profile

Code Location

src/cleveragents/application/services/plan_lifecycle_service.py lines 2120–2135

Fix Required

The _resolve_profile_for_plan() method should use AutomationProfileService.get_profile(profile_name) (which checks built-ins first, then the repository) instead of BUILTIN_PROFILES.get(). The service already has the correct lookup logic.

Subtasks

Update _resolve_profile_for_plan() to use AutomationProfileService.get_profile() or AutomationProfileRepository.get_by_name()
Add BDD scenario: plan with custom profile auto-progresses correctly based on custom thresholds
Add BDD scenario: plan with custom profile uses correct safety constraints during execution

Definition of Done

Plans created with custom automation profiles use the correct custom thresholds during execution
should_auto_progress() returns the correct value for plans with custom profiles
All new BDD scenarios pass
All nox stages pass
Coverage >= 97%
PR merged

Backlog note: This issue was discovered during autonomous operation
on milestone . It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/plan-lifecycle-custom-profile-resolution` - **Commit Message**: `fix(plan-lifecycle): look up custom automation profiles from repository in _resolve_profile_for_plan` - **Milestone**: *(none — see backlog note below)* - **Parent Epic**: #3370 ## Bug Report ### What Was Tested The `PlanLifecycleService._resolve_profile_for_plan()` method was analyzed to understand how automation profiles are resolved during plan execution (used by `should_auto_progress()` and other execution-time decisions). ### Expected Behavior (from spec) Per `docs/specification.md` lines 28456–28465: - The effective automation profile for a plan is resolved at `plan use` time and locked to the plan. - During execution, the plan's stored `automation_profile` reference (profile name + provenance) should be used to look up the full profile including custom profiles. - Custom profiles registered via `agents automation-profile add` should be retrievable by name during plan execution. ### Actual Behavior **Service layer (`src/cleveragents/application/services/plan_lifecycle_service.py`, lines 2131–2135):** ```python def _resolve_profile_for_plan(self, plan: Plan) -> AutomationProfile: if plan.automation_profile is not None: profile_name = plan.automation_profile.profile_name else: profile_name = "manual" return BUILTIN_PROFILES.get(profile_name, BUILTIN_PROFILES["manual"]) ``` The method uses `BUILTIN_PROFILES.get(profile_name, BUILTIN_PROFILES["manual"])`. If a plan was created with a custom profile (e.g., `acme/strict`), this method silently falls back to the `"manual"` built-in profile instead of looking up the custom profile from the database. This means: 1. Plans with custom profiles execute with `"manual"` behavior (all thresholds at 1.0 — maximum human approval required) 2. No error is raised — the silent fallback masks the problem 3. The `should_auto_progress()` method will always return `False` for plans with custom profiles (since `"manual"` has all thresholds at 1.0) ### Steps to Reproduce 1. Register a custom profile with `decompose_task=0.0, select_tool=0.0` (fully automatic) 2. Create a plan using that custom profile 3. Observe that `should_auto_progress()` returns `False` (plan does not auto-advance) because the profile resolved to `"manual"` instead of the custom profile ### Code Location - `src/cleveragents/application/services/plan_lifecycle_service.py` lines 2120–2135 ### Fix Required The `_resolve_profile_for_plan()` method should use `AutomationProfileService.get_profile(profile_name)` (which checks built-ins first, then the repository) instead of `BUILTIN_PROFILES.get()`. The service already has the correct lookup logic. ## Subtasks - [ ] Update `_resolve_profile_for_plan()` to use `AutomationProfileService.get_profile()` or `AutomationProfileRepository.get_by_name()` - [ ] Add BDD scenario: plan with custom profile auto-progresses correctly based on custom thresholds - [ ] Add BDD scenario: plan with custom profile uses correct safety constraints during execution ## Definition of Done - [ ] Plans created with custom automation profiles use the correct custom thresholds during execution - [ ] `should_auto_progress()` returns the correct value for plans with custom profiles - [ ] All new BDD scenarios pass - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] PR merged > **Backlog note:** This issue was discovered during autonomous operation > on milestone <current active milestone>. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 08:33:28 +00:00

#3370 Epic: Automation Profile & Safety Profile CLI — Cost Budgets, Guard Enforcement, and Profile Management

freemo added the

Points

label

2026-04-06 08:37:12 +00:00

freemo referenced this issue

2026-04-06 08:51:05 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 09:07:06 +00:00