UAT: `plan use --automation-profile` rejects custom profiles — only validates against built-in profiles #4576

New issue

Closed

opened 2026-04-08 15:47:38 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-08 15:47:38 +00:00

Owner

Bug Report

Feature Area: CI/CD Integration / Automation Profiles
Severity: High — blocks Example 7 CI/CD workflow and any custom profile usage
Discovered by: UAT worker uat-worker-cicd-001

What Was Tested

The agents plan use --automation-profile <PROFILE> flag behavior when a custom (non-built-in) profile name is provided.

Expected Behavior (from spec)

Per the specification (§ agents plan use, line 12499):

--automation-profile PROFILE: Automation profile name (e.g., trusted, auto, local/careful-auto). Overrides the profile inherited from the action, project, or global config.

The spec explicitly lists local/careful-auto as a valid example, confirming that custom profiles registered via agents automation-profile add should be usable with --automation-profile.

Actual Behavior

The implementation in src/cleveragents/cli/commands/plan.py (lines 1750–1755) only validates against BUILTIN_PROFILES:

if automation_profile not in BUILTIN_PROFILES:
    console.print(
        f"[red]Invalid automation profile:[/red] {automation_profile}. "
        f"Available: {', '.join(sorted(BUILTIN_PROFILES))}"
    )
    raise typer.Abort()

This means any custom profile (e.g., local/careful-auto, acme/strict) will be rejected with "Invalid automation profile" even if it has been successfully registered via agents automation-profile add.

Steps to Reproduce

Register a custom profile: agents automation-profile add --config ./profiles/careful-auto.yaml
Attempt to use it: agents plan use --automation-profile local/careful-auto local/my-action local/my-project
Result: Command aborts with "Invalid automation profile: local/careful-auto. Available: auto, cautious, ci, full-auto, manual, review, supervised, trusted"

Code Location

src/cleveragents/cli/commands/plan.py, lines 1745–1755

Fix Required

The validation should look up the profile from AutomationProfileService (which includes both built-in and custom profiles), not just BUILTIN_PROFILES. Something like:

if automation_profile:
    service = _get_automation_profile_service()
    try:
        service.get_profile(automation_profile)
    except NotFoundError:
        console.print(f"[red]Invalid automation profile:[/red] {automation_profile}.")
        raise typer.Abort()

Impact

Example 7 (CI/CD Integration) uses --automation-profile ci — this works because ci is built-in
Any team using custom profiles (e.g., local/careful-auto) cannot use them with plan use
The automation-profile add command is effectively useless for plan use workflows

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** CI/CD Integration / Automation Profiles **Severity:** High — blocks Example 7 CI/CD workflow and any custom profile usage **Discovered by:** UAT worker uat-worker-cicd-001 --- ## What Was Tested The `agents plan use --automation-profile <PROFILE>` flag behavior when a custom (non-built-in) profile name is provided. ## Expected Behavior (from spec) Per the specification (§ `agents plan use`, line 12499): > `--automation-profile PROFILE`: Automation profile name (e.g., `trusted`, `auto`, `local/careful-auto`). Overrides the profile inherited from the action, project, or global config. The spec explicitly lists `local/careful-auto` as a valid example, confirming that custom profiles registered via `agents automation-profile add` should be usable with `--automation-profile`. ## Actual Behavior The implementation in `src/cleveragents/cli/commands/plan.py` (lines 1750–1755) only validates against `BUILTIN_PROFILES`: ```python if automation_profile not in BUILTIN_PROFILES: console.print( f"[red]Invalid automation profile:[/red] {automation_profile}. " f"Available: {', '.join(sorted(BUILTIN_PROFILES))}" ) raise typer.Abort() ``` This means any custom profile (e.g., `local/careful-auto`, `acme/strict`) will be rejected with "Invalid automation profile" even if it has been successfully registered via `agents automation-profile add`. ## Steps to Reproduce 1. Register a custom profile: `agents automation-profile add --config ./profiles/careful-auto.yaml` 2. Attempt to use it: `agents plan use --automation-profile local/careful-auto local/my-action local/my-project` 3. **Result:** Command aborts with "Invalid automation profile: local/careful-auto. Available: auto, cautious, ci, full-auto, manual, review, supervised, trusted" ## Code Location `src/cleveragents/cli/commands/plan.py`, lines 1745–1755 ## Fix Required The validation should look up the profile from `AutomationProfileService` (which includes both built-in and custom profiles), not just `BUILTIN_PROFILES`. Something like: ```python if automation_profile: service = _get_automation_profile_service() try: service.get_profile(automation_profile) except NotFoundError: console.print(f"[red]Invalid automation profile:[/red] {automation_profile}.") raise typer.Abort() ``` ## Impact - Example 7 (CI/CD Integration) uses `--automation-profile ci` — this works because `ci` is built-in - Any team using custom profiles (e.g., `local/careful-auto`) cannot use them with `plan use` - The `automation-profile add` command is effectively useless for `plan use` workflows --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 commented

2026-04-08 15:51:32 +00:00

Author

Owner

Closing as duplicate of #4008. Both issues describe the same bug: agents plan use --automation-profile rejects custom (user-defined) profiles and only accepts built-in profiles. Issue #4008 is older and more detailed with a specific fix plan.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Closing as duplicate of #4008. Both issues describe the same bug: `agents plan use --automation-profile` rejects custom (user-defined) profiles and only accepts built-in profiles. Issue #4008 is older and more detailed with a specific fix plan. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 closed this issue

2026-04-08 15:52:10 +00:00