UAT: `agents plan use --automation-profile` rejects custom namespaced profiles — only built-in profiles accepted #2931

New issue

Open

opened 2026-04-05 02:51:54 +00:00 by freemo · 2 comments

freemo commented

2026-04-05 02:51:54 +00:00

Owner

Background and Context

The spec (docs/specification.md §agents plan use) states that --automation-profile PROFILE accepts "Automation profile name (e.g., trusted, auto, local/careful-auto)". The spec explicitly shows local/careful-auto as a valid custom namespaced profile. However, the implementation in src/cleveragents/cli/commands/plan.py use_action() function validates the profile against BUILTIN_PROFILES only and rejects any non-built-in profile with an error.

Custom namespaced profiles (e.g., local/careful-auto, myorg/strict-review) are a first-class feature of the automation profile system and must be accepted wherever a profile name is accepted.

Current Behavior

Running agents plan use local/my-action my-project --automation-profile local/careful-auto fails with:

Invalid automation profile: local/careful-auto. Available: auto, full-auto, manual, review, supervised, trusted

The validation at src/cleveragents/cli/commands/plan.py, use_action() function, lines 1539–1549 performs if automation_profile not in BUILTIN_PROFILES: which incorrectly rejects all non-built-in profiles.

Expected Behavior

Custom namespaced profiles (e.g., local/careful-auto, myorg/strict-review) should be accepted by --automation-profile. The validation should allow any valid namespaced name (matching the <namespace>/<name> format), not just built-in profiles. Built-in profiles continue to work as before.

Acceptance Criteria

agents plan use <action> <project> --automation-profile local/careful-auto succeeds without error
agents plan use <action> <project> --automation-profile myorg/strict-review succeeds without error
Built-in profiles (trusted, auto, manual, review, supervised, full-auto) continue to work
Invalid profile names (not matching built-in or namespaced name format) are still rejected with a clear error
All existing tests pass; new tests cover namespaced profile acceptance

Supporting Information

Code location: src/cleveragents/cli/commands/plan.py, use_action() function, lines 1539–1549
Root cause: if automation_profile not in BUILTIN_PROFILES: check does not account for valid custom namespaced profiles
Spec reference: docs/specification.md §agents plan use — --automation-profile example shows local/careful-auto
Parent Epic: #397

Metadata

Branch: fix/m6-plan-use-automation-profile-namespaced
Commit Message: fix(cli): allow custom namespaced automation profiles in plan use
Milestone: v3.5.0
Parent Epic: #397

Subtasks

Investigate use_action() validation logic for --automation-profile in src/cleveragents/cli/commands/plan.py
Update validation to accept valid namespaced profile names (e.g., <namespace>/<name>) in addition to built-in profiles
Add or update unit tests covering namespaced profile acceptance and rejection of truly invalid names
Tests (Behave): Add/update scenarios for agents plan use --automation-profile with custom namespaced profiles
Tests (Robot): Add integration test for agents plan use with a namespaced automation profile
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Background and Context The spec (`docs/specification.md` §agents plan use) states that `--automation-profile PROFILE` accepts "Automation profile name (e.g., `trusted`, `auto`, `local/careful-auto`)". The spec explicitly shows `local/careful-auto` as a valid custom namespaced profile. However, the implementation in `src/cleveragents/cli/commands/plan.py` `use_action()` function validates the profile against `BUILTIN_PROFILES` only and rejects any non-built-in profile with an error. Custom namespaced profiles (e.g., `local/careful-auto`, `myorg/strict-review`) are a first-class feature of the automation profile system and must be accepted wherever a profile name is accepted. ## Current Behavior Running `agents plan use local/my-action my-project --automation-profile local/careful-auto` fails with: ``` Invalid automation profile: local/careful-auto. Available: auto, full-auto, manual, review, supervised, trusted ``` The validation at `src/cleveragents/cli/commands/plan.py`, `use_action()` function, lines 1539–1549 performs `if automation_profile not in BUILTIN_PROFILES:` which incorrectly rejects all non-built-in profiles. ## Expected Behavior Custom namespaced profiles (e.g., `local/careful-auto`, `myorg/strict-review`) should be accepted by `--automation-profile`. The validation should allow any valid namespaced name (matching the `<namespace>/<name>` format), not just built-in profiles. Built-in profiles continue to work as before. ## Acceptance Criteria - [ ] `agents plan use <action> <project> --automation-profile local/careful-auto` succeeds without error - [ ] `agents plan use <action> <project> --automation-profile myorg/strict-review` succeeds without error - [ ] Built-in profiles (`trusted`, `auto`, `manual`, `review`, `supervised`, `full-auto`) continue to work - [ ] Invalid profile names (not matching built-in or namespaced name format) are still rejected with a clear error - [ ] All existing tests pass; new tests cover namespaced profile acceptance ## Supporting Information - **Code location**: `src/cleveragents/cli/commands/plan.py`, `use_action()` function, lines 1539–1549 - **Root cause**: `if automation_profile not in BUILTIN_PROFILES:` check does not account for valid custom namespaced profiles - **Spec reference**: `docs/specification.md` §agents plan use — `--automation-profile` example shows `local/careful-auto` - **Parent Epic**: #397 ## Metadata - **Branch**: `fix/m6-plan-use-automation-profile-namespaced` - **Commit Message**: `fix(cli): allow custom namespaced automation profiles in plan use` - **Milestone**: v3.5.0 - **Parent Epic**: #397 ## Subtasks - [ ] Investigate `use_action()` validation logic for `--automation-profile` in `src/cleveragents/cli/commands/plan.py` - [ ] Update validation to accept valid namespaced profile names (e.g., `<namespace>/<name>`) in addition to built-in profiles - [ ] Add or update unit tests covering namespaced profile acceptance and rejection of truly invalid names - [ ] Tests (Behave): Add/update scenarios for `agents plan use --automation-profile` with custom namespaced profiles - [ ] Tests (Robot): Add integration test for `agents plan use` with a namespaced automation profile - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.5.0 milestone

2026-04-05 02:52:58 +00:00

freemo added a new dependency

2026-04-05 02:53:11 +00:00

#397 Epic: Server & Autonomy Infrastructure

freemo commented

2026-04-05 03:01:16 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Confirmed
MoSCoW: Should Have

Valid finding verified during batch triage.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Confirmed - **MoSCoW**: Should Have Valid finding verified during batch triage. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-05 06:15:22 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3094

freemo commented

2026-04-06 22:45:14 +00:00

Author

Owner

This issue has been moved to the backlog as part of an aggressive grooming of the v3.5.0 milestone. It has been deemed non-critical for the minimal viability of the milestone and will be addressed in a future release.