cleveragents/cleveragents-core

Fork 3

UAT: All 8 built-in automation profile descriptions don't match spec-required values #2091

New issue

Open

opened 2026-04-03 03:59:21 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 03:59:21 +00:00

Owner

Background and Context

During UAT testing, all 8 built-in automation profiles were found to have descriptions that do not match the values defined in docs/specification.md. The specification is the source of truth per project convention, so the implementation must be updated to match it. This affects both the automation-profile list and automation-profile show CLI commands, meaning users cannot rely on the displayed descriptions to understand profile behaviour as documented.

Current Behavior

In src/cleveragents/domain/models/core/automation_profile.py, the BUILTIN_PROFILES dict uses these descriptions:

Profile	Actual Description
`manual`	`"Human approves every action"`
`review`	`"Human reviews before apply"`
`supervised`	`"Human reviews strategy and execution"`
`cautious`	`"Probabilistic gates on most actions"`
`trusted`	`"Auto for most, human for apply and revert"`
`auto`	`"Fully automatic except apply"`
`ci`	`"Designed for CI pipelines"`
`full-auto`	`"No gates, no sandbox, no checkpoints"`

None of these match the spec-required descriptions.

Steps to Reproduce:

agents automation-profile list --format json
# Expected: "description": "Human-driven (default)" for manual
# Actual:   "description": "Human approves every action" for manual

Expected Behavior

Per docs/specification.md lines 17003–17011 (the automation-profile list JSON output) and line 17182 (the automation-profile show output for trusted), the descriptions should be:

Profile	Spec Description (list / show)
`manual`	`"Human-driven (default)"`
`review`	`"Auto-phase, manual decisions"`
`supervised`	`"Auto-plan, manual exec"`
`cautious`	`"Confidence-gated automation"`
`trusted`	`"Auto-exec, manual apply. Day-to-day development"` (full form from `show`)
`auto`	`"Full auto except apply"`
`ci`	`"Full auto, safe CI/CD"`
`full-auto`	`"Complete automation"`

Note: The spec suggests list may show abbreviated descriptions while show uses the full description (e.g., trusted). This distinction should be clarified and implemented correctly during the fix.

Impact

automation-profile list shows wrong descriptions for all 8 built-in profiles.
automation-profile show shows wrong descriptions for all 8 built-in profiles.
Users cannot rely on descriptions to understand profile behaviour as documented.

Metadata

Branch: fix/automation-profile-builtin-descriptions
Commit Message: fix(automation-profile): correct descriptions for all 8 built-in profiles to match spec
Milestone: v3.7.0
Parent Epic: #362

Subtasks

Audit BUILTIN_PROFILES in src/cleveragents/domain/models/core/automation_profile.py (lines ~200–300) and document all 8 mismatched description fields
Clarify with spec whether list uses abbreviated descriptions and show uses full descriptions, or both use the same value
Update all 8 description fields in BUILTIN_PROFILES to match spec-required values
If list vs show use different description lengths, implement the truncation/short-form logic accordingly
Tests (Behave): Add/update scenarios asserting each profile's description matches the spec for both list and show commands
Tests (Robot): Add/update integration test asserting automation-profile list --format json and automation-profile show <profile> return correct descriptions for all 8 profiles
Verify coverage ≥ 97% via nox -s coverage_report
Run nox (all default sessions) and fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
All 8 built-in profile descriptions in BUILTIN_PROFILES exactly match the values specified in docs/specification.md.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly: fix(automation-profile): correct descriptions for all 8 built-in profiles to match spec
The commit body footer includes ISSUES CLOSED: #<this issue number>.
The commit is pushed to the remote on branch fix/automation-profile-builtin-descriptions.
The commit is submitted as a pull request to master, reviewed, and merged.
All nox stages pass.
Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Background and Context During UAT testing, all 8 built-in automation profiles were found to have descriptions that do not match the values defined in `docs/specification.md`. The specification is the source of truth per project convention, so the implementation must be updated to match it. This affects both the `automation-profile list` and `automation-profile show` CLI commands, meaning users cannot rely on the displayed descriptions to understand profile behaviour as documented. ## Current Behavior In `src/cleveragents/domain/models/core/automation_profile.py`, the `BUILTIN_PROFILES` dict uses these descriptions: | Profile | Actual Description | |---|---| | `manual` | `"Human approves every action"` | | `review` | `"Human reviews before apply"` | | `supervised` | `"Human reviews strategy and execution"` | | `cautious` | `"Probabilistic gates on most actions"` | | `trusted` | `"Auto for most, human for apply and revert"` | | `auto` | `"Fully automatic except apply"` | | `ci` | `"Designed for CI pipelines"` | | `full-auto` | `"No gates, no sandbox, no checkpoints"` | None of these match the spec-required descriptions. **Steps to Reproduce:** ```bash agents automation-profile list --format json # Expected: "description": "Human-driven (default)" for manual # Actual: "description": "Human approves every action" for manual ``` ## Expected Behavior Per `docs/specification.md` lines 17003–17011 (the `automation-profile list` JSON output) and line 17182 (the `automation-profile show` output for `trusted`), the descriptions should be: | Profile | Spec Description (list / show) | |---|---| | `manual` | `"Human-driven (default)"` | | `review` | `"Auto-phase, manual decisions"` | | `supervised` | `"Auto-plan, manual exec"` | | `cautious` | `"Confidence-gated automation"` | | `trusted` | `"Auto-exec, manual apply. Day-to-day development"` (full form from `show`) | | `auto` | `"Full auto except apply"` | | `ci` | `"Full auto, safe CI/CD"` | | `full-auto` | `"Complete automation"` | > **Note:** The spec suggests `list` may show abbreviated descriptions while `show` uses the full description (e.g., `trusted`). This distinction should be clarified and implemented correctly during the fix. ## Impact - `automation-profile list` shows wrong descriptions for all 8 built-in profiles. - `automation-profile show` shows wrong descriptions for all 8 built-in profiles. - Users cannot rely on descriptions to understand profile behaviour as documented. --- ## Metadata - **Branch**: `fix/automation-profile-builtin-descriptions` - **Commit Message**: `fix(automation-profile): correct descriptions for all 8 built-in profiles to match spec` - **Milestone**: v3.7.0 - **Parent Epic**: #362 ## Subtasks - [ ] Audit `BUILTIN_PROFILES` in `src/cleveragents/domain/models/core/automation_profile.py` (lines ~200–300) and document all 8 mismatched `description` fields - [ ] Clarify with spec whether `list` uses abbreviated descriptions and `show` uses full descriptions, or both use the same value - [ ] Update all 8 `description` fields in `BUILTIN_PROFILES` to match spec-required values - [ ] If `list` vs `show` use different description lengths, implement the truncation/short-form logic accordingly - [ ] Tests (Behave): Add/update scenarios asserting each profile's description matches the spec for both `list` and `show` commands - [ ] Tests (Robot): Add/update integration test asserting `automation-profile list --format json` and `automation-profile show <profile>` return correct descriptions for all 8 profiles - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions) and fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - All 8 built-in profile descriptions in `BUILTIN_PROFILES` exactly match the values specified in `docs/specification.md`. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly: `fix(automation-profile): correct descriptions for all 8 built-in profiles to match spec` - The commit body footer includes `ISSUES CLOSED: #<this issue number>`. - The commit is pushed to the remote on branch `fix/automation-profile-builtin-descriptions`. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged**. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.7.0 milestone

2026-04-03 03:59:27 +00:00

freemo added a new dependency

2026-04-03 04:00:51 +00:00

#362 Epic: Security & Safety Hardening

freemo referenced this issue

2026-04-03 04:01:23 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 701 — v3.7.0 #2093

freemo referenced this issue

2026-04-03 04:03:55 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:08:26 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:11:31 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 711 — v3.7.0 #2115

freemo self-assigned this

2026-04-03 16:58:08 +00:00

freemo commented

2026-04-03 18:28:19 +00:00

Author

Owner

MoSCoW classification: Should Have

Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

MoSCoW classification: **Should Have** Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner