cleveragents/cleveragents-core

Fork 3

UAT: `agents automation-profile add` JSON output uses wrong data structure — spec requires `thresholds`/`flags` grouping, not `phase_transitions`/`decision_automation`/`self_repair`/`execution_controls` #2081

New issue

Open

opened 2026-04-03 03:53:55 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 03:53:55 +00:00

Owner

Metadata

Branch: fix/automation-profile-add-json-structure
Commit Message: fix(cli): correct automation-profile add JSON output to use thresholds/flags structure per spec
Milestone: v3.7.0
Parent Epic: #362

Summary

The agents automation-profile add command's JSON output uses a different data structure than what the specification requires. The spec defines thresholds (all 11 threshold fields flat) and flags (3 safety booleans) groupings, plus a created timestamp. The implementation reuses _profile_spec_dict() which produces phase_transitions, decision_automation, self_repair, and execution_controls groupings (the show command's structure).

Expected Behavior (from spec)

Per docs/specification.md lines 16792–16820, the JSON output for add should be:

{
  "command": "automation-profile add",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "name": "local/careful-auto",
    "description": "Autonomous execution with mandatory sandbox and manual apply",
    "created": "2026-02-08T14:30:00Z",
    "thresholds": {
      "decompose_task": 0.0,
      "create_tool": 0.0,
      "select_tool": 1.0,
      "edit_code": 0.0,
      "execute_command": 0.0,
      "create_file": 0.0,
      "delete_content": 1.0,
      "access_network": 1.0,
      "install_dependency": 0.0,
      "modify_config": 0.0,
      "approve_plan": 0.0
    },
    "flags": {
      "require_sandbox": true,
      "require_checkpoints": true,
      "allow_unsafe_tools": false
    }
  },
  "timing": { "started": "2026-02-08T14:30:00Z", "duration_ms": 70 },
  "messages": ["Profile registered"]
}

The rich output should also use two separate panels: "Profile Registered" (name, description, created timestamp) and "Confidence Thresholds" (all 11 thresholds + 3 safety flags).

Actual Behavior

In src/cleveragents/cli/commands/automation_profile.py, the add_profile() command (lines 128–130) calls _print_profile() which uses _profile_spec_dict(). This produces:

phase_transitions (decompose_task, create_tool, select_tool)
decision_automation (edit_code, execute_command)
self_repair (create_file, delete_content, access_network, modify_config, approve_plan)
execution_controls (install_dependency, require_sandbox, require_checkpoints, allow_unsafe_tools)

Instead of the spec-required thresholds and flags groupings.

Additionally:

No created timestamp is included in the output
The rich output uses a single panel instead of two separate panels
The panel title is "Profile Added" or "Profile Updated" instead of "Profile Registered"

Impact

Programmatic consumers of --format json output will receive an unexpected structure
The created timestamp is missing from the output
The rich output format doesn't match the spec

Steps to Reproduce

agents automation-profile add --config ./profile.yaml --format json
# Expected: {"data": {"name": "...", "created": "...", "thresholds": {...}, "flags": {...}}}
# Actual: {"data": {"name": "...", "phase_transitions": {...}, "decision_automation": {...}, ...}}

Code Location

src/cleveragents/cli/commands/automation_profile.py lines 128–130: add_profile() calls _print_profile() which uses _profile_spec_dict()
src/cleveragents/cli/commands/automation_profile.py lines 72–100: _profile_spec_dict() — produces wrong structure for add command

Subtasks

Implement _add_profile_dict() function that produces the spec-required thresholds/flags structure with created timestamp (ISO 8601 UTC)
Refactor add_profile() to use _add_profile_dict() instead of _profile_spec_dict() for JSON output
Refactor add_profile() rich output to use two separate panels: "Profile Registered" (name, description, created) and "Confidence Thresholds" (all 11 thresholds + 3 safety flags)
Update messages field in JSON output to ["Profile registered"] (lowercase, matching spec)
Tests (Behave): Add/update scenarios covering add JSON output structure (thresholds, flags, created fields present and correctly grouped)
Tests (Behave): Add/update scenarios covering add rich output two-panel layout and "Profile Registered" panel title
Tests (Robot): Add/update integration test asserting --format json output structure for automation-profile add
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/automation-profile-add-json-structure` - **Commit Message**: `fix(cli): correct automation-profile add JSON output to use thresholds/flags structure per spec` - **Milestone**: v3.7.0 - **Parent Epic**: #362 ## Summary The `agents automation-profile add` command's JSON output uses a different data structure than what the specification requires. The spec defines `thresholds` (all 11 threshold fields flat) and `flags` (3 safety booleans) groupings, plus a `created` timestamp. The implementation reuses `_profile_spec_dict()` which produces `phase_transitions`, `decision_automation`, `self_repair`, and `execution_controls` groupings (the `show` command's structure). ## Expected Behavior (from spec) Per `docs/specification.md` lines 16792–16820, the JSON output for `add` should be: ```json { "command": "automation-profile add", "status": "ok", "exit_code": 0, "data": { "name": "local/careful-auto", "description": "Autonomous execution with mandatory sandbox and manual apply", "created": "2026-02-08T14:30:00Z", "thresholds": { "decompose_task": 0.0, "create_tool": 0.0, "select_tool": 1.0, "edit_code": 0.0, "execute_command": 0.0, "create_file": 0.0, "delete_content": 1.0, "access_network": 1.0, "install_dependency": 0.0, "modify_config": 0.0, "approve_plan": 0.0 }, "flags": { "require_sandbox": true, "require_checkpoints": true, "allow_unsafe_tools": false } }, "timing": { "started": "2026-02-08T14:30:00Z", "duration_ms": 70 }, "messages": ["Profile registered"] } ``` The rich output should also use **two separate panels**: "Profile Registered" (name, description, created timestamp) and "Confidence Thresholds" (all 11 thresholds + 3 safety flags). ## Actual Behavior In `src/cleveragents/cli/commands/automation_profile.py`, the `add_profile()` command (lines 128–130) calls `_print_profile()` which uses `_profile_spec_dict()`. This produces: - `phase_transitions` (decompose_task, create_tool, select_tool) - `decision_automation` (edit_code, execute_command) - `self_repair` (create_file, delete_content, access_network, modify_config, approve_plan) - `execution_controls` (install_dependency, require_sandbox, require_checkpoints, allow_unsafe_tools) Instead of the spec-required `thresholds` and `flags` groupings. Additionally: - No `created` timestamp is included in the output - The rich output uses a single panel instead of two separate panels - The panel title is "Profile Added" or "Profile Updated" instead of "Profile Registered" ## Impact - Programmatic consumers of `--format json` output will receive an unexpected structure - The `created` timestamp is missing from the output - The rich output format doesn't match the spec ## Steps to Reproduce ```bash agents automation-profile add --config ./profile.yaml --format json # Expected: {"data": {"name": "...", "created": "...", "thresholds": {...}, "flags": {...}}} # Actual: {"data": {"name": "...", "phase_transitions": {...}, "decision_automation": {...}, ...}} ``` ## Code Location - `src/cleveragents/cli/commands/automation_profile.py` lines 128–130: `add_profile()` calls `_print_profile()` which uses `_profile_spec_dict()` - `src/cleveragents/cli/commands/automation_profile.py` lines 72–100: `_profile_spec_dict()` — produces wrong structure for `add` command ## Subtasks - [ ] Implement `_add_profile_dict()` function that produces the spec-required `thresholds`/`flags` structure with `created` timestamp (ISO 8601 UTC) - [ ] Refactor `add_profile()` to use `_add_profile_dict()` instead of `_profile_spec_dict()` for JSON output - [ ] Refactor `add_profile()` rich output to use two separate panels: "Profile Registered" (name, description, created) and "Confidence Thresholds" (all 11 thresholds + 3 safety flags) - [ ] Update `messages` field in JSON output to `["Profile registered"]` (lowercase, matching spec) - [ ] Tests (Behave): Add/update scenarios covering `add` JSON output structure (`thresholds`, `flags`, `created` fields present and correctly grouped) - [ ] Tests (Behave): Add/update scenarios covering `add` rich output two-panel layout and "Profile Registered" panel title - [ ] Tests (Robot): Add/update integration test asserting `--format json` output structure for `automation-profile add` - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.7.0 milestone

2026-04-03 03:54:00 +00:00

freemo added a new dependency

2026-04-03 03:54:40 +00:00

#362 Epic: Security & Safety Hardening

freemo referenced this issue

2026-04-03 03:57:57 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:01:23 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 701 — v3.7.0 #2093

freemo referenced this issue

2026-04-03 04:03:55 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo self-assigned this

2026-04-03 16:58:09 +00:00

freemo commented

2026-04-03 18:28:20 +00:00

Author

Owner

MoSCoW classification: Should Have

Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

MoSCoW classification: **Should Have** Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner