UAT: automation-profile add JSON output uses wrong data structure (missing thresholds/flags grouping) #6775

New issue

Open

opened 2026-04-10 02:05:25 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-10 02:05:25 +00:00

Owner

Background and Context

The agents automation-profile add command JSON output schema is defined in docs/specification.md section ##### agents automation-profile add. The spec defines a specific data structure with thresholds and flags sub-objects, but the implementation uses the same generic profile dict used by show, which has a different structure.

Current Behavior

The add_profile command calls _print_profile(profile, title="Profile Added", fmt=fmt) which calls _profile_spec_dict(profile) for non-rich formats. The _profile_spec_dict function returns:

{
  "name": "local/careful-auto",
  "description": "...",
  "source": "custom",
  "schema_version": "1.0",
  "phase_transitions": { "decompose_task": ..., "create_tool": ..., "select_tool": ... },
  "decision_automation": { "edit_code": ..., "execute_command": ... },
  "self_repair": { "create_file": ..., "delete_content": ..., "access_network": ..., "modify_config": ..., "approve_plan": ... },
  "execution_controls": { "install_dependency": ..., "require_sandbox": ..., "require_checkpoints": ..., "allow_unsafe_tools": ... },
  "guards": null
}

Expected Behavior (from spec)

The spec JSON for automation-profile add (docs/specification.md, lines 16882–16912) defines:

{
  "command": "automation-profile add",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "name": "local/careful-auto",
    "description": "Autonomous execution with mandatory sandbox and manual apply",
    "created": "2026-02-08T14:30:00Z",
    "thresholds": {
      "decompose_task": 0.0,
      "create_tool": 0.0,
      "select_tool": 1.0,
      "edit_code": 0.0,
      "execute_command": 0.0,
      "create_file": 0.0,
      "delete_content": 1.0,
      "access_network": 1.0,
      "install_dependency": 0.0,
      "modify_config": 0.0,
      "approve_plan": 0.0
    },
    "flags": {
      "require_sandbox": true,
      "require_checkpoints": true,
      "allow_unsafe_tools": false
    }
  },
  "timing": { "started": "...", "duration_ms": 70 },
  "messages": ["Profile registered"]
}

Key differences:

The spec add output uses a flat thresholds object containing all 11 threshold fields (not split into phase_transitions/decision_automation/self_repair/execution_controls)
The spec add output uses a flat flags object containing require_sandbox, require_checkpoints, allow_unsafe_tools (not embedded in execution_controls)
The spec add output includes a created timestamp field
The spec add output does NOT include source, schema_version, or guards

Impact

Scripts parsing the add command JSON output will receive wrong data: no thresholds or flags keys, instead receiving the show command's grouping structure. The created timestamp is also completely absent.

Steps to Reproduce

Create a profile YAML file test.yaml with valid content
Run agents automation-profile add --config test.yaml --format json
Observe: JSON data has phase_transitions, decision_automation, etc., and schema_version/guards
Expected: JSON data has thresholds, flags, and created fields

Acceptance Criteria

automation-profile add --format json data field contains: name, description, created, thresholds, flags
thresholds is a flat object with all 11 threshold field values
flags contains require_sandbox, require_checkpoints, allow_unsafe_tools
created contains the ISO 8601 timestamp
No source, schema_version, guards, or nested threshold groupings
All tests pass with nox

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Background and Context The `agents automation-profile add` command JSON output schema is defined in `docs/specification.md` section `##### agents automation-profile add`. The spec defines a specific `data` structure with `thresholds` and `flags` sub-objects, but the implementation uses the same generic profile dict used by `show`, which has a different structure. ## Current Behavior The `add_profile` command calls `_print_profile(profile, title="Profile Added", fmt=fmt)` which calls `_profile_spec_dict(profile)` for non-rich formats. The `_profile_spec_dict` function returns: ```json { "name": "local/careful-auto", "description": "...", "source": "custom", "schema_version": "1.0", "phase_transitions": { "decompose_task": ..., "create_tool": ..., "select_tool": ... }, "decision_automation": { "edit_code": ..., "execute_command": ... }, "self_repair": { "create_file": ..., "delete_content": ..., "access_network": ..., "modify_config": ..., "approve_plan": ... }, "execution_controls": { "install_dependency": ..., "require_sandbox": ..., "require_checkpoints": ..., "allow_unsafe_tools": ... }, "guards": null } ``` ## Expected Behavior (from spec) The spec JSON for `automation-profile add` (`docs/specification.md`, lines 16882–16912) defines: ```json { "command": "automation-profile add", "status": "ok", "exit_code": 0, "data": { "name": "local/careful-auto", "description": "Autonomous execution with mandatory sandbox and manual apply", "created": "2026-02-08T14:30:00Z", "thresholds": { "decompose_task": 0.0, "create_tool": 0.0, "select_tool": 1.0, "edit_code": 0.0, "execute_command": 0.0, "create_file": 0.0, "delete_content": 1.0, "access_network": 1.0, "install_dependency": 0.0, "modify_config": 0.0, "approve_plan": 0.0 }, "flags": { "require_sandbox": true, "require_checkpoints": true, "allow_unsafe_tools": false } }, "timing": { "started": "...", "duration_ms": 70 }, "messages": ["Profile registered"] } ``` **Key differences:** 1. The spec `add` output uses a **flat `thresholds` object** containing all 11 threshold fields (not split into phase_transitions/decision_automation/self_repair/execution_controls) 2. The spec `add` output uses a **flat `flags` object** containing `require_sandbox`, `require_checkpoints`, `allow_unsafe_tools` (not embedded in execution_controls) 3. The spec `add` output includes a `created` timestamp field 4. The spec `add` output does NOT include `source`, `schema_version`, or `guards` ## Impact Scripts parsing the `add` command JSON output will receive wrong data: no `thresholds` or `flags` keys, instead receiving the `show` command's grouping structure. The `created` timestamp is also completely absent. ## Steps to Reproduce 1. Create a profile YAML file `test.yaml` with valid content 2. Run `agents automation-profile add --config test.yaml --format json` 3. Observe: JSON `data` has `phase_transitions`, `decision_automation`, etc., and `schema_version`/`guards` 4. Expected: JSON `data` has `thresholds`, `flags`, and `created` fields ## Acceptance Criteria - [ ] `automation-profile add --format json` `data` field contains: `name`, `description`, `created`, `thresholds`, `flags` - [ ] `thresholds` is a flat object with all 11 threshold field values - [ ] `flags` contains `require_sandbox`, `require_checkpoints`, `allow_unsafe_tools` - [ ] `created` contains the ISO 8601 timestamp - [ ] No `source`, `schema_version`, `guards`, or nested threshold groupings - [ ] All tests pass with `nox` --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added this to the v3.4.0 milestone

2026-04-10 02:05:58 +00:00

HAL9000 added the

labels

2026-04-10 02:05:58 +00:00

HAL9000 referenced this issue

2026-04-10 02:08:40 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 4) #6758

HAL9000 added

and removed

labels

2026-04-14 07:19:48 +00:00

HAL9000 commented

2026-04-14 07:19:48 +00:00

Author

Owner

✅ Verified — UAT bug: automation-profile add JSON output has wrong structure. MoSCoW: Should-have. Priority: Medium.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — UAT bug: automation-profile add JSON output has wrong structure. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor