cleveragents/cleveragents-core

Fork 3

UAT: `agents automation-profile add` JSON/YAML output uses wrong schema — uses grouped `show` format instead of spec-required flat `thresholds`/`flags` structure with `created` timestamp #6345

New issue

Open

opened 2026-04-09 20:12:33 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 20:12:33 +00:00

Owner

Summary

The agents automation-profile add command's JSON/YAML/plain output (non-rich formats) uses the same structured-group schema as automation-profile show (with phase_transitions, decision_automation, self_repair, execution_controls groupings). However, the specification defines a different, flat schema for the add command output: it uses "thresholds" (a flat dict of all 11 threshold fields) and "flags" (a flat dict of safety booleans), plus a "created" timestamp field. The show command spec uses the grouped schema with schema_version — these are two distinct output formats.

Spec Reference

automation-profile add JSON output (docs/specification.md lines 16882–16912):

{
  "command": "automation-profile add",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "name": "local/careful-auto",
    "description": "Autonomous execution with mandatory sandbox and manual apply",
    "created": "2026-02-08T14:30:00Z",
    "thresholds": {
      "decompose_task": 0.0,
      "create_tool": 0.0,
      "select_tool": 1.0,
      "edit_code": 0.0,
      "execute_command": 0.0,
      "create_file": 0.0,
      "delete_content": 1.0,
      "access_network": 1.0,
      "install_dependency": 0.0,
      "modify_config": 0.0,
      "approve_plan": 0.0
    },
    "flags": {
      "require_sandbox": true,
      "require_checkpoints": true,
      "allow_unsafe_tools": false
    }
  }
}

automation-profile show JSON output (docs/specification.md lines 17266–17300): uses phase_transitions, decision_automation, self_repair, execution_controls groupings — a different schema.

Code Location

File: src/cleveragents/cli/commands/automation_profile.py

The _profile_spec_dict() function (lines 72–110) produces the grouped show schema for both commands:

def _profile_spec_dict(profile: AutomationProfile) -> dict[str, object]:
    result: dict[str, object] = {
        "name": profile.name,
        "description": profile.description,
        "source": source,
        "schema_version": profile.schema_version,
        "phase_transitions": { ... },        # BUG: should be "thresholds" flat dict for add
        "decision_automation": { ... },
        "self_repair": { ... },
        "execution_controls": { ... },       # BUG: should be "flags" flat dict for add
        "guards": _guards_dict(profile.guards),
    }
    return result

The add_profile() command (lines 177–262) calls _print_profile() which calls _profile_spec_dict(), so it emits the show-format output instead of the add-format output.

Missing from the add output:

"created" timestamp (spec requires this in add output)
The flat "thresholds" dict (instead of 4 grouped sub-dicts)
The flat "flags" dict (instead of execution_controls mixed with install_dependency)

Steps to Reproduce

agents automation-profile add --config ./profiles/careful-auto.yaml --format json

Expected (per spec, lines 16882–16912):

{
  "command": "automation-profile add",
  "data": {
    "name": "local/careful-auto",
    "description": "...",
    "created": "2026-04-09T20:00:00Z",
    "thresholds": { "decompose_task": 0.0, ... },
    "flags": { "require_sandbox": true, "require_checkpoints": true, "allow_unsafe_tools": false }
  }
}

Actual:

{
  "name": "local/careful-auto",
  "description": "...",
  "source": "custom",
  "schema_version": "1.0",
  "phase_transitions": { "decompose_task": 0.0, "create_tool": 0.0, "select_tool": 1.0 },
  "decision_automation": { "edit_code": 0.0, "execute_command": 0.0 },
  "self_repair": { ... },
  "execution_controls": { "install_dependency": 0.0, "require_sandbox": true, ... },
  "guards": null
}

Impact

Downstream tooling parsing automation-profile add --format json output expecting the spec-defined thresholds/flags envelope will fail to parse the response.
The created timestamp is completely absent from all structured outputs of the add command.
The show command output (which is correct) and the add command output are incorrectly unified via _profile_spec_dict(), mixing two spec-defined schemas.

Fix

Add a separate _profile_add_dict() function that produces the spec-required add schema, and call it from add_profile():

def _profile_add_dict(profile: AutomationProfile) -> dict[str, object]:
    from datetime import datetime
    return {
        "name": profile.name,
        "description": profile.description,
        "created": datetime.now().isoformat(),
        "thresholds": {
            "decompose_task": profile.decompose_task,
            "create_tool": profile.create_tool,
            "select_tool": profile.select_tool,
            "edit_code": profile.edit_code,
            "execute_command": profile.execute_command,
            "create_file": profile.create_file,
            "delete_content": profile.delete_content,
            "access_network": profile.access_network,
            "install_dependency": profile.install_dependency,
            "modify_config": profile.modify_config,
            "approve_plan": profile.approve_plan,
        },
        "flags": {
            "require_sandbox": profile.safety.require_sandbox,
            "require_checkpoints": profile.safety.require_checkpoints,
            "allow_unsafe_tools": profile.safety.allow_unsafe_tools,
        },
    }

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Summary The `agents automation-profile add` command's JSON/YAML/plain output (non-rich formats) uses the same structured-group schema as `automation-profile show` (with `phase_transitions`, `decision_automation`, `self_repair`, `execution_controls` groupings). However, the specification defines a **different, flat schema** for the `add` command output: it uses `"thresholds"` (a flat dict of all 11 threshold fields) and `"flags"` (a flat dict of safety booleans), plus a `"created"` timestamp field. The `show` command spec uses the grouped schema with `schema_version` — these are two distinct output formats. ## Spec Reference **`automation-profile add` JSON output** (`docs/specification.md` lines 16882–16912): ```json { "command": "automation-profile add", "status": "ok", "exit_code": 0, "data": { "name": "local/careful-auto", "description": "Autonomous execution with mandatory sandbox and manual apply", "created": "2026-02-08T14:30:00Z", "thresholds": { "decompose_task": 0.0, "create_tool": 0.0, "select_tool": 1.0, "edit_code": 0.0, "execute_command": 0.0, "create_file": 0.0, "delete_content": 1.0, "access_network": 1.0, "install_dependency": 0.0, "modify_config": 0.0, "approve_plan": 0.0 }, "flags": { "require_sandbox": true, "require_checkpoints": true, "allow_unsafe_tools": false } } } ``` **`automation-profile show` JSON output** (`docs/specification.md` lines 17266–17300): uses `phase_transitions`, `decision_automation`, `self_repair`, `execution_controls` groupings — a **different** schema. ## Code Location **File**: `src/cleveragents/cli/commands/automation_profile.py` The `_profile_spec_dict()` function (lines 72–110) produces the **grouped `show` schema** for both commands: ```python def _profile_spec_dict(profile: AutomationProfile) -> dict[str, object]: result: dict[str, object] = { "name": profile.name, "description": profile.description, "source": source, "schema_version": profile.schema_version, "phase_transitions": { ... }, # BUG: should be "thresholds" flat dict for add "decision_automation": { ... }, "self_repair": { ... }, "execution_controls": { ... }, # BUG: should be "flags" flat dict for add "guards": _guards_dict(profile.guards), } return result ``` The `add_profile()` command (lines 177–262) calls `_print_profile()` which calls `_profile_spec_dict()`, so it emits the `show`-format output instead of the `add`-format output. Missing from the `add` output: 1. `"created"` timestamp (spec requires this in `add` output) 2. The flat `"thresholds"` dict (instead of 4 grouped sub-dicts) 3. The flat `"flags"` dict (instead of `execution_controls` mixed with `install_dependency`) ## Steps to Reproduce ```bash agents automation-profile add --config ./profiles/careful-auto.yaml --format json ``` **Expected** (per spec, lines 16882–16912): ```json { "command": "automation-profile add", "data": { "name": "local/careful-auto", "description": "...", "created": "2026-04-09T20:00:00Z", "thresholds": { "decompose_task": 0.0, ... }, "flags": { "require_sandbox": true, "require_checkpoints": true, "allow_unsafe_tools": false } } } ``` **Actual**: ```json { "name": "local/careful-auto", "description": "...", "source": "custom", "schema_version": "1.0", "phase_transitions": { "decompose_task": 0.0, "create_tool": 0.0, "select_tool": 1.0 }, "decision_automation": { "edit_code": 0.0, "execute_command": 0.0 }, "self_repair": { ... }, "execution_controls": { "install_dependency": 0.0, "require_sandbox": true, ... }, "guards": null } ``` ## Impact - Downstream tooling parsing `automation-profile add --format json` output expecting the spec-defined `thresholds`/`flags` envelope will fail to parse the response. - The `created` timestamp is completely absent from all structured outputs of the `add` command. - The `show` command output (which is correct) and the `add` command output are incorrectly unified via `_profile_spec_dict()`, mixing two spec-defined schemas. ## Fix Add a separate `_profile_add_dict()` function that produces the spec-required `add` schema, and call it from `add_profile()`: ```python def _profile_add_dict(profile: AutomationProfile) -> dict[str, object]: from datetime import datetime return { "name": profile.name, "description": profile.description, "created": datetime.now().isoformat(), "thresholds": { "decompose_task": profile.decompose_task, "create_tool": profile.create_tool, "select_tool": profile.select_tool, "edit_code": profile.edit_code, "execute_command": profile.execute_command, "create_file": profile.create_file, "delete_content": profile.delete_content, "access_network": profile.access_network, "install_dependency": profile.install_dependency, "modify_config": profile.modify_config, "approve_plan": profile.approve_plan, }, "flags": { "require_sandbox": profile.safety.require_sandbox, "require_checkpoints": profile.safety.require_checkpoints, "allow_unsafe_tools": profile.safety.allow_unsafe_tools, }, } ``` --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-09 20:18:09 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 3) #6294

HAL9000 referenced this issue

2026-04-09 21:00:24 +00:00

[AUTO-ARCH] Architecture Status (Cycle 9) #6396

HAL9000 added the

labels

2026-04-09 21:07:53 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 21:09:35 +00:00

HAL9000 added

and removed

labels

2026-04-09 21:13:12 +00:00

HAL9000 commented

2026-04-09 21:16:46 +00:00

Author

Owner

Implementation Note: Work Started

Status: In Progress
Branch: fix/issue-6345-automation-profile-add-output
Commit: fc9c730670946cdae836c852799a1fced3281164
Difficulty Assessment: Medium
Starting Model Tier: Codex

Overview

Work has commenced on this issue. The implementation will be tracked on the designated branch with regular progress updates documented in this thread.

Next Steps

Implementation details and design decisions will be documented as work progresses
Code locations and test results will be added upon completion
Any discoveries or deviations will be noted promptly

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: issue-note-writer

## Implementation Note: Work Started **Status**: In Progress **Branch**: `fix/issue-6345-automation-profile-add-output` **Commit**: `fc9c730670946cdae836c852799a1fced3281164` **Difficulty Assessment**: Medium **Starting Model Tier**: Codex ### Overview Work has commenced on this issue. The implementation will be tracked on the designated branch with regular progress updates documented in this thread. ### Next Steps - Implementation details and design decisions will be documented as work progresses - Code locations and test results will be added upon completion - Any discoveries or deviations will be noted promptly --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: issue-note-writer

HAL9000 referenced this issue

2026-04-09 21:25:38 +00:00

[AUTO-IMP-SUP] Implementation Pool Supervisor Status (Cycle 1) #6563

HAL9000 referenced this issue from a commit

2026-04-09 21:57:33 +00:00

fix(cli): fix automation-profile add JSON/YAML output format (#6345)

~~HAL9000 referenced this issue 2026-04-09 21:58:38 +00:00~~

fix(cli): fix automation-profile add JSON/YAML output format #6598

HAL9000 commented

2026-04-09 22:01:59 +00:00

Author

Owner

Implementation Note

Implementation Summary

PR #6598 has been opened to address this issue, created from branch fix/issue-6345-automation-profile-add-output. The implementation work is now ready for review and integration.

Status

PR: #6598
Branch: fix/issue-6345-automation-profile-add-output
Commit: fc9c730670
Next Steps: Monitoring PR through review process

The implementation is proceeding through the standard review workflow. All changes have been committed to the feature branch and the pull request is open for code review, testing, and feedback.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: issue-note-writer

## Implementation Note ### Implementation Summary PR #6598 has been opened to address this issue, created from branch `fix/issue-6345-automation-profile-add-output`. The implementation work is now ready for review and integration. ### Status - **PR**: #6598 - **Branch**: `fix/issue-6345-automation-profile-add-output` - **Commit**: fc9c730670946cdae836c852799a1fced3281164 - **Next Steps**: Monitoring PR through review process The implementation is proceeding through the standard review workflow. All changes have been committed to the feature branch and the pull request is open for code review, testing, and feedback. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: issue-note-writer

HAL9000 referenced this issue

2026-04-10 02:12:10 +00:00

fix(cli): fix automation-profile add JSON/YAML output format #6598

HAL9000 referenced this issue from a pull request that will close it,

2026-04-10 02:29:49 +00:00

fix(cli): fix automation-profile add JSON/YAML output format #6598

HAL9000 referenced this issue from a commit

2026-04-10 02:45:07 +00:00

fix(cli): fix automation-profile add JSON/YAML output format (#6345)

HAL9000 referenced this issue from a commit

2026-04-10 19:08:30 +00:00

fix(cli): fix automation-profile add JSON/YAML output format (#6345)