cleveragents/cleveragents-core

Fork 3

UAT: All `agents skill` commands produce raw JSON output without the spec-required `{command, status, exit_code, data, timing, messages}` envelope #2087

New issue

Open

opened 2026-04-03 03:56:54 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 03:56:54 +00:00

Owner

Metadata

Branch: fix/skill-commands-json-envelope
Commit Message: fix(cli): wrap skill command JSON output in spec-required envelope
Milestone: v3.7.0
Parent Epic: #936

Background and Context

The specification defines that all CLI commands must produce JSON output wrapped in a formal envelope structure: {command, status, exit_code, data, timing, messages} (spec §Output Rendering Framework). The Output Rendering Pipeline Integration (Epic #936) is actively migrating all CLI commands to this envelope via the OutputSession framework.

The agents skill command group (add, remove, list, show, tools, refresh) was implemented before this migration was completed. All six subcommands currently call format_output(data, fmt) directly with raw data dicts/lists, bypassing the spec-required envelope entirely. This means any consumer of agents skill --format json output (scripts, CI pipelines, other agents) receives non-conformant data that breaks against the spec contract.

Current Behavior

All agents skill subcommands produce raw, unwrapped JSON when --format json is used. For example, agents skill add --format json outputs:

{
  "name": "local/test-skill",
  "namespace": "local",
  "short_name": "test-skill",
  "description": "...",
  "config_path": "...",
  "tool_refs": [...],
  "is_new": true
}

The root cause is in src/cleveragents/cli/commands/skill.py. Each command calls format_output() directly without an envelope wrapper. Example from add():

data = _skill_spec_dict(skill, service)
data["is_new"] = is_new
typer.echo(format_output(data, fmt))  # BUG: no envelope wrapper

All six affected commands:

agents skill add --format json — outputs raw skill dict
agents skill remove --format json — outputs {"name": ..., "removed": true, "tools_removed": N}
agents skill list --format json — outputs raw list of skill dicts
agents skill show --format json — outputs raw skill dict
agents skill tools --format json — outputs raw tools list
agents skill refresh --format json — outputs raw refresh result dict

Expected Behavior

Each agents skill subcommand must wrap its output data in the spec-required envelope before calling format_output(). Per spec §agents skill add, the correct JSON output is:

{
  "command": "skill add --config ./skills/devops-toolkit.yaml",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "skill": { "name": "local/devops-toolkit", "..." : "..." },
    "includes": [...],
    "tool_sources": [...],
    "total_tools": 23,
    "mcp_servers": [...]
  },
  "timing": { "duration_ms": 150 },
  "messages": [{ "level": "ok", "text": "Skill registered with 23 tools" }]
}

The fix pattern for each command is:

envelope = {
    "command": f"skill add --config {config}",
    "status": "ok",
    "exit_code": 0,
    "data": { "skill": skill_data, ... },
    "timing": {"duration_ms": elapsed_ms},
    "messages": [{"level": "ok", "text": "Skill registered with N tools"}]
}
typer.echo(format_output(envelope, fmt))

Acceptance Criteria

agents skill add --format json output matches the spec envelope structure with command, status, exit_code, data, timing, and messages keys
agents skill remove --format json output is wrapped in the spec envelope
agents skill list --format json output is wrapped in the spec envelope (skill list in data)
agents skill show --format json output is wrapped in the spec envelope
agents skill tools --format json output is wrapped in the spec envelope (tools list in data)
agents skill refresh --format json output is wrapped in the spec envelope
All envelope fields (command, status, exit_code, data, timing, messages) are present and correctly typed for all six commands
Non-JSON output formats (table, plain, rich) are unaffected
All existing Behave scenarios for agents skill commands continue to pass

Subtasks

Audit src/cleveragents/cli/commands/skill.py — document the exact raw output shape for each of the six subcommands
Implement envelope wrapping for skill add — include data.skill, data.includes, data.tool_sources, data.total_tools, data.mcp_servers
Implement envelope wrapping for skill remove — include data.name, data.removed, data.tools_removed
Implement envelope wrapping for skill list — include data.skills (list), data.total
Implement envelope wrapping for skill show — include data.skill
Implement envelope wrapping for skill tools — include data.tools (list), data.total
Implement envelope wrapping for skill refresh — include data.refreshed, data.changes
Tests (Behave): Add/update scenarios asserting envelope structure for each subcommand's JSON output
Tests (Robot): Add integration tests verifying --format json envelope conformance for all six commands
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
All six agents skill subcommands produce spec-conformant JSON envelopes when invoked with --format json.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(cli): wrap skill command JSON output in spec-required envelope), followed by a blank line, then additional lines providing relevant implementation details.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/skill-commands-json-envelope).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/skill-commands-json-envelope` - **Commit Message**: `fix(cli): wrap skill command JSON output in spec-required envelope` - **Milestone**: v3.7.0 - **Parent Epic**: #936 ## Background and Context The specification defines that all CLI commands must produce JSON output wrapped in a formal envelope structure: `{command, status, exit_code, data, timing, messages}` (spec §Output Rendering Framework). The Output Rendering Pipeline Integration (Epic #936) is actively migrating all CLI commands to this envelope via the `OutputSession` framework. The `agents skill` command group (`add`, `remove`, `list`, `show`, `tools`, `refresh`) was implemented before this migration was completed. All six subcommands currently call `format_output(data, fmt)` directly with raw data dicts/lists, bypassing the spec-required envelope entirely. This means any consumer of `agents skill --format json` output (scripts, CI pipelines, other agents) receives non-conformant data that breaks against the spec contract. ## Current Behavior All `agents skill` subcommands produce raw, unwrapped JSON when `--format json` is used. For example, `agents skill add --format json` outputs: ```json { "name": "local/test-skill", "namespace": "local", "short_name": "test-skill", "description": "...", "config_path": "...", "tool_refs": [...], "is_new": true } ``` The root cause is in `src/cleveragents/cli/commands/skill.py`. Each command calls `format_output()` directly without an envelope wrapper. Example from `add()`: ```python data = _skill_spec_dict(skill, service) data["is_new"] = is_new typer.echo(format_output(data, fmt)) # BUG: no envelope wrapper ``` **All six affected commands:** - `agents skill add --format json` — outputs raw skill dict - `agents skill remove --format json` — outputs `{"name": ..., "removed": true, "tools_removed": N}` - `agents skill list --format json` — outputs raw list of skill dicts - `agents skill show --format json` — outputs raw skill dict - `agents skill tools --format json` — outputs raw tools list - `agents skill refresh --format json` — outputs raw refresh result dict ## Expected Behavior Each `agents skill` subcommand must wrap its output data in the spec-required envelope before calling `format_output()`. Per spec §agents skill add, the correct JSON output is: ```json { "command": "skill add --config ./skills/devops-toolkit.yaml", "status": "ok", "exit_code": 0, "data": { "skill": { "name": "local/devops-toolkit", "..." : "..." }, "includes": [...], "tool_sources": [...], "total_tools": 23, "mcp_servers": [...] }, "timing": { "duration_ms": 150 }, "messages": [{ "level": "ok", "text": "Skill registered with 23 tools" }] } ``` The fix pattern for each command is: ```python envelope = { "command": f"skill add --config {config}", "status": "ok", "exit_code": 0, "data": { "skill": skill_data, ... }, "timing": {"duration_ms": elapsed_ms}, "messages": [{"level": "ok", "text": "Skill registered with N tools"}] } typer.echo(format_output(envelope, fmt)) ``` ## Acceptance Criteria - [ ] `agents skill add --format json` output matches the spec envelope structure with `command`, `status`, `exit_code`, `data`, `timing`, and `messages` keys - [ ] `agents skill remove --format json` output is wrapped in the spec envelope - [ ] `agents skill list --format json` output is wrapped in the spec envelope (skill list in `data`) - [ ] `agents skill show --format json` output is wrapped in the spec envelope - [ ] `agents skill tools --format json` output is wrapped in the spec envelope (tools list in `data`) - [ ] `agents skill refresh --format json` output is wrapped in the spec envelope - [ ] All envelope fields (`command`, `status`, `exit_code`, `data`, `timing`, `messages`) are present and correctly typed for all six commands - [ ] Non-JSON output formats (table, plain, rich) are unaffected - [ ] All existing Behave scenarios for `agents skill` commands continue to pass ## Subtasks - [ ] Audit `src/cleveragents/cli/commands/skill.py` — document the exact raw output shape for each of the six subcommands - [ ] Implement envelope wrapping for `skill add` — include `data.skill`, `data.includes`, `data.tool_sources`, `data.total_tools`, `data.mcp_servers` - [ ] Implement envelope wrapping for `skill remove` — include `data.name`, `data.removed`, `data.tools_removed` - [ ] Implement envelope wrapping for `skill list` — include `data.skills` (list), `data.total` - [ ] Implement envelope wrapping for `skill show` — include `data.skill` - [ ] Implement envelope wrapping for `skill tools` — include `data.tools` (list), `data.total` - [ ] Implement envelope wrapping for `skill refresh` — include `data.refreshed`, `data.changes` - [ ] Tests (Behave): Add/update scenarios asserting envelope structure for each subcommand's JSON output - [ ] Tests (Robot): Add integration tests verifying `--format json` envelope conformance for all six commands - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - All six `agents skill` subcommands produce spec-conformant JSON envelopes when invoked with `--format json`. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(cli): wrap skill command JSON output in spec-required envelope`), followed by a blank line, then additional lines providing relevant implementation details. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/skill-commands-json-envelope`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-03 03:56:58 +00:00

freemo added a new dependency

2026-04-03 03:57:49 +00:00

#936 Epic: Output Rendering Pipeline Integration — Wire Commands to OutputSession Framework

freemo referenced this issue

2026-04-03 03:57:57 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo commented

2026-04-03 03:58:27 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium (confirmed)
Milestone: v3.7.0 (confirmed — part of the Output Rendering Pipeline Epic #936)
MoSCoW: Should Have — All six agents skill subcommands produce raw JSON without the spec-required envelope. This is a systematic spec-alignment gap affecting all programmatic consumers of skill command output. Important for API contract compliance.
Parent Epic: #936 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) - **Milestone**: v3.7.0 (confirmed — part of the Output Rendering Pipeline Epic #936) - **MoSCoW**: Should Have — All six `agents skill` subcommands produce raw JSON without the spec-required envelope. This is a systematic spec-alignment gap affecting all programmatic consumers of skill command output. Important for API contract compliance. - **Parent Epic**: #936 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-03 03:58:31 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:01:23 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 701 — v3.7.0 #2093

freemo referenced this issue

2026-04-03 04:11:31 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 711 — v3.7.0 #2115

freemo referenced this issue

2026-04-03 04:22:00 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023