UAT: JSON/YAML error responses not structured per spec — errors output plain text to stderr instead of structured JSON envelope #5197

New issue

Open

opened 2026-04-09 03:07:32 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-09 03:07:32 +00:00

Owner

Bug Report

Feature Area: CLI Commands — JSON/YAML error response format
Severity: High — when --format json is active and a command fails, the error is NOT returned as a structured JSON envelope; programmatic consumers cannot parse errors
Source: src/cleveragents/cli/commands/project.py, src/cleveragents/cli/main.py (exception handlers)

What Was Tested

Runtime testing of error conditions with --format json active, examining whether errors are returned as structured JSON.

Expected Behavior (from spec §Output Rendering Framework §Error Handling)

The spec defines that errors in json/yaml formats must produce a structured error envelope:

{
  "command": "project show",
  "status": "error",
  "exit_code": 1,
  "error": {
    "code": "NOT_FOUND",
    "message": "Project 'local/nonexistent' not found",
    "detail": "No project with name 'local/nonexistent' exists. Run 'agents project list' to see available projects.",
    "suggestions": [
      "agents project list",
      "agents project create local/nonexistent"
    ]
  },
  "timing": { "duration_ms": 5 }
}

Actual Behavior

When a command fails with --format json, the error is output as plain text to stderr (not JSON to stdout):

$ agents project show nonexistent/project --format json
Project not found: nonexistent/project
(exit code: 1)

The output is:

Plain text (not JSON)
Goes to stderr (not stdout)
Has no error.code, error.detail, or error.suggestions fields
Does not follow the envelope structure

Root Cause

The exception handlers in command files (e.g., project.py) use console.print(...) to output error messages directly to the Rich console, bypassing the format_output function entirely. There is no code path that checks the active format and produces a structured JSON error response.

Example from project.py:

except ProjectNotFoundError as exc:
    console.print(f"[red]Project not found:[/red] {exc}")
    raise typer.Exit(1) from exc

This always outputs Rich markup text, regardless of whether --format json was requested.

Steps to Reproduce

agents project show nonexistent/project --format json
# Expected: {"command": "project show", "status": "error", "exit_code": 1, "error": {...}}
# Actual:   "Project not found: nonexistent/project" (plain text on stderr)

agents session show NONEXISTENT --format json
# Expected: structured JSON error envelope
# Actual:   plain text error on stderr

Impact

CI/CD scripts that use --format json and parse error responses will fail to parse the output
Error handling in automation pipelines is broken for all commands
The spec's documented error format is completely unimplemented

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** CLI Commands — JSON/YAML error response format **Severity:** High — when `--format json` is active and a command fails, the error is NOT returned as a structured JSON envelope; programmatic consumers cannot parse errors **Source:** `src/cleveragents/cli/commands/project.py`, `src/cleveragents/cli/main.py` (exception handlers) --- ## What Was Tested Runtime testing of error conditions with `--format json` active, examining whether errors are returned as structured JSON. ## Expected Behavior (from spec §Output Rendering Framework §Error Handling) The spec defines that errors in `json`/`yaml` formats must produce a structured error envelope: ```json { "command": "project show", "status": "error", "exit_code": 1, "error": { "code": "NOT_FOUND", "message": "Project 'local/nonexistent' not found", "detail": "No project with name 'local/nonexistent' exists. Run 'agents project list' to see available projects.", "suggestions": [ "agents project list", "agents project create local/nonexistent" ] }, "timing": { "duration_ms": 5 } } ``` ## Actual Behavior When a command fails with `--format json`, the error is output as **plain text to stderr** (not JSON to stdout): ``` $ agents project show nonexistent/project --format json Project not found: nonexistent/project (exit code: 1) ``` The output is: - Plain text (not JSON) - Goes to stderr (not stdout) - Has no `error.code`, `error.detail`, or `error.suggestions` fields - Does not follow the envelope structure ## Root Cause The exception handlers in command files (e.g., `project.py`) use `console.print(...)` to output error messages directly to the Rich console, bypassing the `format_output` function entirely. There is no code path that checks the active format and produces a structured JSON error response. Example from `project.py`: ```python except ProjectNotFoundError as exc: console.print(f"[red]Project not found:[/red] {exc}") raise typer.Exit(1) from exc ``` This always outputs Rich markup text, regardless of whether `--format json` was requested. ## Steps to Reproduce ```bash agents project show nonexistent/project --format json # Expected: {"command": "project show", "status": "error", "exit_code": 1, "error": {...}} # Actual: "Project not found: nonexistent/project" (plain text on stderr) agents session show NONEXISTENT --format json # Expected: structured JSON error envelope # Actual: plain text error on stderr ``` ## Impact - CI/CD scripts that use `--format json` and parse error responses will fail to parse the output - Error handling in automation pipelines is broken for all commands - The spec's documented error format is completely unimplemented --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 03:07:35 +00:00

HAL9000 referenced this issue

2026-04-09 03:08:46 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 4) #5160

HAL9000 referenced this issue

2026-04-09 03:09:06 +00:00