UAT: `agents plan tree` JSON output missing spec-required envelope (command, status, exit_code, data, timing, messages) #1417

New issue

Open

opened 2026-04-02 17:45:45 +00:00 by freemo · 0 comments

freemo commented

2026-04-02 17:45:45 +00:00

Owner

Metadata

Branch: fix/m4-plan-tree-json-envelope
Commit Message: fix(cli): wrap plan tree JSON output in spec-required envelope
Milestone: v3.3.0
Parent Epic: #394

Background and context

During UAT of the agents plan tree command, the JSON output format was tested against the specification (docs/specification.md lines 14420–14479). The implementation in src/cleveragents/cli/commands/plan.py (tree_decisions_cmd, line 3661) emits a raw list of tree nodes, but the spec mandates a structured envelope with top-level command, status, exit_code, data, timing, and messages fields. Additionally, tree node fields diverge from the spec: the implementation uses separate question/chosen fields where the spec requires a single description field.

This is a spec-compliance bug in the Decision Framework CLI surface (Epic #394), discovered via UAT.

Current behavior (for bugs)

Running agents plan tree <PLAN_ID> --format json (or calling format_output(tree_data, 'json') directly) produces a raw list:

[
  {
    "decision_id": "...",
    "type": "prompt_definition",
    "sequence": 0,
    "question": "What is the plan?",
    "chosen": "Increase test coverage to 85%",
    "confidence": null,
    "superseded": false,
    "children": [...]
  }
]

Missing from the output:

command, status, exit_code envelope fields
timing field (started, duration_ms)
messages field (should be ["Decision tree rendered"])
data.plan_id field
data.summary section (nodes, depth, child_plans, invariants, superseded count)
data.child_plans section (list of child plans with id, phase, state)
data.decision_ids section (labeled decision IDs for correction)
Tree nodes use question/chosen instead of spec's description field

Reproduction steps:

from cleveragents.application.services.decision_service import DecisionService
from cleveragents.domain.models.core.decision import DecisionType
from cleveragents.cli.commands.plan import build_decision_tree
from cleveragents.cli.formatting import format_output

svc = DecisionService()
plan_id = '01HXM8C2ZK4Q7C2B3F2R4VYV6J'
d1 = svc.record_decision(
    plan_id=plan_id,
    decision_type=DecisionType.PROMPT_DEFINITION,
    question='What is the plan?',
    chosen_option='Increase test coverage to 85%',
)
decisions = svc.list_decisions(plan_id)
tree_data = build_decision_tree(decisions)
print(format_output(tree_data, 'json'))
# Outputs raw list, not spec-required envelope

Expected behavior

Per docs/specification.md lines 14420–14479, the JSON output must be wrapped in the standard CLI envelope:

{
  "command": "plan tree",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "plan_id": "01HXM8C2ZK4Q7C2B3F2R4VYV6J",
    "tree": {
      "type": "prompt_definition",
      "description": "Increase test coverage to 85%",
      "children": []
    },
    "summary": {
      "nodes": 9,
      "depth": 3,
      "child_plans": "2+",
      "invariants": 2,
      "superseded": 0
    },
    "child_plans": [
      { "id": "01HXM9F1A", "phase": "execute", "state": "processing" }
    ],
    "decision_ids": {
      "root": "...",
      "invariant_1": "...",
      "strategy": "..."
    }
  },
  "timing": {
    "started": "2026-02-08T12:58:00Z",
    "duration_ms": 85
  },
  "messages": ["Decision tree rendered"]
}

Tree nodes must use a description field (combining question + chosen option) rather than separate question/chosen fields.

Acceptance criteria

agents plan tree <PLAN_ID> --format json output is wrapped in the standard CLI envelope with command, status, exit_code, data, timing, and messages top-level keys
data.plan_id is present and matches the requested plan ID
data.tree contains the root node with a description field (not question/chosen)
data.summary is present with nodes, depth, child_plans, invariants, and superseded counts
data.child_plans is present (list of child plans with id, phase, state)
data.decision_ids is present (labeled decision IDs for targeted correction)
timing.started and timing.duration_ms are present
messages contains ["Decision tree rendered"]
All existing plan tree table/YAML output formats are unaffected
All nox stages pass
Coverage >= 97%

Supporting information

Spec reference: docs/specification.md lines 14420–14479
Implementation files:
- src/cleveragents/cli/commands/plan.py — tree_decisions_cmd (line 3661), build_decision_tree (line 3585)
Parent Epic: #394 Epic: Decision Framework
Legendary: #375 Legendary: Execution Pipeline & Decisions
Related closed issue: #174 feat(cli): add plan explain and decision tree outputs

Subtasks

Update build_decision_tree (line 3585) to return a dict with plan_id, tree, summary, child_plans, and decision_ids keys instead of a raw list
Rename tree node fields: replace question/chosen with a single description field per spec
Update tree_decisions_cmd (line 3661) to wrap the output in the standard CLI envelope (command, status, exit_code, data, timing, messages)
Populate data.summary with computed counts (nodes, depth, child_plans, invariants, superseded)
Populate data.child_plans with child plan metadata
Populate data.decision_ids with labeled decision IDs
Add/update timing capture around the tree-build operation
Tests (Behave): Add/update scenarios asserting the full envelope structure in JSON output
Tests (Robot): Add/update Robot CLI test asserting envelope fields are present in plan tree --format json
Update docs/reference/plan_cli.md with corrected JSON output example
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass
Coverage >= 97%

## Metadata - **Branch**: `fix/m4-plan-tree-json-envelope` - **Commit Message**: `fix(cli): wrap plan tree JSON output in spec-required envelope` - **Milestone**: v3.3.0 - **Parent Epic**: #394 ## Background and context During UAT of the `agents plan tree` command, the JSON output format was tested against the specification (`docs/specification.md` lines 14420–14479). The implementation in `src/cleveragents/cli/commands/plan.py` (`tree_decisions_cmd`, line 3661) emits a raw list of tree nodes, but the spec mandates a structured envelope with top-level `command`, `status`, `exit_code`, `data`, `timing`, and `messages` fields. Additionally, tree node fields diverge from the spec: the implementation uses separate `question`/`chosen` fields where the spec requires a single `description` field. This is a spec-compliance bug in the Decision Framework CLI surface (Epic #394), discovered via UAT. ## Current behavior (for bugs) Running `agents plan tree <PLAN_ID> --format json` (or calling `format_output(tree_data, 'json')` directly) produces a raw list: ```json [ { "decision_id": "...", "type": "prompt_definition", "sequence": 0, "question": "What is the plan?", "chosen": "Increase test coverage to 85%", "confidence": null, "superseded": false, "children": [...] } ] ``` **Missing from the output:** 1. `command`, `status`, `exit_code` envelope fields 2. `timing` field (`started`, `duration_ms`) 3. `messages` field (should be `["Decision tree rendered"]`) 4. `data.plan_id` field 5. `data.summary` section (`nodes`, `depth`, `child_plans`, `invariants`, `superseded` count) 6. `data.child_plans` section (list of child plans with `id`, `phase`, `state`) 7. `data.decision_ids` section (labeled decision IDs for correction) 8. Tree nodes use `question`/`chosen` instead of spec's `description` field **Reproduction steps:** ```python from cleveragents.application.services.decision_service import DecisionService from cleveragents.domain.models.core.decision import DecisionType from cleveragents.cli.commands.plan import build_decision_tree from cleveragents.cli.formatting import format_output svc = DecisionService() plan_id = '01HXM8C2ZK4Q7C2B3F2R4VYV6J' d1 = svc.record_decision( plan_id=plan_id, decision_type=DecisionType.PROMPT_DEFINITION, question='What is the plan?', chosen_option='Increase test coverage to 85%', ) decisions = svc.list_decisions(plan_id) tree_data = build_decision_tree(decisions) print(format_output(tree_data, 'json')) # Outputs raw list, not spec-required envelope ``` ## Expected behavior Per `docs/specification.md` lines 14420–14479, the JSON output must be wrapped in the standard CLI envelope: ```json { "command": "plan tree", "status": "ok", "exit_code": 0, "data": { "plan_id": "01HXM8C2ZK4Q7C2B3F2R4VYV6J", "tree": { "type": "prompt_definition", "description": "Increase test coverage to 85%", "children": [] }, "summary": { "nodes": 9, "depth": 3, "child_plans": "2+", "invariants": 2, "superseded": 0 }, "child_plans": [ { "id": "01HXM9F1A", "phase": "execute", "state": "processing" } ], "decision_ids": { "root": "...", "invariant_1": "...", "strategy": "..." } }, "timing": { "started": "2026-02-08T12:58:00Z", "duration_ms": 85 }, "messages": ["Decision tree rendered"] } ``` Tree nodes must use a `description` field (combining question + chosen option) rather than separate `question`/`chosen` fields. ## Acceptance criteria - [ ] `agents plan tree <PLAN_ID> --format json` output is wrapped in the standard CLI envelope with `command`, `status`, `exit_code`, `data`, `timing`, and `messages` top-level keys - [ ] `data.plan_id` is present and matches the requested plan ID - [ ] `data.tree` contains the root node with a `description` field (not `question`/`chosen`) - [ ] `data.summary` is present with `nodes`, `depth`, `child_plans`, `invariants`, and `superseded` counts - [ ] `data.child_plans` is present (list of child plans with `id`, `phase`, `state`) - [ ] `data.decision_ids` is present (labeled decision IDs for targeted correction) - [ ] `timing.started` and `timing.duration_ms` are present - [ ] `messages` contains `["Decision tree rendered"]` - [ ] All existing `plan tree` table/YAML output formats are unaffected - [ ] All nox stages pass - [ ] Coverage >= 97% ## Supporting information - **Spec reference**: `docs/specification.md` lines 14420–14479 - **Implementation files**: - `src/cleveragents/cli/commands/plan.py` — `tree_decisions_cmd` (line 3661), `build_decision_tree` (line 3585) - **Parent Epic**: #394 Epic: Decision Framework - **Legendary**: #375 Legendary: Execution Pipeline & Decisions - **Related closed issue**: #174 `feat(cli): add plan explain and decision tree outputs` ## Subtasks - [ ] Update `build_decision_tree` (line 3585) to return a dict with `plan_id`, `tree`, `summary`, `child_plans`, and `decision_ids` keys instead of a raw list - [ ] Rename tree node fields: replace `question`/`chosen` with a single `description` field per spec - [ ] Update `tree_decisions_cmd` (line 3661) to wrap the output in the standard CLI envelope (`command`, `status`, `exit_code`, `data`, `timing`, `messages`) - [ ] Populate `data.summary` with computed counts (`nodes`, `depth`, `child_plans`, `invariants`, `superseded`) - [ ] Populate `data.child_plans` with child plan metadata - [ ] Populate `data.decision_ids` with labeled decision IDs - [ ] Add/update `timing` capture around the tree-build operation - [ ] Tests (Behave): Add/update scenarios asserting the full envelope structure in JSON output - [ ] Tests (Robot): Add/update Robot CLI test asserting envelope fields are present in `plan tree --format json` - [ ] Update `docs/reference/plan_cli.md` with corrected JSON output example - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass - Coverage >= 97%

freemo added the

labels

2026-04-02 17:46:13 +00:00

freemo referenced this issue