cleveragents/cleveragents-core

Fork 3

[BUG] `agents plan artifacts` does not include validation summary or apply summary in JSON output #9084

New issue

Open

opened 2026-04-14 07:22:06 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-14 07:22:06 +00:00

Owner

Metadata

Commit Message: fix(plan): include validation_summary and apply_summary in plan artifacts JSON output
Branch: fix/plan-artifacts-missing-validation-apply-summary

Background and Context

Per the specification (docs/specification.md — §agents plan artifacts), the agents plan artifacts <PLAN_ID> command should include a validation summary (when available) and an apply summary (showing apply metadata such as files changed, validations run, and completed timestamp) in its output.

Two scenarios in features/plan_diff_artifacts.feature are currently tagged @tdd_expected_fail under @tdd_issue_4253, confirming this is a known missing implementation:

@tdd_issue @tdd_issue_4253 @tdd_expected_fail
Scenario: Plan artifacts shows validation results when available
  Given a plan with a changeset and validation summary
  When I request the artifacts in JSON format
  Then the artifacts JSON should contain validation summary

@tdd_issue @tdd_issue_4253 @tdd_expected_fail
Scenario: Artifacts include apply summary from metadata
  Given a plan with a changeset and apply summary metadata
  When I request the artifacts in JSON format
  Then the artifacts JSON should contain apply summary

The plan_artifacts command in src/cleveragents/cli/commands/plan.py does not include validation_summary or apply_summary metadata in its JSON output, even when this data is available on the plan object.

Expected Behavior

Per the specification, agents plan artifacts <PLAN_ID> should include:

Validation summary when available — showing validation results from the plan execution
Apply summary — showing apply metadata (files changed, validations run, completed timestamp)

Acceptance Criteria

agents plan artifacts <PLAN_ID> --format json includes validation_summary when plan has validation data
agents plan artifacts <PLAN_ID> --format json includes apply_summary when plan has apply metadata
Both @tdd_expected_fail scenarios in plan_diff_artifacts.feature (tagged @tdd_issue_4253) pass without the @tdd_expected_fail tag
No regression in existing artifacts tests
All output formats (json, yaml, plain, rich) include the new fields

Subtasks

Update plan_artifacts command in src/cleveragents/cli/commands/plan.py to include validation_summary when available on the plan
Update plan_artifacts command to include apply_summary from plan metadata when available
Add validation_summary field to artifacts JSON output structure
Add apply_summary field to artifacts JSON output structure
Remove @tdd_expected_fail tags from the two affected scenarios in features/plan_diff_artifacts.feature
Verify all output formats (json, yaml, plain, rich) include the new fields

Definition of Done

agents plan artifacts <PLAN_ID> --format json includes validation_summary when plan has validation data
agents plan artifacts <PLAN_ID> --format json includes apply_summary when plan has apply metadata
Both @tdd_expected_fail scenarios in plan_diff_artifacts.feature pass
No regression in existing artifacts tests

Automated by CleverAgents Bot
Supervisor: UAT Test Pool | Agent: uat-test-pool-supervisor

## Metadata - **Commit Message**: `fix(plan): include validation_summary and apply_summary in plan artifacts JSON output` - **Branch**: `fix/plan-artifacts-missing-validation-apply-summary` ## Background and Context Per the specification (`docs/specification.md` — §agents plan artifacts), the `agents plan artifacts <PLAN_ID>` command should include a **validation summary** (when available) and an **apply summary** (showing apply metadata such as files changed, validations run, and completed timestamp) in its output. Two scenarios in `features/plan_diff_artifacts.feature` are currently tagged `@tdd_expected_fail` under `@tdd_issue_4253`, confirming this is a known missing implementation: ```gherkin @tdd_issue @tdd_issue_4253 @tdd_expected_fail Scenario: Plan artifacts shows validation results when available Given a plan with a changeset and validation summary When I request the artifacts in JSON format Then the artifacts JSON should contain validation summary @tdd_issue @tdd_issue_4253 @tdd_expected_fail Scenario: Artifacts include apply summary from metadata Given a plan with a changeset and apply summary metadata When I request the artifacts in JSON format Then the artifacts JSON should contain apply summary ``` The `plan_artifacts` command in `src/cleveragents/cli/commands/plan.py` does not include `validation_summary` or `apply_summary` metadata in its JSON output, even when this data is available on the plan object. ## Expected Behavior Per the specification, `agents plan artifacts <PLAN_ID>` should include: 1. **Validation summary** when available — showing validation results from the plan execution 2. **Apply summary** — showing apply metadata (files changed, validations run, completed timestamp) ## Acceptance Criteria - [ ] `agents plan artifacts <PLAN_ID> --format json` includes `validation_summary` when plan has validation data - [ ] `agents plan artifacts <PLAN_ID> --format json` includes `apply_summary` when plan has apply metadata - [ ] Both `@tdd_expected_fail` scenarios in `plan_diff_artifacts.feature` (tagged `@tdd_issue_4253`) pass without the `@tdd_expected_fail` tag - [ ] No regression in existing artifacts tests - [ ] All output formats (json, yaml, plain, rich) include the new fields ## Subtasks - [ ] Update `plan_artifacts` command in `src/cleveragents/cli/commands/plan.py` to include `validation_summary` when available on the plan - [ ] Update `plan_artifacts` command to include `apply_summary` from plan metadata when available - [ ] Add `validation_summary` field to artifacts JSON output structure - [ ] Add `apply_summary` field to artifacts JSON output structure - [ ] Remove `@tdd_expected_fail` tags from the two affected scenarios in `features/plan_diff_artifacts.feature` - [ ] Verify all output formats (json, yaml, plain, rich) include the new fields ## Definition of Done - [ ] `agents plan artifacts <PLAN_ID> --format json` includes `validation_summary` when plan has validation data - [ ] `agents plan artifacts <PLAN_ID> --format json` includes `apply_summary` when plan has apply metadata - [ ] Both `@tdd_expected_fail` scenarios in `plan_diff_artifacts.feature` pass - [ ] No regression in existing artifacts tests --- **Automated by CleverAgents Bot** Supervisor: UAT Test Pool | Agent: uat-test-pool-supervisor

HAL9000 added the

labels

2026-04-14 07:25:12 +00:00

HAL9000 added this to the v3.3.0 milestone

2026-04-14 07:37:29 +00:00

HAL9000 referenced this issue

2026-04-14 07:45:02 +00:00

[AUTO-UAT-POOL] Status: UAT Test Pool Supervisor (Cycle 1) #9113

HAL9000 commented

2026-04-14 09:02:18 +00:00

Author

Owner

🔍 Triage Decision — [AUTO-OWNR-2]

Status: ✅ VERIFIED

MoSCoW: Must have
Priority: High
Milestone: v3.3.0

Reasoning: The agents plan artifacts command omits required validation summary and apply summary fields from JSON output, violating the CLI output spec. This is a spec compliance bug that must be resolved for v3.3.0.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

## 🔍 Triage Decision — [AUTO-OWNR-2] **Status:** ✅ VERIFIED **MoSCoW:** Must have **Priority:** High **Milestone:** v3.3.0 **Reasoning:** The `agents plan artifacts` command omits required validation summary and apply summary fields from JSON output, violating the CLI output spec. This is a spec compliance bug that must be resolved for v3.3.0. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

hurui200320 referenced this issue

2026-04-14 09:40:48 +00:00

[BUG] agents plan artifacts returns empty changeset despite files being generated during execution #9184

HAL9000 referenced this issue

2026-04-14 09:58:07 +00:00

[BUG] agents plan artifacts returns empty changeset despite files being generated during execution #9184

HAL9000 referenced this issue from a pull request that will close it,

2026-04-14 11:47:08 +00:00

fix(plan): include validation_summary and apply_summary in plan artifacts JSON output #9227

HAL9000 added a new dependency

2026-04-14 12:02:36 +00:00

#9227 fix(plan): include validation_summary and apply_summary in plan artifacts JSON output

HAL9000 added

and removed

labels

2026-04-14 12:56:14 +00:00

HAL9000 commented

2026-04-14 12:56:15 +00:00

Author

Owner

✅ Triage: Verified [AUTO-OWNR-1]

Valid bug: agents plan artifacts does not include validation_summary or apply_summary in JSON output. These are required fields per the spec for the artifacts command.

Assigning to v3.3.0 (Corrections + Subplans + Checkpoints) as artifact inspection is part of the correction workflow. Priority High — missing required output fields.

MoSCoW: Must Have — validation and apply summaries are required for users to make informed decisions about applying artifacts.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Triage: Verified** [AUTO-OWNR-1] Valid bug: `agents plan artifacts` does not include `validation_summary` or `apply_summary` in JSON output. These are required fields per the spec for the artifacts command. Assigning to **v3.3.0** (Corrections + Subplans + Checkpoints) as artifact inspection is part of the correction workflow. Priority **High** — missing required output fields. MoSCoW: **Must Have** — validation and apply summaries are required for users to make informed decisions about applying artifacts. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 referenced this issue

2026-04-14 12:56:58 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Pool Supervisor (Cycle 1) #9249

HAL9000 referenced this issue

2026-04-14 13:16:16 +00:00

fix(plan): include validation_summary and apply_summary in plan artifacts JSON output #9227