[BUG] agents plan artifacts returns empty changeset despite files being generated during execution #9184

New issue

Open

opened 2026-04-14 09:40:47 +00:00 by hurui200320 · 2 comments

hurui200320 commented

2026-04-14 09:40:47 +00:00

Member

Metadata

Branch: fix/plan-artifacts-empty-changeset-diff
Commit Message: fix(plan): compute plan artifacts diff from pre/post-execution commits instead of working tree
Milestone: v3.2.0
Parent Epic: TBD — requires manual linking

Description

Background

The agents plan artifacts <PLAN_ID> command is intended to list all files created or modified during the Execute phase of a plan, enabling users to review generated artifacts before applying them. After a successful agents plan execute run, the artifacts command exits 0 but reports an empty changeset, making pre-apply review impossible.

Current Behavior

After a successful agents plan execute run that generated 9 files in a git worktree sandbox, running agents plan artifacts <PLAN_ID> exits 0 but returns:

changeset_summary: { total: 0, files_changed: [] }

The root cause is that the diff calculation compares HEAD against the current working tree, which is clean after execution completes — producing a zero-diff result.

Reproduction steps:

agents resource add git-checkout local/rune-project --path /workspace/rune (repo must have at least 1 commit)
agents project create --resource local/rune-project local/rune-encoder
agents action create --config /workspace/build-rune-encoder.yaml (action with strategy_actor and execution_actor set)
agents plan use local/build-rune-encoder local/rune-encoder --automation-profile trusted
agents plan execute <PLAN_ID> — wait for completion
agents plan artifacts <PLAN_ID> → returns files_changed: [] despite 9 files in sandbox

Expected Behavior

Per spec §agents plan artifacts: "List all artifacts created during execution."

The command should compare the pre-execution commit (initial state of the sandbox branch) against the post-execution commit (state after the execution actor ran), and return a table of all files created/modified:

Artifacts
  Path                                    Size    Action
  --------------------------------------- ------- -------
  src/main/kotlin/RuneBase64.kt           ~2 KB   created
  build.gradle.kts                        ~1 KB   created
  settings.gradle.kts                     ~0.2 KB created
  ...
[OK] 9 artifacts listed

Acceptance Criteria

agents plan artifacts <PLAN_ID> returns a non-empty changeset listing all files created/modified during the Execute phase
The diff is computed between the pre-execution commit and the post-execution commit on the sandbox branch (not HEAD vs working tree)
The command output matches the spec-required Artifacts table format
agents plan apply --yes <PLAN_ID> and agents plan artifacts <PLAN_ID> agree on the set of changed files
The fix does not regress agents plan apply behavior

Supporting Information

Verified workaround: agents plan apply --yes <PLAN_ID> correctly identifies and applies the same 9 files, confirming the apply-side diff logic uses the correct commit comparison. Only plan artifacts is broken.
Related issue #9084: Covers missing validation_summary and apply_summary metadata fields in plan artifacts JSON output — that is a separate, distinct bug from the empty changeset reported here.
Related issue #4312: TDD tracking ticket for a failing Robot test Plan Artifacts Shows Changeset Metadata — the fix here should resolve that test failure.

Subtasks

Identify where plan artifacts computes its diff (locate the diff/changeset calculation code path)
Determine how the pre-execution and post-execution commit SHAs are stored on the Plan model
Fix the diff calculation to compare pre-execution commit vs post-execution commit (matching the mechanism used by plan apply)
Tests (Behave): Add/update BDD scenarios covering plan artifacts returning a non-empty changeset after execution
Tests (Robot): Verify robot/plan_diff_artifacts.robot — Plan Artifacts Shows Changeset Metadata passes and remove @tdd_expected_fail tag (see #4312)
Run nox (all default sessions), fix any errors
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata - **Branch**: `fix/plan-artifacts-empty-changeset-diff` - **Commit Message**: `fix(plan): compute plan artifacts diff from pre/post-execution commits instead of working tree` - **Milestone**: v3.2.0 - **Parent Epic**: TBD — requires manual linking ## Description ### Background The `agents plan artifacts <PLAN_ID>` command is intended to list all files created or modified during the Execute phase of a plan, enabling users to review generated artifacts before applying them. After a successful `agents plan execute` run, the artifacts command exits 0 but reports an empty changeset, making pre-apply review impossible. ### Current Behavior After a successful `agents plan execute` run that generated 9 files in a git worktree sandbox, running `agents plan artifacts <PLAN_ID>` exits 0 but returns: ``` changeset_summary: { total: 0, files_changed: [] } ``` The root cause is that the diff calculation compares `HEAD` against the current working tree, which is clean after execution completes — producing a zero-diff result. **Reproduction steps:** 1. `agents resource add git-checkout local/rune-project --path /workspace/rune` (repo must have at least 1 commit) 2. `agents project create --resource local/rune-project local/rune-encoder` 3. `agents action create --config /workspace/build-rune-encoder.yaml` (action with `strategy_actor` and `execution_actor` set) 4. `agents plan use local/build-rune-encoder local/rune-encoder --automation-profile trusted` 5. `agents plan execute <PLAN_ID>` — wait for completion 6. `agents plan artifacts <PLAN_ID>` → returns `files_changed: []` despite 9 files in sandbox ### Expected Behavior Per spec §`agents plan artifacts`: *"List all artifacts created during execution."* The command should compare the pre-execution commit (initial state of the sandbox branch) against the post-execution commit (state after the execution actor ran), and return a table of all files created/modified: ``` Artifacts Path Size Action --------------------------------------- ------- ------- src/main/kotlin/RuneBase64.kt ~2 KB created build.gradle.kts ~1 KB created settings.gradle.kts ~0.2 KB created ... [OK] 9 artifacts listed ``` ### Acceptance Criteria - `agents plan artifacts <PLAN_ID>` returns a non-empty changeset listing all files created/modified during the Execute phase - The diff is computed between the pre-execution commit and the post-execution commit on the sandbox branch (not `HEAD` vs working tree) - The command output matches the spec-required Artifacts table format - `agents plan apply --yes <PLAN_ID>` and `agents plan artifacts <PLAN_ID>` agree on the set of changed files - The fix does not regress `agents plan apply` behavior ### Supporting Information - **Verified workaround:** `agents plan apply --yes <PLAN_ID>` correctly identifies and applies the same 9 files, confirming the apply-side diff logic uses the correct commit comparison. Only `plan artifacts` is broken. - **Related issue #9084:** Covers missing `validation_summary` and `apply_summary` metadata fields in `plan artifacts` JSON output — that is a separate, distinct bug from the empty changeset reported here. - **Related issue #4312:** TDD tracking ticket for a failing Robot test `Plan Artifacts Shows Changeset Metadata` — the fix here should resolve that test failure. ## Subtasks - [ ] Identify where `plan artifacts` computes its diff (locate the diff/changeset calculation code path) - [ ] Determine how the pre-execution and post-execution commit SHAs are stored on the Plan model - [ ] Fix the diff calculation to compare pre-execution commit vs post-execution commit (matching the mechanism used by `plan apply`) - [ ] Tests (Behave): Add/update BDD scenarios covering `plan artifacts` returning a non-empty changeset after execution - [ ] Tests (Robot): Verify `robot/plan_diff_artifacts.robot` — `Plan Artifacts Shows Changeset Metadata` passes and remove `@tdd_expected_fail` tag (see #4312) - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

hurui200320 added the

Rows
Columns