cleveragents/cleveragents-core

Fork 3

UAT: `agents plan correct --dry-run` uses `analyze_impact` instead of `generate_dry_run_report` — missing warnings and recompute time estimate #4915

New issue

Open

opened 2026-04-08 20:19:52 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-08 20:19:52 +00:00

Owner

Bug Report

Feature Area: Correction Model — agents plan correct --dry-run output
Severity: Medium (dry-run output is incomplete vs spec)
Found by: UAT tester, code analysis
Spec reference: docs/specification.md §agents plan correct --dry-run (lines ~15200–15320)

What Was Tested

The agents plan correct --dry-run command's output was compared against the specification.

Expected Behavior (from spec)

The spec shows a "Dry Run — Correction Preview" output that includes:

╭─ Dry Run — Correction Preview ─────────────────────────────────────╮
│ Mode: revert                                                        │
│ Target: 01HXM9A1C2Q7W3R5G8Z0P4Q1X9                                 │
│ Affected Decisions: 3                                               │
│ Estimated Recompute Time: 6s                                        │
│ Warnings:                                                           │
│   - Revert will invalidate 3 decisions and archive artifacts        │
│   - Medium risk: 4-10 decisions affected                            │
╰─────────────────────────────────────────────────────────────────────╯

To execute this correction, remove --dry-run and add --yes

The CorrectionService.generate_dry_run_report() method exists and produces a CorrectionDryRunReport with:

warnings: list of human-readable warnings
estimated_recompute_time_seconds: estimated wall-clock seconds
decisions_to_invalidate: decisions that would be marked invalid
impact.excluded_decisions: decisions NOT affected (preserved)

Actual Behavior

The correct_decision CLI function (lines 3267–3304 of plan.py) calls svc.analyze_impact() directly for dry-run, bypassing generate_dry_run_report():

if dry_run:
    # Analyze and display impact
    impact = svc.analyze_impact(
        request.correction_id,
        decision_tree=decision_tree,
        influence_edges=influence_edges,
    )
    # ... shows "Correction Impact (Dry Run)" panel

The output panel shows:

Correction ID, Mode, Target Decision, Guidance
Affected Decisions, Affected Files, Risk Level, Estimated Cost

Missing from dry-run output:

warnings list (e.g., "High risk: more than 10 decisions affected", "Tier 0: root decision targeted")
estimated_recompute_time_seconds (estimated wall-clock time to recompute)
decisions_to_invalidate (decisions that would be marked invalid)
excluded_decisions (decisions NOT affected — preserved)
The "To execute this correction, remove --dry-run and add --yes" hint

Additionally, calling analyze_impact directly for dry-run mutates the correction status (transitions from PENDING to ANALYZING), whereas generate_dry_run_report is designed to preserve the original status.

Code Location

src/cleveragents/cli/commands/plan.py, lines 3267–3304 (correct_decision function, if dry_run: branch)
src/cleveragents/application/services/correction_service.py, lines 312–405 (generate_dry_run_report — exists but unused by CLI)

Impact

Users cannot see warnings before executing a correction (e.g., "High risk", "Tier 0: root targeted")
Users cannot see estimated recompute time in dry-run preview
The "To execute this correction, remove --dry-run and add --yes" hint is absent
Dry-run incorrectly mutates correction status (PENDING → ANALYZING) instead of preserving it

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** Correction Model — `agents plan correct --dry-run` output **Severity:** Medium (dry-run output is incomplete vs spec) **Found by:** UAT tester, code analysis **Spec reference:** `docs/specification.md` §`agents plan correct --dry-run` (lines ~15200–15320) --- ### What Was Tested The `agents plan correct --dry-run` command's output was compared against the specification. ### Expected Behavior (from spec) The spec shows a "Dry Run — Correction Preview" output that includes: ``` ╭─ Dry Run — Correction Preview ─────────────────────────────────────╮ │ Mode: revert │ │ Target: 01HXM9A1C2Q7W3R5G8Z0P4Q1X9 │ │ Affected Decisions: 3 │ │ Estimated Recompute Time: 6s │ │ Warnings: │ │ - Revert will invalidate 3 decisions and archive artifacts │ │ - Medium risk: 4-10 decisions affected │ ╰─────────────────────────────────────────────────────────────────────╯ To execute this correction, remove --dry-run and add --yes ``` The `CorrectionService.generate_dry_run_report()` method exists and produces a `CorrectionDryRunReport` with: - `warnings`: list of human-readable warnings - `estimated_recompute_time_seconds`: estimated wall-clock seconds - `decisions_to_invalidate`: decisions that would be marked invalid - `impact.excluded_decisions`: decisions NOT affected (preserved) ### Actual Behavior The `correct_decision` CLI function (lines 3267–3304 of `plan.py`) calls `svc.analyze_impact()` directly for dry-run, bypassing `generate_dry_run_report()`: ```python if dry_run: # Analyze and display impact impact = svc.analyze_impact( request.correction_id, decision_tree=decision_tree, influence_edges=influence_edges, ) # ... shows "Correction Impact (Dry Run)" panel ``` The output panel shows: - Correction ID, Mode, Target Decision, Guidance - Affected Decisions, Affected Files, Risk Level, Estimated Cost **Missing from dry-run output:** - `warnings` list (e.g., "High risk: more than 10 decisions affected", "Tier 0: root decision targeted") - `estimated_recompute_time_seconds` (estimated wall-clock time to recompute) - `decisions_to_invalidate` (decisions that would be marked invalid) - `excluded_decisions` (decisions NOT affected — preserved) - The "To execute this correction, remove --dry-run and add --yes" hint Additionally, calling `analyze_impact` directly for dry-run **mutates the correction status** (transitions from PENDING to ANALYZING), whereas `generate_dry_run_report` is designed to preserve the original status. ### Code Location - `src/cleveragents/cli/commands/plan.py`, lines 3267–3304 (`correct_decision` function, `if dry_run:` branch) - `src/cleveragents/application/services/correction_service.py`, lines 312–405 (`generate_dry_run_report` — exists but unused by CLI) ### Impact - Users cannot see warnings before executing a correction (e.g., "High risk", "Tier 0: root targeted") - Users cannot see estimated recompute time in dry-run preview - The "To execute this correction, remove --dry-run and add --yes" hint is absent - Dry-run incorrectly mutates correction status (PENDING → ANALYZING) instead of preserving it --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-08 20:20:23 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 referenced this issue

2026-04-08 20:21:39 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 referenced this issue

2026-04-08 20:22:25 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 referenced this issue

2026-04-08 22:58:56 +00:00

Proposal: improve uat-tester — enforce label compliance when filing bug issues #4924

HAL9000 added the

labels

2026-04-08 22:59:08 +00:00

HAL9000 added this to the v3.3.0 milestone

2026-04-08 23:00:08 +00:00

HAL9000 commented

2026-04-08 23:00:21 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — dry-run output is incomplete vs spec; users cannot see warnings before executing corrections, but this does not block core correction functionality
Milestone: v3.3.0 — Corrections + Subplans + Checkpoints milestone; this is a correction model bug
Story Points: 3 — M — fix requires wiring generate_dry_run_report() to CLI instead of analyze_impact(), plus output panel update
MoSCoW: Should Have — the generate_dry_run_report() method already exists; this is a wiring bug that degrades UX but doesn't block corrections from working
Parent Epic: Correction Model epic (v3.3.0)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — dry-run output is incomplete vs spec; users cannot see warnings before executing corrections, but this does not block core correction functionality - **Milestone**: v3.3.0 — Corrections + Subplans + Checkpoints milestone; this is a correction model bug - **Story Points**: 3 — M — fix requires wiring `generate_dry_run_report()` to CLI instead of `analyze_impact()`, plus output panel update - **MoSCoW**: Should Have — the `generate_dry_run_report()` method already exists; this is a wiring bug that degrades UX but doesn't block corrections from working - **Parent Epic**: Correction Model epic (v3.3.0) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 referenced this issue

2026-04-08 23:01:17 +00:00

Proposal: update specification — correction dry-run uses analyze_impact() instead of generate_dry_run_report() #4938

freemo referenced this issue

2026-04-08 23:38:32 +00:00

Proposal: update specification — correction dry-run uses analyze_impact() instead of generate_dry_run_report() #4938

freemo added a new dependency

2026-04-08 23:41:55 +00:00

#4959 EPIC: Correction Engine — Revert & Append Modes with plan.attempt Tracking

freemo referenced this issue

2026-04-08 23:43:03 +00:00

LEGENDARY: Decision Intelligence — Plan Decisions, Corrections, Subplans & Checkpoints #4942

HAL9000 referenced this issue

2026-04-09 00:20:50 +00:00

Proposal: update specification — correction dry-run uses analyze_impact() instead of generate_dry_run_report() #4938

HAL9000 added the

Points

label

2026-04-09 05:42:33 +00:00

HAL9000 commented

2026-04-09 05:43:00 +00:00

Author

Owner

Label compliance fix applied:

Added missing label: Points/3 (M — medium complexity)
Reason: Issue is in State/Verified but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing label: `Points/3` (M — medium complexity) - Reason: Issue is in `State/Verified` but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer