cleveragents/cleveragents-core

Fork 3

UAT: `agents plan apply` CLI bypasses `PlanApplyService.apply_with_validation_gate()` — validation gate is implemented but never invoked #3231

New issue

Open

opened 2026-04-05 08:06:50 +00:00 by freemo · 3 comments

freemo commented

2026-04-05 08:06:50 +00:00

Owner

Metadata

Branch: fix/plan-apply-validation-gate-wiring
Commit Message: fix(cli): wire agents plan apply through PlanApplyService.apply_with_validation_gate
Milestone: v3.5.0
Parent Epic: #397

Bug Report

Feature Area: Validation-Gated Apply (M6: Autonomy Hardening)

What was tested

The agents plan apply CLI command and the PlanApplyService.apply_with_validation_gate() method.

Expected behavior (from spec)

Per docs/specification.md §Apply Phase and the M6 acceptance criterion "Validation-gated apply (apply blocked if validations fail)": The agents plan apply command must check that all required Execute-phase validations have passed before committing changes. If any required validation failed, apply must be blocked with an actionable error message.

The PlanApplyService.apply_with_validation_gate() method (lines 482-662 of plan_apply_service.py) implements this gate correctly:

Checks validation_summary for required_failed > 0
Blocks apply and transitions plan to constrained state
Returns ApplyOutcome.CONSTRAINED with an actionable error message

Actual behavior (from code analysis)

The _lifecycle_apply_with_id() function in src/cleveragents/cli/commands/plan.py (lines 853-933) calls the lifecycle service directly, completely bypassing the validation gate:

# Lines 895-906 — NO validation gate
service = _get_lifecycle_service()
# ...
service.apply_plan(plan_id)          # Direct call — no validation check
# ...
service.start_apply(plan_id)         # Direct call — no validation check
# ...
service.complete_apply(plan_id)      # Direct call — no validation check

The _get_apply_service() function (line 2698) creates a PlanApplyService instance, but it is only used for diff and artifacts commands (lines 2753, 2786) — never for the actual apply command.

Similarly, the A2A facade's _handle_plan_apply (lines 425-433 of facade.py) calls svc.apply_plan(plan_id) directly without going through PlanApplyService.apply_with_validation_gate().

Code location

src/cleveragents/cli/commands/plan.py, lines 853-933 (_lifecycle_apply_with_id — bypasses gate)
src/cleveragents/cli/commands/plan.py, lines 2698-2705 (_get_apply_service — only used for diff/artifacts)
src/cleveragents/application/services/plan_apply_service.py, lines 482-662 (apply_with_validation_gate — never called from CLI)
src/cleveragents/a2a/facade.py, lines 425-433 (_handle_plan_apply — bypasses gate)

Steps to reproduce

Create and execute a plan that has required validation failures
Run agents plan apply <PLAN_ID>
Observe that apply succeeds even though required validations failed
The plan is applied without any validation check

Impact

Critical: The M6 acceptance criterion "Validation-gated apply (apply blocked if validations fail)" is completely non-functional
Plans with failed required validations can be applied, potentially committing broken changes
The PlanApplyService.apply_with_validation_gate() implementation is dead code — it has comprehensive BDD tests (features/apply_pipeline.feature) but is never called in production
Both the CLI and A2A facade bypass the gate

Subtasks

Replace direct service.apply_plan() calls in _lifecycle_apply_with_id() with PlanApplyService.apply_with_validation_gate()
Wire _handle_plan_apply in A2A facade to use PlanApplyService.apply_with_validation_gate()
Update CLI output to display ApplyResult fields (outcome, files_changed, validations_total)
Add integration test verifying apply is blocked when validations fail
Verify ApplyOutcome.CONSTRAINED causes non-zero exit code

Definition of Done

agents plan apply calls PlanApplyService.apply_with_validation_gate() instead of direct lifecycle service calls
Apply is blocked with an actionable error when required validations have failed
The A2A facade _handle_plan_apply also uses the validation gate
Integration test verifies the gate blocks apply on validation failure
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/plan-apply-validation-gate-wiring` - **Commit Message**: `fix(cli): wire agents plan apply through PlanApplyService.apply_with_validation_gate` - **Milestone**: v3.5.0 - **Parent Epic**: #397 ## Bug Report **Feature Area**: Validation-Gated Apply (M6: Autonomy Hardening) ### What was tested The `agents plan apply` CLI command and the `PlanApplyService.apply_with_validation_gate()` method. ### Expected behavior (from spec) Per `docs/specification.md` §Apply Phase and the M6 acceptance criterion "Validation-gated apply (apply blocked if validations fail)": The `agents plan apply` command must check that all required Execute-phase validations have passed before committing changes. If any required validation failed, apply must be blocked with an actionable error message. The `PlanApplyService.apply_with_validation_gate()` method (lines 482-662 of `plan_apply_service.py`) implements this gate correctly: - Checks `validation_summary` for `required_failed > 0` - Blocks apply and transitions plan to `constrained` state - Returns `ApplyOutcome.CONSTRAINED` with an actionable error message ### Actual behavior (from code analysis) The `_lifecycle_apply_with_id()` function in `src/cleveragents/cli/commands/plan.py` (lines 853-933) calls the lifecycle service directly, **completely bypassing the validation gate**: ```python # Lines 895-906 — NO validation gate service = _get_lifecycle_service() # ... service.apply_plan(plan_id) # Direct call — no validation check # ... service.start_apply(plan_id) # Direct call — no validation check # ... service.complete_apply(plan_id) # Direct call — no validation check ``` The `_get_apply_service()` function (line 2698) creates a `PlanApplyService` instance, but it is **only used for `diff` and `artifacts` commands** (lines 2753, 2786) — never for the actual `apply` command. Similarly, the A2A facade's `_handle_plan_apply` (lines 425-433 of `facade.py`) calls `svc.apply_plan(plan_id)` directly without going through `PlanApplyService.apply_with_validation_gate()`. ### Code location - `src/cleveragents/cli/commands/plan.py`, lines 853-933 (`_lifecycle_apply_with_id` — bypasses gate) - `src/cleveragents/cli/commands/plan.py`, lines 2698-2705 (`_get_apply_service` — only used for diff/artifacts) - `src/cleveragents/application/services/plan_apply_service.py`, lines 482-662 (`apply_with_validation_gate` — never called from CLI) - `src/cleveragents/a2a/facade.py`, lines 425-433 (`_handle_plan_apply` — bypasses gate) ### Steps to reproduce 1. Create and execute a plan that has required validation failures 2. Run `agents plan apply <PLAN_ID>` 3. Observe that apply succeeds even though required validations failed 4. The plan is applied without any validation check ### Impact - **Critical**: The M6 acceptance criterion "Validation-gated apply (apply blocked if validations fail)" is completely non-functional - Plans with failed required validations can be applied, potentially committing broken changes - The `PlanApplyService.apply_with_validation_gate()` implementation is dead code — it has comprehensive BDD tests (`features/apply_pipeline.feature`) but is never called in production - Both the CLI and A2A facade bypass the gate ## Subtasks - [ ] Replace direct `service.apply_plan()` calls in `_lifecycle_apply_with_id()` with `PlanApplyService.apply_with_validation_gate()` - [ ] Wire `_handle_plan_apply` in A2A facade to use `PlanApplyService.apply_with_validation_gate()` - [ ] Update CLI output to display `ApplyResult` fields (outcome, files_changed, validations_total) - [ ] Add integration test verifying apply is blocked when validations fail - [ ] Verify `ApplyOutcome.CONSTRAINED` causes non-zero exit code ## Definition of Done - [ ] `agents plan apply` calls `PlanApplyService.apply_with_validation_gate()` instead of direct lifecycle service calls - [ ] Apply is blocked with an actionable error when required validations have failed - [ ] The A2A facade `_handle_plan_apply` also uses the validation gate - [ ] Integration test verifies the gate blocks apply on validation failure - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.5.0 milestone

2026-04-05 08:14:14 +00:00

freemo added a new dependency

2026-04-05 08:16:19 +00:00

#397 Epic: Server & Autonomy Infrastructure

freemo referenced this issue

2026-04-05 08:19:02 +00:00

[CA-AUTO] System Watchdog — Session Tracker — watchdog-1 — 2026-04-05 #3197

freemo commented

2026-04-05 08:22:22 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Critical (confirmed — this is a fundamental acceptance criterion for v3.5.0 that is completely non-functional. The validation gate exists as dead code.)
Milestone: v3.5.0 (confirmed — "Validation-gated apply" is an explicit M6 acceptance criterion)
MoSCoW: Must Have — the v3.5.0 acceptance criteria explicitly state "validation-gated apply (apply blocked if validations fail)." Without this fix, the entire validation gate feature is dead code. This directly blocks milestone completion.
Parent Epic: #397 (Server & Autonomy Infrastructure — the apply pipeline is core autonomy infrastructure)

This is an excellent bug report with thorough code analysis showing the exact bypass path. The fix is well-scoped: wire _lifecycle_apply_with_id() through PlanApplyService.apply_with_validation_gate() instead of direct lifecycle service calls.

This should be prioritized immediately — it is one of the most impactful open issues for v3.5.0 completion.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Critical (confirmed — this is a fundamental acceptance criterion for v3.5.0 that is completely non-functional. The validation gate exists as dead code.) - **Milestone**: v3.5.0 (confirmed — "Validation-gated apply" is an explicit M6 acceptance criterion) - **MoSCoW**: Must Have — the v3.5.0 acceptance criteria explicitly state "validation-gated apply (apply blocked if validations fail)." Without this fix, the entire validation gate feature is dead code. This directly blocks milestone completion. - **Parent Epic**: #397 (Server & Autonomy Infrastructure — the apply pipeline is core autonomy infrastructure) This is an excellent bug report with thorough code analysis showing the exact bypass path. The fix is well-scoped: wire `_lifecycle_apply_with_id()` through `PlanApplyService.apply_with_validation_gate()` instead of direct lifecycle service calls. **This should be prioritized immediately** — it is one of the most impactful open issues for v3.5.0 completion. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo commented

2026-04-05 08:22:46 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified ✅
Priority: Critical — The agents plan apply CLI command bypasses the validation gate entirely. Per the specification, the Apply phase MUST validate changes before merging them into project resources. Skipping this gate means unsafe or invalid changes could be applied directly, violating the core safety model of the plan lifecycle.
Milestone: v3.5.0 (already assigned — milestone is past due)
MoSCoW: Must Have — The validation gate is a fundamental safety mechanism in the plan lifecycle. The specification's four-phase model (Action → Strategize → Execute → Apply) requires validation at the Apply phase boundary. This is a showstopper for v3.5.0.

This is the 4th Critical Must Have bug in v3.5.0. The milestone is severely behind with critical safety and correctness issues.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified ✅ - **Priority**: Critical — The `agents plan apply` CLI command bypasses the validation gate entirely. Per the specification, the Apply phase MUST validate changes before merging them into project resources. Skipping this gate means unsafe or invalid changes could be applied directly, violating the core safety model of the plan lifecycle. - **Milestone**: v3.5.0 (already assigned — milestone is past due) - **MoSCoW**: Must Have — The validation gate is a fundamental safety mechanism in the plan lifecycle. The specification's four-phase model (Action → Strategize → Execute → Apply) requires validation at the Apply phase boundary. This is a showstopper for v3.5.0. This is the **4th Critical Must Have bug** in v3.5.0. The milestone is severely behind with critical safety and correctness issues. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Must have

label

2026-04-05 08:23:52 +00:00

freemo commented

2026-04-05 08:28:02 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Critical — agents plan apply bypasses the validation-gated apply pipeline, which is a core M6 acceptance criterion
Milestone: v3.5.0 (already assigned — correct)
MoSCoW: Must Have — the spec requires validation-gated apply. If the CLI bypasses PlanApplyService.apply_with_validation_gate, the entire safety mechanism is circumvented. This is essential for milestone completion.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Critical — `agents plan apply` bypasses the validation-gated apply pipeline, which is a core M6 acceptance criterion - **Milestone**: v3.5.0 (already assigned — correct) - **MoSCoW**: Must Have — the spec requires validation-gated apply. If the CLI bypasses `PlanApplyService.apply_with_validation_gate`, the entire safety mechanism is circumvented. This is essential for milestone completion. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-05 08:40:27 +00:00

[CA-AUTO] System Watchdog — Session Tracker — watchdog-1 — 2026-04-05 #3197

freemo referenced this issue

2026-04-05 08:42:35 +00:00

[CA-AUTO] Architecture Supervisor — Session Tracker — 2026-04-05 #3180

freemo referenced this issue

2026-04-05 08:49:48 +00:00

[CA-AUTO] Project Owner — project-owner-1 — Session Tracker #2793

freemo referenced this issue

2026-04-05 09:01:27 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.2.0–v3.7.0 — Session Tracker #3094