docs(spec): update specification — validation gate blocks on empty validation summary
CI / helm (push) Successful in 33s
CI / build (push) Successful in 1m2s
CI / lint (push) Successful in 1m12s
CI / quality (push) Successful in 1m18s
CI / typecheck (push) Successful in 1m38s
CI / security (push) Successful in 1m42s
CI / push-validation (push) Successful in 20s
CI / e2e_tests (push) Successful in 4m14s
CI / integration_tests (push) Successful in 4m50s
CI / unit_tests (push) Successful in 6m1s
CI / docker (push) Successful in 1m31s
CI / coverage (push) Successful in 10m48s
CI / status-check (push) Successful in 3s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (pull_request) Has been skipped
CI / helm (pull_request) Successful in 39s
CI / push-validation (pull_request) Successful in 37s
CI / build (pull_request) Successful in 59s
CI / lint (pull_request) Successful in 1m14s
CI / typecheck (pull_request) Successful in 1m26s
CI / quality (pull_request) Successful in 1m34s
CI / security (pull_request) Successful in 1m43s
CI / benchmark-regression (pull_request) Failing after 1m7s
CI / integration_tests (pull_request) Successful in 3m36s
CI / unit_tests (pull_request) Successful in 4m44s
CI / e2e_tests (pull_request) Successful in 5m19s
CI / docker (pull_request) Successful in 1m30s
CI / coverage (pull_request) Successful in 11m8s
CI / status-check (pull_request) Successful in 3s
CI / benchmark-publish (push) Successful in 1h17m43s

This commit was merged in pull request #8225.
This commit is contained in:
2026-05-08 10:21:23 +00:00
committed by Forgejo
parent af6e54f0b2
commit 57881a075b
3 changed files with 25 additions and 4 deletions
+7
View File
@@ -653,6 +653,13 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
delays from 15s to 2s, and reduced idle sleep from 60s to 10s for dramatically
faster throughput.
- **Specification — Validation Gate Empty-Run Guard** (#8146): Updated `docs/specification.md`
to document the security invariant introduced in PR #7786 (fixing issue #7508). The spec now
explicitly states that `ApplyValidationSummary.all_required_passed` returns `False` when no
validations have been run (empty summary), blocking apply. Added a prominent danger admonition
block, updated the validation process results section, the `final_validation_results` data
model description, and two milestone acceptance criteria to reflect the corrected blocking
behavior for empty validation summaries and no-attachment runs.
### Fixed
+1
View File
@@ -22,6 +22,7 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed the benchmark workflow separation (#9040): moved the benchmark-regression job out of the default PR workflow into a dedicated scheduled workflow, reducing median PR CI turnaround time from 99-132 minutes to under 30 minutes.
* HAL 9000 has contributed the agent-evolution-pool-supervisor PR metadata assignment (#7888): the supervisor now automatically looks up the Type/Automation label and earliest open milestone before dispatching improvement PR creation workers, ensuring all generated improvement PRs have correct Type labels and milestone assignments.
* HAL 9000 has contributed the decision recording hook for the Strategize phase (issue #8522): captures every decision point with question, chosen option, alternatives, confidence, rationale, and full context snapshot for replay and correction.
* HAL 9000 has contributed automated specification maintenance, documentation updates, and bot-driven PR authorship.
* This project was made possible thanks to considerable donation of time, money, and resources by CleverThis, Inc.
* HAL 9000 has contributed automated bug fixes, CLI output formatting improvements, and ongoing maintenance as part of the CleverAgents automation system.
* HAL 9000 has contributed the file edit encoding parameter fix (PR #8258 / issue #7559).
+17 -4
View File
@@ -20019,6 +20019,18 @@ Apply should perform (configurable) validations before committing:
Validation runs during **Execute**, not during Apply. By the time a plan reaches the Apply phase, all required validations have already passed. Apply commits the sandbox changes to the real resources and does not re-run validations.
!!! danger "Security Invariant — Empty Validation Summary Blocks Apply"
Apply **requires** that at least one required validation was executed during the Execute phase. If no validations were run — because no validations are attached to the plan's resources, or because the validation collection step produced an empty set — the apply gate **blocks** and the plan cannot proceed to Apply.
This is an intentional security invariant (introduced in PR #7786, fixing issue #7508): apply cannot proceed without at least one validation having run. A plan with no validation attachments is treated the same as a plan with failing validations — it cannot be applied.
**Blocking conditions for the apply gate**:
- Any required validation returned `passed: false` (validation failure)
- No required validations were executed at all (empty validation summary)
- A plan has no validation attachments (no validations configured)
The `ApplyValidationSummary.all_required_passed` property returns `True` **only when** at least one required validation was executed AND none of the required validations failed. An empty summary always yields `False`.
For full details on validation — including the Validation type system, modes, attachment scoping, failure handling, fix-then-revalidate loops, and data model — see the **Validation** section under Core Concepts.
#### Apply Data Model
@@ -22757,7 +22769,8 @@ Validation is the **final step** of the Execute phase. The execution actor's wor
2. **Collect applicable validations**: The system resolves all validations from the resource-direct, project, and plan scopes (as described in Attachment Resolution above).
3. **Execute validations**: Each applicable validation is invoked as a standard tool call. The validation reads the sandbox state and returns its structured result. Validations may be run in parallel since they are read-only and cannot interfere with each other.
4. **Process results**:
- **All required validations pass**: Execution is complete. The plan transitions to the review/Apply phase.
- **All required validations pass (and at least one ran)**: Execution is complete. The plan transitions to the review/Apply phase.
- **No required validations were executed (empty summary)**: The apply gate is blocked. This is treated equivalently to a required validation failure — the plan cannot proceed to Apply. This condition arises when no validations are attached to the plan's resources. See the security invariant note in [Validation in Apply](#validation-in-apply).
- **Any required validation fails**: The execution actor enters the fix-then-revalidate loop (see Validation Failure Handling below).
- **Informational validations fail**: Results are recorded in the plan's validation summary. No fix attempts are made.
@@ -23076,7 +23089,7 @@ The validation-related fields stored in the plan data model:
| Field | Location | Description |
|-------|----------|-------------|
| `validation_summary` | `plan.execution` | Array of validation results collected during Execute. Each entry includes the validation name, mode, `passed`, `message`, `data`, execution duration, and attempt number. |
| `final_validation_results` | `plan.apply` | Snapshot of the final validation state at the time execution completed (all required validations passing). Identical to the last state of `validation_summary` but stored separately for quick reference. |
| `final_validation_results` | `plan.apply` | Snapshot of the final validation state at the time execution completed (all required validations passing, and at least one required validation was executed). Identical to the last state of `validation_summary` but stored separately for quick reference. A plan can only reach the Apply phase if this snapshot is non-empty and all required entries have `passed: true`. |
| `validation_attempts` | `plan.execution` | Total number of validation invocations across all fix-then-revalidate loops. Useful for understanding how much effort was spent on validation remediation. |
| `validation_fix_history` | `plan.execution` | Per-validation log of fix attempts. Each entry records: the validation that failed, the fix the actor applied, and whether the subsequent re-validation passed. Enables auditing of the fix-then-revalidate process. |
@@ -47123,7 +47136,7 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a
| 6 | Hierarchical decomposition creates 4+ levels of subplans | §Subplan Architecture — Hierarchy | `plan tree` shows 4+ nesting levels for complex tasks |
| 7 | Parallel execution scales to 10+ concurrent subplans | §Subplan Execution — Parallel | 10 subplans execute concurrently without deadlock or resource starvation |
| 8 | Decision correction recomputes only affected subtree | §Correction Model — Selective Recomputation | Unaffected decisions preserved; only downstream decisions recomputed |
| 9 | Validation-gated apply: required validations must pass before apply | §Validation Abstraction — Apply Gating | Apply blocked when required validation fails; informational validation does not block |
| 9 | Validation-gated apply: required validations must pass before apply | §Validation Abstraction — Apply Gating | Apply blocked when required validation fails; apply also blocked when no validations were run (empty summary); informational validation does not block |
| 10 | A realistic porting task completes autonomously | §Autonomy Acceptance | End-to-end task (e.g., port a Python module) completes without human intervention |
| 11 | `agents automation-profile add/list/show` functional | §CLI Commands — automation-profile | Round-trip: add profile, list shows it, show displays details |
| 12 | `agents plan guard` shows active guards for a plan | §CLI Commands — plan guard | Command shows denylist, budget caps, tool limits |
@@ -47135,7 +47148,7 @@ This section defines the ordered milestone plan for CleverAgents v3.x, mapping a
- **Guard evaluation order**: Denylist checked first (fast reject), then budget caps, then tool call limits.
- **Autonomy threshold**: `0.0` = always automatic; `1.0` = always manual; eight built-in profiles from `manual` to `full-auto`.
- **Subtree recomputation**: Only decisions with `depends_on` edges to the corrected decision are recomputed; others preserved.
- **Validation gating**: `required` validations block apply; `informational` validations log but do not block.
- **Validation gating**: `required` validations block apply; an empty validation summary (no validations ran) also blocks apply; `informational` validations log but do not block.
#### Definition of Done