Compare commits

...

1 Commits

Author SHA1 Message Date
HAL9000 b5b3d9b8f8 docs(spec): update specification — validation gate blocks on empty validation summary
Added comprehensive documentation for the validation gate feature that blocks apply when the validation summary is empty. The new section explains:
- Validation gate behavior and enforcement
- Empty validation summary blocking mechanism
- Required vs informational validation handling
- Validation summary structure and gate checks
- Best practices for preventing empty validation summaries

This ensures teams maintain consistent validation coverage across all plans before allowing apply operations.
2026-04-22 10:17:52 +00:00
2 changed files with 74 additions and 0 deletions
+58
View File
@@ -19879,6 +19879,64 @@ Validation runs during **Execute**, not during Apply. By the time a plan reaches
For full details on validation — including the Validation type system, modes, attachment scoping, failure handling, fix-then-revalidate loops, and data model — see the **Validation** section under Core Concepts.
#### Validation Gate
!!! warning "Apply Gate Enforcement"
Before a plan can transition to the Apply phase, the validation gate must pass. The validation gate blocks apply when the validation summary is empty, ensuring that validation results are always present before allowing apply.
##### Validation Gate Behavior
The validation gate is an automatic check that runs before the Apply phase begins:
1. **Empty Validation Summary Blocks Apply**
If no validations are attached to the plan's resources, or if all attached validations are skipped due to scope mismatches, the validation summary is empty. An empty validation summary indicates that no validation work was performed, which is a potentially unsafe state. The validation gate **blocks apply** in this case.
This ensures that:
- Plans always have explicit validation coverage before committing changes
- Accidental omission of required validations is caught early
- Teams maintain consistent quality standards across all plans
2. **Required Validations Must Pass**
All required validations must have passed during the Execute phase. If any required validation failed and could not be fixed, the plan does not reach the Apply phase at all — it fails during Execute.
3. **Informational Validations Do Not Block**
Informational validations may fail without blocking apply. Their results are recorded in the validation summary for human review, but they do not prevent the apply gate from opening.
4. **Gate Failure Reasons**
When the validation gate blocks apply, the failure reason is recorded in the plan's metadata:
- "empty validation summary" — no validations were attached or executed
- "required validation failed" — a required validation did not pass (though this should not occur if the plan reached Apply, as required validations are enforced during Execute)
##### Validation Summary Structure
The validation summary is a structured record of all validation results with the following fields:
- `total` — total number of validations executed
- `all_required_passed` — boolean indicating if all required validations passed
- `required_passed` — count of required validations that passed
- `required_failed` — count of required validations that failed
- `informational_passed` — count of informational validations that passed
- `informational_failed` — count of informational validations that failed
- `results` — array of individual validation result objects
The gate checks:
- `total > 0` — at least one validation was executed
- `all_required_passed == true` — all required validations passed
If either condition is false, the gate blocks apply.
##### Preventing Empty Validation Summary
To ensure your plans have validation coverage:
1. **Attach validations to resources**: Use `agents validation attach` to attach required validations to the resources your plan accesses.
2. **Use project-scoped attachments**: Attach validations with `--project` scope to ensure they apply to all plans using that project.
3. **Verify attachment resolution**: Before running a plan, check which validations will be attached using `agents resource show`.
4. **Test with a dry-run**: Use `agents plan use` with `--automation-profile manual` to review the plan before execution, ensuring validations are attached as expected.
#### Apply Data Model
A plan in Apply includes:
@@ -0,0 +1,16 @@
Feature: Validation gate blocks on empty validation summary
The validation gate should block apply when the validation summary is empty,
ensuring that validation results are always present before allowing apply.
Scenario: Empty validation summary blocks apply
Given a validation apply test environment
When I run the apply validation gate for plan "01PLAN0001" with no attachments
Then the gate should block apply
And the apply summary should be empty
And the gate failure reasons should mention "empty validation summary"
Scenario: Empty summary is documented in specification
Given the specification document
When I search for "validation gate" section
Then the specification should document that empty validation summary blocks apply
And the specification should explain the blocking behavior