freemo
904cf2b3d4
Added three new modules for M3 Epic #8137: - invariant_enforcer.py detects violations during execution with actionable error messages; raises InvariantEnforcementError (blocking) or InvariantEnforcementWarning (non-blocking). - structural_component_validator.py validates output via flexible substring, wildcard, regex, and field-path matching instead of exact character equality, making tests resilient to cosmetic changes. - phase_transition_gate.py gates plan phase transitions by verifying invariant enforcement decisions and required validation results, blocking Apply when critical constraints are violated. All modules persist results as validation_response decisions in the decision tree for auditability. Includes Behave BDD test scenarios. ISSUES CLOSED: #8137
50 lines
2.5 KiB
Gherkin
50 lines
2.5 KiB
Gherkin
Feature: Structural Component Validator - flexible output validation (not exact matching)
|
|
|
|
As a plan execution system
|
|
I want to validate outputs against structural components rather than exact strings
|
|
So that tests remain resilient to formatting changes and cosmetic modifications.
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Present expectation with plain substring match passes (case insensitive)
|
|
Given a present structural expectation on keyword "backward compatibility"
|
|
And the actual output reads: "The API update maintains backward COMPATIBILITY across all endpoints"
|
|
When I validate the outputs against the expectations
|
|
Then total_checks should equal 1
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Absent expectation passes when prohibited text is missing
|
|
Given an absent structural expectation on keyword "secret key"
|
|
And the actual output reads: "Configuration loaded from environment variables"
|
|
When I validate the outputs against the expectations
|
|
Then the ValidationOutcome should have passed=True
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Absent expectation fails when prohibited text is found
|
|
Given an absent structural expectation on keyword "password"
|
|
And the actual output reads: "Using password auth for database connection"
|
|
When I validate the outputs against the expectations
|
|
Then the ValidationOutcome should have passed=False
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Regex pattern matching works across whitespace variations
|
|
Given a pattern expectation on regex "status\\s*:\\s*(passed|ok)"
|
|
And the actual output reads: "Status : passed" (with varying whitespace)
|
|
When I validate the outputs against the expectations
|
|
Then total_checks should equal 1
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Nested JSON field path validation finds existing fields
|
|
Given a present expectation on field path "data.status.passed"
|
|
And the actual output reads: '{"data": {"status": {"passed": true}}}'
|
|
When I validate the outputs against the expectations
|
|
Then total_checks should equal 1
|
|
|
|
@structural_validator @m3_epic_8137
|
|
Scenario: Multiple structural expectations aggregate correctly
|
|
Given a present structural expectation on keyword "database"
|
|
And an absent structural expectation on keyword "password"
|
|
And the actual output reads: "Database initialized. Status is ok, no password used."
|
|
When I validate the outputs against the expectations
|
|
Then the ValidationOutcome should have passed=True
|
|
And total_checks should equal 2
|