a49f37eb1f
CI / integration_tests (pull_request) Has started running
CI / push-validation (pull_request) Successful in 38s
CI / helm (pull_request) Successful in 45s
CI / build (pull_request) Successful in 47s
CI / lint (pull_request) Successful in 1m11s
CI / quality (pull_request) Successful in 1m16s
CI / typecheck (pull_request) Successful in 1m24s
CI / security (pull_request) Successful in 1m23s
CI / unit_tests (pull_request) Failing after 4m57s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Has been cancelled
Master passed the 96.5 floor only by rounding (96.465 -> 96.5). Add genuine behave coverage + omit a structurally-uncoverable module so the floor has real headroom. fail_under stays 96.5 (ratchet rule: raise only after coverage holds). - omit src/cleveragents/application/services/__init__.py (noxfile + pyproject): 100% of its "missing" lines are inside an `if TYPE_CHECKING:` block (never executes at runtime) and slipcover has no per-line pragma to exclude them. - features/coverage_validation_error_paths.feature (+ steps): 16 scenarios exercising the defensive error-path branches in core/validation.py that the existing structural_validation.feature does not reach (non-dict node, dup decision_id, non-list children, invalid child ULID, missing decision fields, wrong-typed confidence/parent/sequence, malformed structured-output, unknown dispatcher target). Routed through the public validate_structured_component_output dispatcher; pure functions, no DB/CLI. Full engine run: NOX_EXIT=0, no dead chunks, 96.623% (validation.py now fully covered). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
147 lines
6.2 KiB
Gherkin
147 lines
6.2 KiB
Gherkin
@coverage
|
|
Feature: Structural validation error-path coverage
|
|
Exercises uncovered defensive branches in cleveragents.core.validation
|
|
(plan-tree, decision-dict, and structured-output validators) that the
|
|
existing structural_validation.feature does not reach. Pure-function tests
|
|
routed through the public validate_structured_component_output dispatcher.
|
|
|
|
# -- plan tree validator --
|
|
|
|
Scenario: Plan tree node that is not a dict
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[123]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "is not a dict"
|
|
|
|
Scenario: Plan tree with a duplicate decision_id
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A"}, {"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A"}]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "duplicate decision_id"
|
|
|
|
Scenario: Plan tree node with a blank type
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[{"type": " "}]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'type' must be a non-empty string"
|
|
|
|
Scenario: Plan tree node with a blank question
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[{"question": ""}]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'question' must be a non-empty string"
|
|
|
|
Scenario: Plan tree children is not a list
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[{"children": "nope"}]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'children' must be a list"
|
|
|
|
Scenario: Plan tree child with an invalid ULID
|
|
When I structurally validate "plan_tree" with the JSON payload
|
|
"""
|
|
[{"children": [{"decision_id": "not-a-ulid"}]}]
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "must be a valid ULID"
|
|
|
|
# -- decision dict validator --
|
|
|
|
Scenario: Decision dict missing all required fields
|
|
When I structurally validate "decision" with the JSON payload
|
|
"""
|
|
{}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "missing required field"
|
|
|
|
Scenario: Decision dict with an invalid decision_id ULID
|
|
When I structurally validate "decision" with the JSON payload
|
|
"""
|
|
{"decision_id": "bad", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": 0.5, "parent": "(root)", "is_correction": false, "superseded": false}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "must be a valid ULID"
|
|
|
|
Scenario: Decision dict with a wrong-typed confidence
|
|
When I structurally validate "decision" with the JSON payload
|
|
"""
|
|
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": "high", "parent": "(root)", "is_correction": false, "superseded": false}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "must be float or None"
|
|
|
|
Scenario: Decision dict with a non-string parent
|
|
When I structurally validate "decision" with the JSON payload
|
|
"""
|
|
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": null, "parent": 123, "is_correction": false, "superseded": false}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'parent' must be a string"
|
|
|
|
Scenario: Decision dict with a non-integer sequence
|
|
When I structurally validate "decision" with the JSON payload
|
|
"""
|
|
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": "five", "question": "q", "chosen": "c", "confidence": null, "parent": "(root)", "is_correction": false, "superseded": false}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'sequence' must be an integer"
|
|
|
|
# -- structured output validator --
|
|
|
|
Scenario: Structured output with a blank command
|
|
When I structurally validate "structured_output" with the JSON payload
|
|
"""
|
|
{"command": "", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok"}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'command' must be a non-empty string"
|
|
|
|
Scenario: Structured output with an invalid session_id
|
|
When I structurally validate "structured_output" with the JSON payload
|
|
"""
|
|
{"command": "run", "session_id": "bad", "status": "ok"}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'session_id' must be a valid ULID"
|
|
|
|
Scenario: Structured output with a non-integer exit_code
|
|
When I structurally validate "structured_output" with the JSON payload
|
|
"""
|
|
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "exit_code": "x"}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'exit_code' must be a non-negative integer"
|
|
|
|
Scenario: Structured output with elements that are not a list
|
|
When I structurally validate "structured_output" with the JSON payload
|
|
"""
|
|
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "elements": "nope"}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "'elements' must be a list"
|
|
|
|
Scenario: Structured output with a non-dict element
|
|
When I structurally validate "structured_output" with the JSON payload
|
|
"""
|
|
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "elements": [123]}
|
|
"""
|
|
Then the structural validation fails
|
|
And the structural errors include "is not a dict"
|
|
|
|
# -- dispatcher --
|
|
|
|
Scenario: Unknown target type raises a ValidationError
|
|
When I structurally validate the unknown target type "totally_unknown"
|
|
Then a structural ValidationError is raised
|