Files
cleveragents-core/features/coverage_validation_error_paths.feature
drew a49f37eb1f
CI / integration_tests (pull_request) Has started running
CI / push-validation (pull_request) Successful in 38s
CI / helm (pull_request) Successful in 45s
CI / build (pull_request) Successful in 47s
CI / lint (pull_request) Successful in 1m11s
CI / quality (pull_request) Successful in 1m16s
CI / typecheck (pull_request) Successful in 1m24s
CI / security (pull_request) Successful in 1m23s
CI / unit_tests (pull_request) Failing after 4m57s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Has been cancelled
test(coverage): de-razor master coverage 96.465% -> 96.623%
Master passed the 96.5 floor only by rounding (96.465 -> 96.5). Add genuine
behave coverage + omit a structurally-uncoverable module so the floor has real
headroom. fail_under stays 96.5 (ratchet rule: raise only after coverage holds).

- omit src/cleveragents/application/services/__init__.py (noxfile + pyproject):
  100% of its "missing" lines are inside an `if TYPE_CHECKING:` block (never
  executes at runtime) and slipcover has no per-line pragma to exclude them.
- features/coverage_validation_error_paths.feature (+ steps): 16 scenarios
  exercising the defensive error-path branches in core/validation.py that the
  existing structural_validation.feature does not reach (non-dict node, dup
  decision_id, non-list children, invalid child ULID, missing decision fields,
  wrong-typed confidence/parent/sequence, malformed structured-output, unknown
  dispatcher target). Routed through the public
  validate_structured_component_output dispatcher; pure functions, no DB/CLI.

Full engine run: NOX_EXIT=0, no dead chunks, 96.623% (validation.py now fully
covered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 17:07:39 -04:00

147 lines
6.2 KiB
Gherkin

@coverage
Feature: Structural validation error-path coverage
Exercises uncovered defensive branches in cleveragents.core.validation
(plan-tree, decision-dict, and structured-output validators) that the
existing structural_validation.feature does not reach. Pure-function tests
routed through the public validate_structured_component_output dispatcher.
# -- plan tree validator --
Scenario: Plan tree node that is not a dict
When I structurally validate "plan_tree" with the JSON payload
"""
[123]
"""
Then the structural validation fails
And the structural errors include "is not a dict"
Scenario: Plan tree with a duplicate decision_id
When I structurally validate "plan_tree" with the JSON payload
"""
[{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A"}, {"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A"}]
"""
Then the structural validation fails
And the structural errors include "duplicate decision_id"
Scenario: Plan tree node with a blank type
When I structurally validate "plan_tree" with the JSON payload
"""
[{"type": " "}]
"""
Then the structural validation fails
And the structural errors include "'type' must be a non-empty string"
Scenario: Plan tree node with a blank question
When I structurally validate "plan_tree" with the JSON payload
"""
[{"question": ""}]
"""
Then the structural validation fails
And the structural errors include "'question' must be a non-empty string"
Scenario: Plan tree children is not a list
When I structurally validate "plan_tree" with the JSON payload
"""
[{"children": "nope"}]
"""
Then the structural validation fails
And the structural errors include "'children' must be a list"
Scenario: Plan tree child with an invalid ULID
When I structurally validate "plan_tree" with the JSON payload
"""
[{"children": [{"decision_id": "not-a-ulid"}]}]
"""
Then the structural validation fails
And the structural errors include "must be a valid ULID"
# -- decision dict validator --
Scenario: Decision dict missing all required fields
When I structurally validate "decision" with the JSON payload
"""
{}
"""
Then the structural validation fails
And the structural errors include "missing required field"
Scenario: Decision dict with an invalid decision_id ULID
When I structurally validate "decision" with the JSON payload
"""
{"decision_id": "bad", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": 0.5, "parent": "(root)", "is_correction": false, "superseded": false}
"""
Then the structural validation fails
And the structural errors include "must be a valid ULID"
Scenario: Decision dict with a wrong-typed confidence
When I structurally validate "decision" with the JSON payload
"""
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": "high", "parent": "(root)", "is_correction": false, "superseded": false}
"""
Then the structural validation fails
And the structural errors include "must be float or None"
Scenario: Decision dict with a non-string parent
When I structurally validate "decision" with the JSON payload
"""
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": 0, "question": "q", "chosen": "c", "confidence": null, "parent": 123, "is_correction": false, "superseded": false}
"""
Then the structural validation fails
And the structural errors include "'parent' must be a string"
Scenario: Decision dict with a non-integer sequence
When I structurally validate "decision" with the JSON payload
"""
{"decision_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "plan_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "type": "t", "sequence": "five", "question": "q", "chosen": "c", "confidence": null, "parent": "(root)", "is_correction": false, "superseded": false}
"""
Then the structural validation fails
And the structural errors include "'sequence' must be an integer"
# -- structured output validator --
Scenario: Structured output with a blank command
When I structurally validate "structured_output" with the JSON payload
"""
{"command": "", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok"}
"""
Then the structural validation fails
And the structural errors include "'command' must be a non-empty string"
Scenario: Structured output with an invalid session_id
When I structurally validate "structured_output" with the JSON payload
"""
{"command": "run", "session_id": "bad", "status": "ok"}
"""
Then the structural validation fails
And the structural errors include "'session_id' must be a valid ULID"
Scenario: Structured output with a non-integer exit_code
When I structurally validate "structured_output" with the JSON payload
"""
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "exit_code": "x"}
"""
Then the structural validation fails
And the structural errors include "'exit_code' must be a non-negative integer"
Scenario: Structured output with elements that are not a list
When I structurally validate "structured_output" with the JSON payload
"""
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "elements": "nope"}
"""
Then the structural validation fails
And the structural errors include "'elements' must be a list"
Scenario: Structured output with a non-dict element
When I structurally validate "structured_output" with the JSON payload
"""
{"command": "run", "session_id": "01ARZ3NDEKTSV4XXFFJFRC889A", "status": "ok", "elements": [123]}
"""
Then the structural validation fails
And the structural errors include "is not a dict"
# -- dispatcher --
Scenario: Unknown target type raises a ValidationError
When I structurally validate the unknown target type "totally_unknown"
Then a structural ValidationError is raised