cleveragents-core/features/validation_edge_cases.feature

Feature: Validation edge cases
  As a developer
  I want edge-case validations to be handled gracefully
  So that malformed outputs, missing resources, timeouts, and ordering conflicts are detected

  Background:
    Given a validation edge case test environment

  # ── Malformed tool output handling ─────────────────────────────

  Scenario: Detect tool output missing required status field
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "missing_required_status_field"
    Then validation edge should report error containing "missing required field: status"

  Scenario: Detect tool output with wrong type for status
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "wrong_type_for_status"
    Then validation edge should report error containing "invalid type for field 'status'"

  Scenario: Detect tool output with null result payload
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "null_result_payload"
    Then validation edge should report error containing "result payload must not be null"

  Scenario: Detect tool output with extra null fields
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "extra_null_fields"
    Then validation edge should report error containing "unexpected null fields"

  Scenario: Detect empty tool output object
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "empty_output_object"
    Then validation edge should report error containing "must contain at least"

  Scenario: Detect string tool output instead of object
    Given validation edge fixture "malformed_tool_output" is loaded
    When validation edge processes fixture "output_is_string_not_object"
    Then validation edge should report error containing "must be an object"

  # ── Missing resource references ────────────────────────────────

  Scenario: Detect dangling action reference in plan
    Given validation edge fixture "missing_resources" is loaded
    When validation edge checks resource fixture "dangling_action_reference"
    Then validation edge should report error containing "action not found"

  Scenario: Detect unresolvable project path
    Given validation edge fixture "missing_resources" is loaded
    When validation edge checks resource fixture "unresolvable_project_path"
    Then validation edge should report error containing "unresolvable project path"

  Scenario: Detect missing actor for execution phase
    Given validation edge fixture "missing_resources" is loaded
    When validation edge checks resource fixture "missing_actor_for_phase"
    Then validation edge should report error containing "missing actor for phase"

  Scenario: Detect dangling parent plan reference
    Given validation edge fixture "missing_resources" is loaded
    When validation edge checks resource fixture "dangling_parent_plan_id"
    Then validation edge should report error containing "parent plan not found"

  Scenario: Detect circular resource dependency
    Given validation edge fixture "missing_resources" is loaded
    When validation edge checks resource fixture "circular_resource_dependency"
    Then validation edge should report error containing "circular dependency detected"

  # ── Validation timeout behavior ────────────────────────────────

  Scenario: Detect slow schema validator exceeding timeout
    Given validation edge fixture "validation_timeouts" is loaded
    When validation edge simulates timeout for fixture "slow_schema_validator"
    Then validation edge should report error containing "validation timeout"

  Scenario: Detect zero timeout causes immediate failure
    Given validation edge fixture "validation_timeouts" is loaded
    When validation edge simulates timeout for fixture "zero_timeout_immediate_fail"
    Then validation edge should report error containing "exceeded 0ms deadline"

  Scenario: Detect negative timeout is rejected
    Given validation edge fixture "validation_timeouts" is loaded
    When validation edge simulates timeout for fixture "negative_timeout_rejected"
    Then validation edge should report error containing "invalid timeout"

  Scenario: Detect timeout with partial results reported
    Given validation edge fixture "validation_timeouts" is loaded
    When validation edge simulates timeout for fixture "timeout_with_partial_results"
    Then validation edge should report error containing "completed 5/20 items"

  # ── Invalid schema transform results ───────────────────────────

  Scenario: Detect transform returning non-dict schema
    Given validation edge fixture "invalid_schema_transforms" is loaded
    When validation edge applies transform fixture "transform_returns_non_dict"
    Then validation edge should report error containing "expected dict, got str"

  Scenario: Detect transform returning schema without type field
    Given validation edge fixture "invalid_schema_transforms" is loaded
    When validation edge applies transform fixture "transform_returns_missing_type"
    Then validation edge should report error containing "without 'type' field"

  Scenario: Detect transform returning null schema
    Given validation edge fixture "invalid_schema_transforms" is loaded
    When validation edge applies transform fixture "transform_returns_null_schema"
    Then validation edge should report error containing "returned null schema"

  # ── Mixed ordering and duplicate attachment IDs ────────────────

  Scenario: Required validations ordered before informational
    Given validation edge fixture "mixed_ordering" is loaded
    When validation edge sorts ordering fixture "required_before_informational"
    Then validation edge ordering should match expected order

  Scenario: Duplicate attachment IDs across required validations rejected
    Given validation edge fixture "mixed_ordering" is loaded
    When validation edge sorts ordering fixture "duplicate_attachment_ids_rejected"
    Then validation edge should report error containing "duplicate attachment ID"

  Scenario: Duplicate attachment IDs across levels rejected
    Given validation edge fixture "mixed_ordering" is loaded
    When validation edge sorts ordering fixture "duplicate_attachment_ids_across_levels"
    Then validation edge should report error containing "duplicate attachment ID"

  Scenario: Unknown validation level is rejected
    Given validation edge fixture "mixed_ordering" is loaded
    When validation edge sorts ordering fixture "unknown_validation_level"
    Then validation edge should report error containing "unknown validation level"

  # ── Concurrent validation conflicts ────────────────────────────

  Scenario: Concurrent validators on same resource detect conflict
    Given validation edge has two concurrent validators for resource "shared-config"
    When validation edge runs both validators concurrently
    Then validation edge should detect a concurrent modification conflict

  # ── Rollback on validation failure ─────────────────────────────

  Scenario: Validation failure triggers state rollback
    Given validation edge has a staged change set with 3 pending changes
    When validation edge encounters a required validation failure
    Then validation edge should rollback all 3 pending changes
    And validation edge rollback result should contain "rolled back"