Feature: Semantic validation service As a plan executor I want semantic validation checks for Python source code So that syntax errors, missing imports, and broken references are caught early Background: Given a semantic validation test environment # ── Severity enum ────────────────────────────────────────────── Scenario: SemanticValidationSeverity has three levels Then the semantic severity enum has values "info", "warn", "error" # ── SemanticCheckResult model ────────────────────────────────── Scenario: SemanticCheckResult normalised output has passed message data Given a semantic check result with passed true and message "All good" Then the semantic check result passed is true And the semantic check result message is "All good" And the semantic check result data is none Scenario: SemanticCheckResult with failure and data Given a semantic check result with passed false and message "Error found" and data Then the semantic check result passed is false And the semantic check result has data key "detail" # ── SyntaxCheckRule ──────────────────────────────────────────── Scenario: SyntaxCheckRule passes for valid Python Given valid Python source code When the syntax check rule runs Then the semantic check result passed is true Scenario: SyntaxCheckRule fails for invalid Python Given Python source code with a syntax error When the syntax check rule runs Then the semantic check result passed is false And the semantic check result message contains "Syntax error" And the semantic check result severity is "error" # ── MissingImportRule ────────────────────────────────────────── Scenario: MissingImportRule passes for valid imports Given Python source with standard imports When the missing import rule runs Then the semantic check result passed is true Scenario: MissingImportRule flags suspicious private imports Given Python source with suspicious private imports When the missing import rule runs Then the semantic check result passed is false And the semantic check result message contains "suspicious import" Scenario: MissingImportRule skips on syntax error Given Python source code with a syntax error When the missing import rule runs Then the semantic check result passed is true And the semantic check result message contains "Skipped" # ── BrokenReferenceRule ──────────────────────────────────────── Scenario: BrokenReferenceRule passes for well-defined code Given Python source with all names defined When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule flags undefined names Given Python source with undefined references When the broken reference rule runs Then the semantic check result passed is false And the semantic check result message contains "broken reference" # ── DependencyCycleRule ──────────────────────────────────────── Scenario: DependencyCycleRule passes for no duplicate relative imports Given Python source with unique relative imports When the dependency cycle rule runs Then the semantic check result passed is true Scenario: DependencyCycleRule flags duplicate relative imports Given Python source with duplicate relative imports When the dependency cycle rule runs Then the semantic check result passed is false And the semantic check result message contains "Duplicate relative imports" # ── APIMisuseRule ────────────────────────────────────────────── Scenario: APIMisuseRule passes for clean code Given Python source without API misuse When the API misuse rule runs Then the semantic check result passed is true Scenario: APIMisuseRule flags eval usage Given Python source using eval When the API misuse rule runs Then the semantic check result passed is false And the semantic check result message contains "API misuse" # ── MissingSymbolRule ────────────────────────────────────────── Scenario: MissingSymbolRule passes for well-scoped functions Given Python source with well-scoped functions When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule flags undefined symbols in functions Given Python source with undefined symbols in functions When the missing symbol rule runs Then the semantic check result passed is false And the semantic check result message contains "missing symbol" # ── SemanticRuleRegistry ─────────────────────────────────────── Scenario: Default registry has all six built-in rules Given a default semantic rule registry Then the registry has 6 rules And the registry contains rule "syntax_error" And the registry contains rule "missing_import" And the registry contains rule "broken_reference" And the registry contains rule "duplicate_import" And the registry contains rule "api_misuse" And the registry contains rule "missing_symbol" Scenario: Registry lookup returns None for unknown rule Given a default semantic rule registry Then the registry returns None for rule "nonexistent" Scenario: Registry remove returns true for existing rule Given a default semantic rule registry When the rule "syntax_error" is removed from the registry Then the registry has 5 rules And the registry returns None for rule "syntax_error" Scenario: Registry remove returns false for unknown rule Given a default semantic rule registry Then removing rule "nonexistent" returns false # ── SemanticValidationCache ──────────────────────────────────── Scenario: Cache returns None for uncached entries Given a semantic validation cache Then the cache returns None for rule "check" and hash "abc123" Scenario: Cache stores and retrieves results by rule and hash Given a semantic validation cache And a cached semantic result for rule "check" and hash "abc123" Then the cache returns the result for rule "check" and hash "abc123" Scenario: Cache invalidation removes entries Given a semantic validation cache And a cached semantic result for rule "check" and hash "abc123" When the cache entry for rule "check" and hash "abc123" is invalidated Then the cache returns None for rule "check" and hash "abc123" Scenario: Cache clear removes all entries Given a semantic validation cache And a cached semantic result for rule "a" and hash "h1" And a cached semantic result for rule "b" and hash "h2" When the semantic cache is cleared Then the semantic cache has 0 entries Scenario: Cache compute_hash returns consistent SHA-256 Then the semantic cache hash for "hello" is consistent # ── Severity mapping ─────────────────────────────────────────── Scenario: ERROR severity maps to REQUIRED mode Then semantic severity "error" maps to validation mode "required" Scenario: WARN severity maps to INFORMATIONAL mode Then semantic severity "warn" maps to validation mode "informational" Scenario: INFO severity maps to INFORMATIONAL mode Then semantic severity "info" maps to validation mode "informational" Scenario: resolve_severity uses default mapping Then resolve_severity for "syntax_error" returns "error" And resolve_severity for "missing_import" returns "warn" And resolve_severity for "duplicate_import" returns "error" And resolve_severity for "missing_symbol" returns "info" Scenario: resolve_severity uses custom mapping Given a custom severity mapping with "syntax_error" as "warn" Then resolve_severity with custom mapping for "syntax_error" returns "warn" Scenario: resolve_severity falls back to info for unknown rule Then resolve_severity for "unknown_rule" returns "info" # ── SemanticValidationService ────────────────────────────────── Scenario: Service check_file runs all rules on valid Python Given a semantic validation service And valid Python source code When the service checks the file Then all semantic check results passed Scenario: Service check_file detects syntax error Given a semantic validation service And Python source code with a syntax error When the service checks the file Then at least one semantic check result failed And a failing result has message containing "Syntax error" Scenario: Service check_file with specific rules Given a semantic validation service And valid Python source code When the service checks the file with rules "syntax_error" Then only 1 semantic check result is returned Scenario: Service returns empty when disabled Given a semantic validation service with semantic disabled And valid Python source code When the service checks the file Then 0 semantic check results are returned Scenario: Service returns empty for Python when python disabled Given a semantic validation service with python disabled And valid Python source code When the service checks the python file Then 0 semantic check results are returned Scenario: Service uses cache for repeated checks Given a semantic validation service And valid Python source code When the service checks the file twice Then the cache has entries # ── Pipeline integration ─────────────────────────────────────── Scenario: as_pipeline_results returns normalised dicts Given a semantic validation service And valid Python source code When the service returns pipeline results Then each pipeline result has keys "passed" and "message" and "data" And each pipeline result has key "severity" And each pipeline result has key "mode" Scenario: normalise_output returns standard schema Given a semantic check result with passed true and message "OK" When the result is normalised Then the normalised output has keys "passed" and "message" and "data" # ── Config keys ──────────────────────────────────────────────── Scenario: Config key constants are defined Then config key "validation.semantic.enabled" exists And config key "validation.semantic.python.enabled" exists And config key "validation.semantic.severity_mapping" exists # ── Required vs informational attachment modes ───────────────── Scenario: Error severity findings integrate as required in pipeline Given a semantic validation service And Python source code with a syntax error When the service returns pipeline results Then a pipeline result with severity "error" has mode "required" Scenario: Warn severity findings integrate as informational in pipeline Given a semantic validation service And Python source using eval When the service returns pipeline results Then a pipeline result with severity "warn" has mode "informational" # ── BrokenReferenceRule scope-awareness ───────────────────────── Scenario: BrokenReferenceRule passes for function-local variables Given Python source with function-local variables When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for annotated assignments Given Python source with annotated assignment When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for with-statement variables Given Python source with with-statement variable When the broken reference rule runs Then the semantic check result passed is true # ── MissingSymbolRule class/nested function support ──────────── Scenario: MissingSymbolRule detects undefined symbols in class methods Given Python source with a class method using undefined symbol When the missing symbol rule runs Then the semantic check result passed is false And the semantic check result message contains "missing symbol" Scenario: MissingSymbolRule passes for nested function calls Given Python source with nested functions When the missing symbol rule runs Then the semantic check result passed is true # ── MissingSymbolRule comprehension scope ─────────────────────── Scenario: MissingSymbolRule does not flag comprehension variables Given Python source with comprehension variables When the missing symbol rule runs Then the semantic check result passed is true # ── DuplicateImportRule alias ────────────────────────────────── Scenario: DuplicateImportRule passes for no duplicate relative imports Given Python source with unique relative imports When the duplicate import rule runs Then the semantic check result passed is true Scenario: DuplicateImportRule flags duplicate relative imports Given Python source with duplicate relative imports When the duplicate import rule runs Then the semantic check result passed is false And the semantic check result message contains "Duplicate relative imports" # ── APIMisuseRule AST-based detection ────────────────────────── Scenario: APIMisuseRule flags exec usage Given Python source using exec When the API misuse rule runs Then the semantic check result passed is false And the semantic check result message contains "API misuse" Scenario: APIMisuseRule flags os.popen usage Given Python source using os.popen When the API misuse rule runs Then the semantic check result passed is false And the semantic check result message contains "API misuse" Scenario: APIMisuseRule flags subprocess.run usage Given Python source using subprocess.run When the API misuse rule runs Then the semantic check result passed is false And the semantic check result message contains "API misuse" Scenario: APIMisuseRule ignores eval in string literals Given Python source with eval in a string literal When the API misuse rule runs Then the semantic check result passed is true # ── Non-Python file handling ─────────────────────────────────── Scenario: Service returns empty for non-Python files Given a semantic validation service And valid Python source code And a non-Python filename When the service checks the non-python file Then 0 semantic check results are returned # ── Empty source ─────────────────────────────────────────────── Scenario: Service handles empty source code Given a semantic validation service And empty Python source code When the service checks the file Then all semantic check results passed # ── Cache with max size ──────────────────────────────────────── Scenario: Cache respects max size with LRU eviction Given a semantic validation cache with max size 2 And a cached semantic result for rule "a" and hash "h1" And a cached semantic result for rule "b" and hash "h2" And a cached semantic result for rule "c" and hash "h3" Then the semantic cache has 2 entries # ── Safe initialisation and cleanup ──────────────────────────── Scenario: Cache rejects invalid max size on construction Then creating a semantic validation cache with max size 0 raises ValueError Scenario: Cache put updates existing entry without growing Given a semantic validation cache And a cached semantic result for rule "r1" and hash "h1" When the cache entry for rule "r1" and hash "h1" is overwritten with message "updated" Then the cache returns the updated result for rule "r1" and hash "h1" with message "updated" And the semantic cache has 1 entries Scenario: Registry list_rules returns sorted names Given a default semantic rule registry Then the registry list_rules returns sorted rule names Scenario: Service exposes registry property for inspection Given a semantic validation service Then the service registry property returns a registry with 6 rules Scenario: resolve_severity falls back to info for invalid severity value Given a custom severity mapping with "syntax_error" as "bogus_value" Then resolve_severity with custom mapping for "syntax_error" returns "info" Scenario: Service functions with non-dict severity mapping config Given a semantic validation service with non-dict severity mapping And valid Python source code When the service checks the file Then all semantic check results passed # ── MissingImportRule from-import form ───────────────────────── Scenario: MissingImportRule flags suspicious from-import with private module Given Python source with from-import of private module When the missing import rule runs Then the semantic check result passed is false And the semantic check result message contains "suspicious import" # ── BrokenReferenceRule advanced scope patterns ──────────────── Scenario: BrokenReferenceRule passes for code with varargs and kwargs Given Python source with varargs and kwargs function When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for from-import bindings Given Python source with from-import bindings When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for except handler variable Given Python source with except handler variable When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for tuple and starred unpacking Given Python source with tuple and starred unpacking When the broken reference rule runs Then the semantic check result passed is true Scenario: BrokenReferenceRule passes for with-statement without as clause Given Python source with with-statement without as clause When the broken reference rule runs Then the semantic check result passed is true # ── APIMisuseRule safe attribute on flagged module ────────────── Scenario: APIMisuseRule passes for safe method call on flagged module Given Python source calling a safe method on a flagged module When the API misuse rule runs Then the semantic check result passed is true # ── MissingSymbolRule function-local binding patterns ────────── Scenario: MissingSymbolRule passes for function with all parameter kinds Given Python source with function using varargs kwonly and kwargs When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule passes for function with for-loop variable Given Python source with function containing for-loop variable When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule passes for function with with-statement variable Given Python source with function containing with-statement variable When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule passes for function with except handler variable Given Python source with function containing except handler variable When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule passes for function with local import Given Python source with function containing local import When the missing symbol rule runs Then the semantic check result passed is true Scenario: MissingSymbolRule passes for function with local from-import Given Python source with function containing local from-import When the missing symbol rule runs Then the semantic check result passed is true