CoreRasurae
7e6f6fae37
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 12s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 17s
CI / typecheck (pull_request) Successful in 31s
CI / security (pull_request) Successful in 31s
CI / unit_tests (pull_request) Successful in 1m46s
CI / docker (pull_request) Successful in 38s
CI / integration_tests (pull_request) Successful in 2m52s
CI / coverage (pull_request) Successful in 3m41s
CI / benchmark-regression (pull_request) Successful in 23m24s
CI / lint (push) Successful in 14s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 17s
CI / typecheck (push) Successful in 33s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 35s
CI / unit_tests (push) Successful in 2m13s
CI / docker (push) Successful in 38s
CI / integration_tests (push) Successful in 2m58s
CI / coverage (push) Successful in 3m44s
CI / benchmark-publish (push) Successful in 13m19s
Add comprehensive semantic validation test suites covering five
new fixture categories: language porting mismatches, dependency
graph violations, API surface changes, cross-file symbols, and
circular import detection.
New BDD scenarios (38) exercise all six built-in rules against
fixture-driven inputs. Robot Framework integration tests (11)
validate end-to-end rule execution via the SemanticValidationService.
ASV benchmarks (6 suites) establish performance baselines for
batch validation throughput and per-rule latency.
Files added:
- features/fixtures/validation/{language_porting_mismatches,
dependency_graph_violations, api_surface_changes,
cross_file_symbols, circular_import_detection}.json
- features/semantic_validation_suite.feature
- features/steps/semantic_validation_suite_steps.py
- robot/semantic_validation_suite.robot
- robot/helper_semantic_validation_suite.py
- benchmarks/semantic_validation_suite_bench.py
- docs/reference/semantic_validation_coverage.md
All 11 nox sessions pass (lint, format, typecheck, security_scan,
dead_code, unit_tests, integration_tests, docs, build, benchmark,
coverage_report). Coverage remains at 97%.
ISSUES CLOSED: #316
201 lines
9.6 KiB
Gherkin
201 lines
9.6 KiB
Gherkin
Feature: Semantic validation suite fixtures
|
|
As a plan executor
|
|
I want semantic validation suites covering porting mismatches, dependency violations,
|
|
API surface changes, cross-file symbols, and duplicate relative import detection
|
|
So that error patterns are caught and categorised before code reaches production
|
|
|
|
Background:
|
|
Given a semantic validation suite test environment
|
|
|
|
# ── Language-porting mismatches ─────────────────────────────────
|
|
|
|
Scenario: Porting fixture list_comprehension_side_effect passes api_misuse rule
|
|
Given the porting fixture "list_comprehension_side_effect"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture dynamic_attribute_access passes broken_reference rule
|
|
Given the porting fixture "dynamic_attribute_access"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture multiple_inheritance_diamond passes syntax_error rule
|
|
Given the porting fixture "multiple_inheritance_diamond"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture metaclass_usage passes syntax_error rule
|
|
Given the porting fixture "metaclass_usage"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture unpacking_star_expression passes broken_reference rule
|
|
Given the porting fixture "unpacking_star_expression"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture walrus_operator passes syntax_error rule
|
|
Given the porting fixture "walrus_operator"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture eval_in_porting_context fails api_misuse rule
|
|
Given the porting fixture "eval_in_porting_context"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Porting fixture exec_for_code_generation fails api_misuse rule
|
|
Given the porting fixture "exec_for_code_generation"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
# ── Dependency graph violations ─────────────────────────────────
|
|
|
|
Scenario: Dependency fixture duplicate_relative_imports fails duplicate_import rule
|
|
Given the dependency fixture "duplicate_relative_imports"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Dependency fixture triple_relative_import_cycle fails duplicate_import rule
|
|
Given the dependency fixture "triple_relative_import_cycle"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Dependency fixture mixed_valid_imports passes duplicate_import rule
|
|
Given the dependency fixture "mixed_valid_imports"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Dependency fixture private_module_import fails missing_import rule
|
|
Given the dependency fixture "private_module_import"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Dependency fixture from_private_module_import fails missing_import rule
|
|
Given the dependency fixture "from_private_module_import"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Dependency fixture valid_stdlib_private_import passes missing_import rule
|
|
Given the dependency fixture "valid_stdlib_private_import"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
# ── API surface changes ─────────────────────────────────────────
|
|
|
|
Scenario: API fixture renamed_function_reference fails broken_reference rule
|
|
Given the API surface fixture "renamed_function_reference"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture missing_symbol_in_function fails missing_symbol rule
|
|
Given the API surface fixture "missing_symbol_in_function"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture incompatible_type_usage fails broken_reference rule
|
|
Given the API surface fixture "incompatible_type_usage"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture valid_api_usage passes broken_reference rule
|
|
Given the API surface fixture "valid_api_usage"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture class_method_missing_symbol fails missing_symbol rule
|
|
Given the API surface fixture "class_method_missing_symbol"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture eval_based_dynamic_call fails api_misuse rule
|
|
Given the API surface fixture "eval_based_dynamic_call"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture aliased_pickle_not_caught documents known limitation
|
|
Given the API surface fixture "aliased_pickle_not_caught"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: API fixture from_import_alias_not_caught documents known limitation
|
|
Given the API surface fixture "from_import_alias_not_caught"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
# ── Cross-file symbol resolution ────────────────────────────────
|
|
|
|
Scenario: Cross-file fixture undefined_cross_module_reference fails broken_reference rule
|
|
Given the cross-file fixture "undefined_cross_module_reference"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Cross-file fixture properly_imported_cross_module passes broken_reference rule
|
|
Given the cross-file fixture "properly_imported_cross_module"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Cross-file fixture nested_function_scoped_symbol passes missing_symbol rule
|
|
Given the cross-file fixture "nested_function_scoped_symbol"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Cross-file fixture undefined_in_class_method fails missing_symbol rule
|
|
Given the cross-file fixture "undefined_in_class_method"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Cross-file fixture wildcard_import_symbol fails broken_reference rule
|
|
Given the cross-file fixture "wildcard_import_symbol"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Cross-file fixture deeply_nested_closure passes missing_symbol rule
|
|
Given the cross-file fixture "deeply_nested_closure"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
# ── Duplicate relative import detection ─────────────────────────
|
|
|
|
Scenario: Duplicate import fixture direct_circular_import fails duplicate_import rule
|
|
Given the circular import fixture "direct_circular_import"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Duplicate import fixture multiple_self_references fails duplicate_import rule
|
|
Given the circular import fixture "multiple_self_references"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Duplicate import fixture clean_relative_imports passes duplicate_import rule
|
|
Given the circular import fixture "clean_relative_imports"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Duplicate import fixture absolute_imports_not_flagged passes duplicate_import rule
|
|
Given the circular import fixture "absolute_imports_not_flagged"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
Scenario: Duplicate import fixture syntax_error_skips_check passes duplicate_import rule
|
|
Given the circular import fixture "syntax_error_skips_cycle_check"
|
|
When the suite runs the expected rule
|
|
Then the suite result matches the fixture expectation
|
|
|
|
# ── Full suite validation ───────────────────────────────────────
|
|
|
|
Scenario: All porting fixtures load and validate schema
|
|
Then all porting fixtures load without error
|
|
|
|
Scenario: All dependency fixtures load and validate schema
|
|
Then all dependency fixtures load without error
|
|
|
|
Scenario: All API surface fixtures load and validate schema
|
|
Then all API surface fixtures load without error
|
|
|
|
Scenario: All cross-file fixtures load and validate schema
|
|
Then all cross-file fixtures load without error
|
|
|
|
Scenario: All circular import fixtures load and validate schema
|
|
Then all circular import fixtures load without error
|