CoreRasurae
7e6f6fae37
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 12s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 17s
CI / typecheck (pull_request) Successful in 31s
CI / security (pull_request) Successful in 31s
CI / unit_tests (pull_request) Successful in 1m46s
CI / docker (pull_request) Successful in 38s
CI / integration_tests (pull_request) Successful in 2m52s
CI / coverage (pull_request) Successful in 3m41s
CI / benchmark-regression (pull_request) Successful in 23m24s
CI / lint (push) Successful in 14s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 17s
CI / typecheck (push) Successful in 33s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 35s
CI / unit_tests (push) Successful in 2m13s
CI / docker (push) Successful in 38s
CI / integration_tests (push) Successful in 2m58s
CI / coverage (push) Successful in 3m44s
CI / benchmark-publish (push) Successful in 13m19s
Add comprehensive semantic validation test suites covering five
new fixture categories: language porting mismatches, dependency
graph violations, API surface changes, cross-file symbols, and
circular import detection.
New BDD scenarios (38) exercise all six built-in rules against
fixture-driven inputs. Robot Framework integration tests (11)
validate end-to-end rule execution via the SemanticValidationService.
ASV benchmarks (6 suites) establish performance baselines for
batch validation throughput and per-rule latency.
Files added:
- features/fixtures/validation/{language_porting_mismatches,
dependency_graph_violations, api_surface_changes,
cross_file_symbols, circular_import_detection}.json
- features/semantic_validation_suite.feature
- features/steps/semantic_validation_suite_steps.py
- robot/semantic_validation_suite.robot
- robot/helper_semantic_validation_suite.py
- benchmarks/semantic_validation_suite_bench.py
- docs/reference/semantic_validation_coverage.md
All 11 nox sessions pass (lint, format, typecheck, security_scan,
dead_code, unit_tests, integration_tests, docs, build, benchmark,
coverage_report). Coverage remains at 97%.
ISSUES CLOSED: #316
77 lines
4.0 KiB
Plaintext
77 lines
4.0 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for semantic validation fixture suites
|
|
Library Process
|
|
Library OperatingSystem
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER_SCRIPT} robot/helper_semantic_validation_suite.py
|
|
|
|
*** Test Cases ***
|
|
Suite Loads All Porting Fixtures
|
|
[Documentation] Verify all language-porting mismatch fixtures load and validate
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} porting_fixtures cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} porting-fixtures-ok
|
|
|
|
Suite Loads All Dependency Fixtures
|
|
[Documentation] Verify all dependency graph violation fixtures load and validate
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} dependency_fixtures cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} dependency-fixtures-ok
|
|
|
|
Suite Loads All API Surface Fixtures
|
|
[Documentation] Verify all API surface change fixtures load and validate
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} api_fixtures cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} api-fixtures-ok
|
|
|
|
Suite Loads All Cross-File Fixtures
|
|
[Documentation] Verify all cross-file symbol resolution fixtures load and validate
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} cross_file_fixtures cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} cross-file-fixtures-ok
|
|
|
|
Suite Loads All Circular Import Fixtures
|
|
[Documentation] Verify all circular import detection fixtures load and validate
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} circular_fixtures cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} circular-fixtures-ok
|
|
|
|
Suite Runs Porting Fixture Through Rules
|
|
[Documentation] Run a porting fixture through its expected rule and verify result
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} run_porting cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} run-porting-ok
|
|
|
|
Suite Runs Dependency Fixture Through Rules
|
|
[Documentation] Run a dependency fixture through its expected rule and verify result
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} run_dependency cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} run-dependency-ok
|
|
|
|
Suite Runs API Surface Fixture Through Rules
|
|
[Documentation] Run an API surface fixture through its expected rule and verify result
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} run_api cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} run-api-ok
|
|
|
|
Suite Runs Cross-File Fixture Through Rules
|
|
[Documentation] Run a cross-file fixture through its expected rule and verify result
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} run_cross_file cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} run-cross-file-ok
|
|
|
|
Suite Runs Circular Import Fixture Through Rules
|
|
[Documentation] Run a circular import fixture through its expected rule and verify result
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} run_circular cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} run-circular-ok
|
|
|
|
Suite Unknown Command Fails Gracefully
|
|
[Documentation] Unknown command returns error exit code
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} nonexistent_command cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 1
|