CoreRasurae
a2be3e67b0
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 13s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 59s
CI / integration_tests (pull_request) Successful in 4m40s
CI / unit_tests (pull_request) Successful in 11m32s
CI / docker (pull_request) Successful in 1m1s
CI / benchmark-regression (pull_request) Successful in 28m54s
CI / coverage (pull_request) Successful in 44m11s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 16s
CI / typecheck (push) Successful in 31s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 42s
CI / integration_tests (push) Successful in 2m53s
CI / unit_tests (push) Successful in 11m20s
CI / docker (push) Successful in 9s
CI / benchmark-publish (push) Successful in 12m18s
CI / coverage (push) Successful in 43m33s
Added SemanticValidationService with built-in Python checks for syntax errors, missing imports, broken references, dependency cycles, API misuse, and missing symbols. Implemented SemanticRuleRegistry for extensible validator registration. Integrated with ValidationPipeline as informational by default. Added per-project/per-plan config keys, severity mapping (info/warn/error), output schema normalization, and file-hash-based caching. Includes Behave BDD scenarios, Robot integration tests, ASV benchmarks, and reference documentation. ISSUES CLOSED: #207
108 lines
5.6 KiB
Plaintext
108 lines
5.6 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for the semantic validation service
|
|
Library Process
|
|
Library OperatingSystem
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER_SCRIPT} robot/helper_semantic_validation.py
|
|
|
|
*** Test Cases ***
|
|
Semantic Validation Models Are Valid
|
|
[Documentation] Verify SemanticCheckResult and SemanticValidationSeverity models
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} models cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} models-ok
|
|
|
|
Semantic Validation Syntax Check Passes For Valid Code
|
|
[Documentation] SyntaxCheckRule passes for valid Python source
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} syntax_pass cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} syntax-pass-ok
|
|
|
|
Semantic Validation Syntax Check Fails For Invalid Code
|
|
[Documentation] SyntaxCheckRule fails for code with syntax errors
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} syntax_fail cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} syntax-fail-ok
|
|
|
|
Semantic Validation Registry Has Built-in Rules
|
|
[Documentation] Default registry contains all six built-in rules
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} registry cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} registry-ok
|
|
|
|
Semantic Validation Cache Works Correctly
|
|
[Documentation] Cache stores and retrieves results by rule name and hash
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} cache cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} cache-ok
|
|
|
|
Semantic Validation Service Runs All Rules
|
|
[Documentation] SemanticValidationService.check_file runs all registered rules
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} service_check cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} service-check-ok
|
|
|
|
Semantic Validation Severity Mapping
|
|
[Documentation] Severity levels map correctly to validation modes
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} severity_mapping cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} severity-mapping-ok
|
|
|
|
Semantic Validation Pipeline Integration
|
|
[Documentation] as_pipeline_results returns normalised dicts with expected keys
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} pipeline_integration cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} pipeline-integration-ok
|
|
|
|
Semantic Validation Config Disabled
|
|
[Documentation] Service returns empty results when disabled via config
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} config_disabled cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} config-disabled-ok
|
|
|
|
Semantic Validation API Misuse Detection
|
|
[Documentation] APIMisuseRule detects eval() usage
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} api_misuse cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} api-misuse-ok
|
|
|
|
Semantic Validation Broken Reference Scope Aware
|
|
[Documentation] BrokenReferenceRule does not false-positive on function-local variables
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} broken_ref_scope cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} broken-ref-scope-ok
|
|
|
|
Semantic Validation Missing Symbol Detects Class Methods
|
|
[Documentation] MissingSymbolRule detects undefined symbols inside class methods
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} missing_symbol_class cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} missing-symbol-class-ok
|
|
|
|
Semantic Validation Cache LRU Eviction
|
|
[Documentation] Cache evicts least-recently-used entries when max_size exceeded
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} cache_lru cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} cache-lru-ok
|
|
|
|
Semantic Validation Non Python Files Skipped
|
|
[Documentation] Service returns empty results for non-Python files
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} non_python_skip cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} non-python-skip-ok
|
|
|
|
Semantic Validation Duplicate Import Rule Alias
|
|
[Documentation] DuplicateImportRule is accessible via DependencyCycleRule backward-compat alias
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} duplicate_import_alias cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} duplicate-import-alias-ok
|
|
|
|
Semantic Validation Unknown Command Returns Error
|
|
[Documentation] Running with unknown command exits with code 1
|
|
${result}= Run Process ${PYTHON} ${HELPER_SCRIPT} nonexistent_cmd cwd=${WORKSPACE}
|
|
Should Be Equal As Integers ${result.rc} 1
|
|
Should Contain ${result.stderr} Unknown command
|