brent.edwards
3eecb79003
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 46s
CI / typecheck (pull_request) Successful in 56s
CI / security (pull_request) Successful in 56s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 29s
CI / e2e_tests (pull_request) Successful in 2m7s
CI / unit_tests (pull_request) Successful in 3m8s
CI / docker (pull_request) Successful in 9s
CI / integration_tests (pull_request) Successful in 3m57s
CI / coverage (pull_request) Successful in 7m48s
CI / benchmark-regression (pull_request) Successful in 40m48s
TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839
60 lines
2.9 KiB
Plaintext
60 lines
2.9 KiB
Plaintext
*** Settings ***
|
|
Documentation Scale test fixture validation and performance baseline checks.
|
|
... Validates fixture loading, metadata structure, threshold matrices,
|
|
... and profile generation simulation.
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_scale_test.py
|
|
|
|
*** Test Cases ***
|
|
Load Scale Metadata Fixture
|
|
[Documentation] Load scale_metadata.json and verify structure
|
|
${result}= Run Process ${PYTHON} ${HELPER} load-metadata cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-metadata-ok
|
|
|
|
Validate Scale Metadata Profiles
|
|
[Documentation] Verify all expected scale profiles are present with correct fields
|
|
${result}= Run Process ${PYTHON} ${HELPER} validate-profiles cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-profiles-ok
|
|
|
|
Load And Validate Baseline Thresholds
|
|
[Documentation] Load baseline_thresholds.json and verify threshold matrix structure
|
|
${result}= Run Process ${PYTHON} ${HELPER} validate-thresholds cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-thresholds-ok
|
|
|
|
Verify Threshold Monotonicity
|
|
[Documentation] Verify p50 < p95 < p99 across all threshold entries
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-monotonicity cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-monotonicity-ok
|
|
|
|
Simulate Profile File Distribution
|
|
[Documentation] Generate file count distributions and verify they match language mix
|
|
${result}= Run Process ${PYTHON} ${HELPER} simulate-distribution cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-distribution-ok
|
|
|
|
Validate Generator Instructions Exist
|
|
[Documentation] Verify the generator instructions document is present
|
|
${result}= Run Process ${PYTHON} ${HELPER} check-generator-docs cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} scale-generator-docs-ok
|