CoreRasurae
80f1664a0d
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 23s
CI / lint (pull_request) Successful in 3m20s
CI / typecheck (pull_request) Successful in 3m54s
CI / security (pull_request) Successful in 4m13s
CI / quality (pull_request) Successful in 4m17s
CI / integration_tests (pull_request) Successful in 9m21s
CI / unit_tests (pull_request) Successful in 9m42s
CI / docker (pull_request) Successful in 14s
CI / e2e_tests (pull_request) Successful in 12m1s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 26s
CI / lint (push) Successful in 3m19s
CI / quality (push) Successful in 3m41s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m2s
CI / integration_tests (push) Successful in 9m3s
CI / unit_tests (push) Successful in 9m24s
CI / docker (push) Successful in 1m9s
CI / e2e_tests (push) Successful in 13m51s
CI / coverage (push) Successful in 11m25s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Failing after 24m52s
CI / benchmark-regression (pull_request) Failing after 37m6s
Implemented Robot Framework integration test suite validating the CI/CD workflow (Specification Example 7). Tests cover ci-profile configuration (automation-profile, format, log level), idempotent resource and project registration with duplicate-detection assertions, three validation tools (ci-lint, ci-typecheck, ci-tests) registration and resource attachment via ToolRegistryService, action creation with typed arguments and invariants per spec Step 2, plan lifecycle with phase-by-phase completion through all phases (strategize, execute, apply) until terminal applied state, and JSON output structure verification including plan_id, phase, state, action, projects, and arguments fields. Post-review fixes applied (round 1): - H1: Fix truncated invariant #2 text to match spec line 39051 - H2: Fix definition_of_done to match spec's 5-bullet-point format - M1: Register all 3 spec validations (ci-lint, ci-typecheck, ci-tests) - M3: Add branch property to resource registration per spec Step 3 - M4: Replace phantom resource_id with real registered resource - M5: Add projects and arguments field assertions to JSON output test - M7: Replace 'polling-based' with 'phase-by-phase' in docs/comments - L1: Use timezone-aware datetime (timezone.utc) - L2: Use tempfile for resource path instead of hardcoded /tmp - L3: Clarify json_output() docstring to reflect as_cli_dict() scope - L4: Tighten attachment count assertion from >= 1 to == 1 (now == 3) Post-review fixes applied (round 2): - M2: Use Setup Test Environment With Database Isolation for pabot safety - L1: Replace remaining hardcoded /tmp paths with tempfile (validation_attach, resource_idempotent duplicate registration) - L2: Fix stale 'Polling-based' comment missed in round 1 - L5: Align action description with spec line 39027 - M1/L3/L6: Document automation_profile propagation gap in use_action() with TODO for when production code wires the field onto Plan - L4: Add comment explaining local actor name deviation from spec Includes CHANGELOG update describing the integration test scope. ISSUES CLOSED: #771
75 lines
3.8 KiB
Plaintext
75 lines
3.8 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration test for Workflow Example 7: CI/CD integration,
|
|
... automated PR review and fix (ci profile).
|
|
...
|
|
... Validates the CI/CD automation workflow covering: ci-profile
|
|
... configuration, idempotent resource and project registration,
|
|
... validation registration and attachment, plan lifecycle with
|
|
... ci automation profile and phase-by-phase completion through
|
|
... all phases until terminal applied state, and JSON output
|
|
... structure verification.
|
|
Library Process
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment With Database Isolation
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_wf07_cicd.py
|
|
|
|
*** Test Cases ***
|
|
WF07 CI Profile Config Set And Get
|
|
[Documentation] Set and verify ci automation profile, json format, and log level via ConfigService.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} config-ci-profile cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-config-ci-profile-ok
|
|
|
|
WF07 Idempotent Resource Registration
|
|
[Documentation] Register a git-checkout resource twice and verify only one exists.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} resource-idempotent cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-resource-idempotent-ok
|
|
|
|
WF07 Idempotent Project Registration
|
|
[Documentation] Create a project twice and verify it exists exactly once.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} project-idempotent cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-project-idempotent-ok
|
|
|
|
WF07 Validation Registration And Attachment
|
|
[Documentation] Register three validation tools, attach to a resource, and verify pipeline execution.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} validation-attach cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-validation-attach-ok
|
|
|
|
WF07 CI Plan Lifecycle
|
|
[Documentation] Create an action with arguments and invariants, run plan lifecycle with
|
|
... phase-by-phase completion through all phases, then verify applied and cancelled terminal states.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} ci-plan-lifecycle cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-ci-plan-lifecycle-ok
|
|
|
|
WF07 JSON Output Parsing
|
|
[Documentation] Run commands with JSON output and verify structure including plan_id,
|
|
... phase, state, and action fields.
|
|
[Tags] cicd integration workflow7
|
|
${result}= Run Process ${PYTHON} ${HELPER} json-output cwd=${WORKSPACE} timeout=30s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf07-json-output-ok
|