85f8970d00
CI / build (push) Successful in 21s
CI / quality (push) Has been cancelled
CI / security (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
Robot Framework integration test for Specification Workflow Example 8: Cloud Infrastructure Management with the supervised automation profile. Test suite exercises: - Custom resource type registration (local/terraform-state) via YAML fixture with copy_on_write sandbox strategy and CLI args validation - Custom skill creation (local/terraform-ops) with 3 anonymous tools (terraform_plan, terraform_show, cloud_metrics) and skill composition via includes (local/file-ops) - Supervised profile gating: verifies both strategize-to-execute (create_tool=1.0) and execute-to-apply (select_tool=1.0) transitions require explicit human approval via should_auto_progress() - Action creation with supervised profile, typed arguments (STRING, FLOAT), and invariant propagation to plans via InvariantSource.ACTION - Mocked infrastructure analysis producing right-sizing recommendations for over-provisioned resources while skipping critical instances - Invariant enforcement blocking modifications to resources tagged critical:true while allowing non-critical changes Fixture files provide deterministic mock Terraform state (3 AWS resources), CloudWatch metrics (CPU at 12% avg on m5.2xlarge), and sample HCL configuration. ISSUES CLOSED: #772 Reviewed-by: freemo (reviewer-pool-1) Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
91 lines
5.2 KiB
Plaintext
91 lines
5.2 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration test for Workflow Example 8: Cloud Infrastructure
|
|
... Management with the supervised automation profile.
|
|
...
|
|
... Validates custom resource types (local/terraform-state), custom
|
|
... skills with tool composition (local/terraform-ops), supervised
|
|
... profile gating, infrastructure analysis with mocked Terraform
|
|
... operations, optimization recommendations, and invariant
|
|
... enforcement.
|
|
...
|
|
... Spec reference: docs/specification.md, Example 8
|
|
... Issue: #772
|
|
Library Process
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_wf08_cloud_infra_supervised.py
|
|
|
|
*** Test Cases ***
|
|
WF08 Register Terraform Resource Type
|
|
[Documentation] Register local/terraform-state custom resource type
|
|
... from YAML fixture with filesystem_copy sandbox strategy,
|
|
... cli_args, and resource instance creation.
|
|
[Tags] wf08 resource registration integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} register-terraform-resource-type cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Registration failed: ${result.stderr}
|
|
Should Contain ${result.stdout} register-terraform-resource-type-ok
|
|
|
|
WF08 Register Terraform Skill With Tool Composition
|
|
[Documentation] Register local/terraform-ops skill with 3 anonymous tools
|
|
... (terraform_plan, terraform_show, cloud_metrics) and skill
|
|
... composition via includes (local/file-ops).
|
|
[Tags] wf08 skill composition integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} register-terraform-skill cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Skill registration failed: ${result.stderr}
|
|
Should Contain ${result.stdout} register-terraform-skill-ok
|
|
|
|
WF08 Supervised Profile Gating Behavior
|
|
[Documentation] Verify the supervised automation profile gates both
|
|
... strategize-to-execute and execute-to-apply transitions,
|
|
... requiring explicit human approval at each step.
|
|
[Tags] wf08 profile supervised integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} supervised-profile-behavior cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Profile check failed: ${result.stderr}
|
|
Should Contain ${result.stdout} supervised-profile-behavior-ok
|
|
|
|
WF08 Create Infra Optimize Action With Invariants
|
|
[Documentation] Create local/infra-optimize action with supervised profile,
|
|
... two arguments (optimization_targets, min_savings_threshold),
|
|
... two invariants, and verify plan creation with correct state
|
|
... and invariant propagation.
|
|
[Tags] wf08 action plan invariants integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} create-infra-optimize-action cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Action/plan failed: ${result.stderr}
|
|
Should Contain ${result.stdout} create-infra-optimize-action-ok
|
|
|
|
WF08 Infrastructure Analysis Produces Optimization Recommendations
|
|
[Documentation] Run mocked infrastructure analysis using Terraform state
|
|
... and cloud metrics fixtures. Verify that over-provisioned
|
|
... resources receive right-sizing recommendations and that
|
|
... critical resources are skipped.
|
|
[Tags] wf08 analysis recommendations integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} infra-analysis-recommendations cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Analysis failed: ${result.stderr}
|
|
Should Contain ${result.stdout} infra-analysis-recommendations-ok
|
|
|
|
WF08 Invariant Enforcement Blocks Critical Resource Deletion
|
|
[Documentation] Verify that action invariants propagate to plans and
|
|
... that the invariant "Never delete or modify resources
|
|
... tagged with 'critical: true'" blocks modifications to
|
|
... critical infrastructure while allowing changes to
|
|
... non-critical resources.
|
|
[Tags] wf08 invariants enforcement integration
|
|
${result}= Run Process ${PYTHON} ${HELPER} invariant-enforcement cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0 Invariant enforcement failed: ${result.stderr}
|
|
Should Contain ${result.stdout} invariant-enforcement-ok
|