HAL9000
03d2df26ce
CI / lint (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 56s
CI / build (pull_request) Successful in 54s
CI / typecheck (pull_request) Successful in 1m21s
CI / security (pull_request) Successful in 1m21s
CI / push-validation (pull_request) Successful in 30s
CI / helm (pull_request) Successful in 49s
CI / integration_tests (pull_request) Successful in 10m11s
CI / unit_tests (pull_request) Successful in 11m31s
CI / docker (pull_request) Successful in 2m54s
CI / coverage (pull_request) Successful in 12m17s
CI / status-check (pull_request) Successful in 3s
The shared `format_data` serializer introduced for the CLI→Application
A2A boundary returns raw payloads without the `{"data": ...}` envelope
that the legacy CLI `format_output` wraps around. Two test-step
definitions (`step_artifacts_json_validation`,
`step_artifacts_json_apply_summary`) still unwrapped that envelope and
crashed with `KeyError: 'data'`, errrring the Behave scenarios
`Plan artifacts shows validation results when available` and
`Artifacts include apply summary from metadata`.
Also remove the stale `@tdd_expected_fail` tag from the Robot scenario
`WF02 Mocked Generation Produces Test Artifacts Only`: the scenario
exercises the `_cleveragents/plan/artifacts` A2A dispatch path that this
PR added and now passes naturally; the `tdd_expected_fail_listener`
inverts the passing result to a failure with "Bug appears to be fixed.
Remove the tdd_expected_fail tag".
Adds a CHANGELOG entry covering both the boundary refactor and these
test alignments.
Refs: #9962
Refs: #4253
57 lines
3.1 KiB
Plaintext
57 lines
3.1 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration test for Workflow Example 2: automated test generation
|
|
... for a module using the trusted automation profile.
|
|
...
|
|
... Validates trusted-profile lifecycle behavior, mocked LLM-based
|
|
... deterministic test generation, coverage validation enforcement,
|
|
... and invariant-safe artifacts (test-only file changes).
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment With Database Isolation
|
|
Suite Teardown Cleanup Test Environment
|
|
Force Tags wf02 integration trusted workflow2 test-generation
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_wf02_test_generation.py
|
|
|
|
*** Test Cases ***
|
|
WF02 Trusted Profile Auto Runs Strategize And Execute
|
|
[Documentation] Verify trusted profile auto-runs strategize and execute
|
|
... while not auto-applying (human approval gate retained).
|
|
[Tags] tdd_issue tdd_issue_4296 lifecycle automation-profile trusted
|
|
${result}= Run Process ${PYTHON} ${HELPER} trusted-lifecycle cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf02-trusted-lifecycle-ok
|
|
|
|
WF02 Mocked Generation Produces Test Artifacts Only
|
|
[Documentation] Verify mocked LLM responses generate deterministic test
|
|
... files, assert user-facing plan artifacts command path,
|
|
... and artifacts list only test paths (no prod edits).
|
|
[Tags] tdd_issue tdd_issue_4296 mocking artifacts invariants
|
|
${result}= Run Process ${PYTHON} ${HELPER} mocked-generation-artifacts cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf02-mocked-generation-artifacts-ok
|
|
|
|
WF02 Coverage Validation Enforced For Target Threshold
|
|
[Documentation] Verify required coverage validation blocks below-threshold
|
|
... runs and passes after deterministic generated tests.
|
|
[Tags] tdd_issue tdd_issue_4296 validation coverage
|
|
${result}= Run Process ${PYTHON} ${HELPER} coverage-validation cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf02-coverage-validation-ok
|
|
|
|
WF02 Path Safety Guardrails Reject Unsafe Destinations
|
|
[Documentation] Verify generated-file path guardrails reject absolute,
|
|
... traversal, and non-tests destinations.
|
|
[Tags] tdd_issue tdd_issue_4296 guardrails path-safety
|
|
${result}= Run Process ${PYTHON} ${HELPER} path-safety-guardrails cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} wf02-path-safety-guardrails-ok
|