forked from cleveragents/cleveragents-core
brent.edwards
3eecb79003
TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839
98 lines
4.7 KiB
Plaintext
98 lines
4.7 KiB
Plaintext
*** Settings ***
|
|
Documentation M6 autonomy acceptance E2E smoke tests — A2A facade, guardrails, audit
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_m6_autonomy_acceptance.py
|
|
|
|
*** Test Cases ***
|
|
M6 A2A Facade Session Lifecycle
|
|
[Documentation] Dispatch session.create and session.close via local facade
|
|
${result}= Run Process ${PYTHON} ${HELPER} facade-session cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-facade-session-ok
|
|
|
|
M6 A2A Facade Plan Lifecycle
|
|
[Documentation] Dispatch plan create/execute/status/diff/apply operations
|
|
${result}= Run Process ${PYTHON} ${HELPER} facade-plan cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-facade-plan-ok
|
|
|
|
M6 A2A Facade Unknown Operation Error
|
|
[Documentation] Verify unknown operations raise A2aOperationNotFoundError
|
|
${result}= Run Process ${PYTHON} ${HELPER} facade-unknown-op cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-facade-unknown-op-ok
|
|
|
|
M6 A2A Event Queue Publish Subscribe
|
|
[Documentation] Publish events and verify local subscriber receives them
|
|
${result}= Run Process ${PYTHON} ${HELPER} event-queue cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-event-queue-ok
|
|
|
|
M6 A2A Transport Stub Rejects All
|
|
[Documentation] Verify HTTP transport stub raises A2aNotAvailableError
|
|
${result}= Run Process ${PYTHON} ${HELPER} transport-stub cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-transport-stub-ok
|
|
|
|
M6 A2A Version Negotiation
|
|
[Documentation] Negotiate supported and unsupported A2A versions
|
|
${result}= Run Process ${PYTHON} ${HELPER} version-negotiation cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-version-negotiation-ok
|
|
|
|
M6 Guard Denylist Enforcement
|
|
[Documentation] Verify denylist guard blocks denied tools
|
|
${result}= Run Process ${PYTHON} ${HELPER} guard-denylist cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-guard-denylist-ok
|
|
|
|
M6 Guard Budget Enforcement
|
|
[Documentation] Verify cost budget and call limit guards work
|
|
${result}= Run Process ${PYTHON} ${HELPER} guard-budget cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-guard-budget-ok
|
|
|
|
M6 Profile Resolution Precedence
|
|
[Documentation] Verify plan > action > project > global resolution
|
|
${result}= Run Process ${PYTHON} ${HELPER} profile-resolution cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-profile-resolution-ok
|
|
|
|
M6 Fixture Loading
|
|
[Documentation] Load all M6 fixture files and verify structure
|
|
${result}= Run Process ${PYTHON} ${HELPER} fixture-loading cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-fixture-loading-ok
|
|
|
|
M6 Full Autonomy Acceptance Flow
|
|
[Documentation] End-to-end: facade dispatch + guard check + profile resolution
|
|
${result}= Run Process ${PYTHON} ${HELPER} full-flow cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m6-full-flow-ok
|