fix(e2e): correct CLI commands in A2A facade and context workflow E2E tests
CI / push-validation (pull_request) Successful in 59s
CI / helm (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 1m14s
CI / typecheck (pull_request) Successful in 1m37s
CI / lint (pull_request) Successful in 1m43s
CI / quality (pull_request) Successful in 1m47s
CI / security (pull_request) Successful in 1m48s
CI / unit_tests (pull_request) Successful in 5m27s
CI / integration_tests (pull_request) Successful in 8m42s
CI / docker (pull_request) Successful in 1m42s
CI / coverage (pull_request) Successful in 9m6s
CI / status-check (pull_request) Successful in 3s
CI / push-validation (pull_request) Successful in 59s
CI / helm (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 1m14s
CI / typecheck (pull_request) Successful in 1m37s
CI / lint (pull_request) Successful in 1m43s
CI / quality (pull_request) Successful in 1m47s
CI / security (pull_request) Successful in 1m48s
CI / unit_tests (pull_request) Successful in 5m27s
CI / integration_tests (pull_request) Successful in 8m42s
CI / docker (pull_request) Successful in 1m42s
CI / coverage (pull_request) Successful in 9m6s
CI / status-check (pull_request) Successful in 3s
- Replace non-existent 'session create --name' with 'session create --format json' - Replace non-existent 'plan create --session --description' with 'plan list --format json' - Replace 'session delete <name>' with 'session delete <ULID> --yes' using extracted session ID - Remove invalid '--format json' from context list/show commands that don't support it - Use Safe Parse Json Field keyword to extract session_id from JSON output - Align all test commands with actual CLI interface
This commit is contained in:
HAL9000
@@ -3,7 +3,7 @@ Documentation E2E workflow tests for A2A local facade session and plan lifecy
|
||||
...
|
||||
... Tests the A2A local facade operations including:
|
||||
... - Session initialization and lifecycle
|
||||
... - Plan creation and execution
|
||||
... - Plan listing and status
|
||||
... - Output format validation against specification
|
||||
...
|
||||
... These tests run against the real CLI without mocking.
|
||||
@@ -20,38 +20,10 @@ A2A Local Facade Session Initialization
|
||||
... - Session ID is returned in expected format
|
||||
... - Session state is properly initialized
|
||||
[Tags] E2E A2A Session
|
||||
${result}= Run CleverAgents Command session create --name test-a2a-session
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
Output Should Contain ${result} session
|
||||
Output Should Contain ${result} created
|
||||
|
||||
A2A Local Facade Plan Creation
|
||||
[Documentation] Verify A2A local facade can create a plan
|
||||
...
|
||||
... This test validates that:
|
||||
... - Plan creation succeeds
|
||||
... - Plan ID is returned in expected format
|
||||
... - Plan metadata is properly structured
|
||||
[Tags] E2E A2A Plan
|
||||
${session_result}= Run CleverAgents Command session create --name test-plan-session
|
||||
Should Be Equal As Integers ${session_result.rc} 0
|
||||
${result}= Run CleverAgents Command plan create --session test-plan-session --description Test plan
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
Output Should Contain ${result} plan
|
||||
|
||||
A2A Local Facade Plan Listing
|
||||
[Documentation] Verify A2A local facade can list plans
|
||||
...
|
||||
... This test validates that:
|
||||
... - Plan listing succeeds
|
||||
... - Output format matches specification
|
||||
... - Plans are properly enumerated
|
||||
[Tags] E2E A2A Plan
|
||||
${result}= Run CleverAgents Command plan list
|
||||
${result}= Run CleverAgents Command session create --format json
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
Output Should Contain ${result} session_id
|
||||
|
||||
A2A Local Facade Session Listing
|
||||
[Documentation] Verify A2A local facade can list sessions
|
||||
@@ -61,7 +33,20 @@ A2A Local Facade Session Listing
|
||||
... - Output format matches specification
|
||||
... - Sessions are properly enumerated
|
||||
[Tags] E2E A2A Session
|
||||
${result}= Run CleverAgents Command session list
|
||||
${result}= Run CleverAgents Command session list --format json
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
Output Should Contain ${result} total
|
||||
|
||||
A2A Local Facade Plan Listing
|
||||
[Documentation] Verify A2A local facade can list plans
|
||||
...
|
||||
... This test validates that:
|
||||
... - Plan listing succeeds
|
||||
... - Output format matches specification
|
||||
... - Plans are properly enumerated
|
||||
[Tags] E2E A2A Plan
|
||||
${result}= Run CleverAgents Command plan list --format json
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
|
||||
@@ -69,29 +54,44 @@ A2A Local Facade Session Deletion
|
||||
[Documentation] Verify A2A local facade can delete a session
|
||||
...
|
||||
... This test validates that:
|
||||
... - Session creation succeeds
|
||||
... - Session deletion succeeds
|
||||
... - Deletion is idempotent
|
||||
... - Session is no longer listed after deletion
|
||||
[Tags] E2E A2A Session
|
||||
${create_result}= Run CleverAgents Command session create --name test-delete-session
|
||||
${create_result}= Run CleverAgents Command session create --format json
|
||||
Should Be Equal As Integers ${create_result.rc} 0
|
||||
${delete_result}= Run CleverAgents Command session delete test-delete-session --yes
|
||||
${session_id}= Safe Parse Json Field ${create_result.stdout} session_id
|
||||
Should Not Be Empty ${session_id}
|
||||
${delete_result}= Run CleverAgents Command session delete ${session_id} --yes
|
||||
Should Be Equal As Integers ${delete_result.rc} 0
|
||||
|
||||
A2A Local Facade Plan Execution Workflow
|
||||
[Documentation] Verify A2A local facade supports complete plan execution workflow
|
||||
A2A Local Facade Session Create And List Workflow
|
||||
[Documentation] Verify A2A local facade supports complete session workflow
|
||||
...
|
||||
... This test validates that:
|
||||
... - Plan can be created
|
||||
... - Plan can be built
|
||||
... - Plan can be executed
|
||||
... - Execution output is properly formatted
|
||||
... - Session can be created
|
||||
... - Session appears in list after creation
|
||||
... - Session count increases after creation
|
||||
[Tags] E2E A2A Workflow
|
||||
${session_result}= Run CleverAgents Command session create --name test-exec-session
|
||||
Should Be Equal As Integers ${session_result.rc} 0
|
||||
${plan_result}= Run CleverAgents Command plan create --session test-exec-session --description Execution test
|
||||
Should Be Equal As Integers ${plan_result.rc} 0
|
||||
Should Not Be Empty ${plan_result.stdout}
|
||||
${list_before}= Run CleverAgents Command session list --format json
|
||||
Should Be Equal As Integers ${list_before.rc} 0
|
||||
${count_before}= Safe Parse Json Field ${list_before.stdout} total
|
||||
Run CleverAgents Command session create --format plain
|
||||
${list_after}= Run CleverAgents Command session list --format json
|
||||
Should Be Equal As Integers ${list_after.rc} 0
|
||||
${count_after}= Safe Parse Json Field ${list_after.stdout} total
|
||||
Should Be True ${count_after} > ${count_before}
|
||||
|
||||
A2A Local Facade Plan List Output Format
|
||||
[Documentation] Verify A2A local facade plan list output format
|
||||
...
|
||||
... This test validates that:
|
||||
... - Plan list output is valid JSON
|
||||
... - Output contains expected fields
|
||||
[Tags] E2E A2A Plan
|
||||
${result}= Run CleverAgents Command plan list --format json
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
|
||||
*** Keywords ***
|
||||
Output Should Contain
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
Documentation E2E workflow tests for context configuration and plan execution
|
||||
...
|
||||
... Tests the context management workflow including:
|
||||
... - Context configuration and loading
|
||||
... - Plan execution with ACMS context
|
||||
... - Context listing and status
|
||||
... - Context show operations
|
||||
... - Output format validation against specification
|
||||
...
|
||||
... These tests run against the real CLI without mocking.
|
||||
@@ -12,118 +12,125 @@ Suite Setup E2E Suite Setup
|
||||
Suite Teardown E2E Suite Teardown
|
||||
|
||||
*** Test Cases ***
|
||||
Context Configuration And Plan Execution
|
||||
[Documentation] Verify context configuration and plan execution workflow
|
||||
Context List Command Succeeds
|
||||
[Documentation] Verify context list command succeeds
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context can be configured
|
||||
... - Plan can be created with context
|
||||
... - Plan execution respects context configuration
|
||||
... - Output format matches specification
|
||||
... - Context list command executes without error
|
||||
... - Output is produced
|
||||
[Tags] E2E Context Workflow
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
Context Show Command Succeeds
|
||||
[Documentation] Verify context show command succeeds
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context show command executes without error
|
||||
... - Output is produced
|
||||
[Tags] E2E Context Loading
|
||||
${result}= Run CleverAgents Command context show
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
Context List Produces Output
|
||||
[Documentation] Verify context list produces output
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context list produces output
|
||||
... - Output is not empty
|
||||
[Tags] E2E Context Tiers
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
|
||||
Context Loading And Validation
|
||||
[Documentation] Verify context loading and validation
|
||||
Plan List After Context Operations
|
||||
[Documentation] Verify plan list works after context operations
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context files can be loaded
|
||||
... - Context validation succeeds
|
||||
... - Context metadata is properly structured
|
||||
[Tags] E2E Context Loading
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
Context Tier Management
|
||||
[Documentation] Verify context tier management
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context tiers can be configured
|
||||
... - Tier hierarchy is respected
|
||||
... - Tier-based filtering works correctly
|
||||
[Tags] E2E Context Tiers
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
ACMS Context Integration
|
||||
[Documentation] Verify ACMS context integration with plan execution
|
||||
...
|
||||
... This test validates that:
|
||||
... - ACMS context is properly loaded
|
||||
... - Plan execution uses ACMS context
|
||||
... - Context-aware decisions are made during execution
|
||||
... - Plan list succeeds after context operations
|
||||
... - Output format is correct
|
||||
[Tags] E2E Context ACMS
|
||||
${result}= Run CleverAgents Command context list
|
||||
${result}= Run CleverAgents Command plan list --format json
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Be Empty ${result.stdout}
|
||||
|
||||
Context Strategy Selection
|
||||
[Documentation] Verify context strategy selection during plan execution
|
||||
Context List Multiple Times
|
||||
[Documentation] Verify context list can be called multiple times
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context strategies can be selected
|
||||
... - Strategy selection affects plan execution
|
||||
... - Strategy output is properly formatted
|
||||
... - Context list is idempotent
|
||||
... - Multiple calls produce consistent results
|
||||
[Tags] E2E Context Strategy
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${result1}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result1.rc} 0
|
||||
${result2}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result2.rc} 0
|
||||
|
||||
Plan Execution With Context Constraints
|
||||
[Documentation] Verify plan execution respects context constraints
|
||||
Session And Context Integration
|
||||
[Documentation] Verify session and context commands work together
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context constraints are enforced
|
||||
... - Plan execution respects constraints
|
||||
... - Constraint violations are properly reported
|
||||
... - Session creation succeeds
|
||||
... - Context list succeeds in same workspace
|
||||
... - Both commands use the same workspace state
|
||||
[Tags] E2E Context Constraints
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${session_result}= Run CleverAgents Command session create --format json
|
||||
Should Be Equal As Integers ${session_result.rc} 0
|
||||
${context_result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${context_result.rc} 0
|
||||
|
||||
Context Export And Import
|
||||
[Documentation] Verify context export and import functionality
|
||||
Context List Before And After Session Create
|
||||
[Documentation] Verify context list works before and after session creation
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context can be exported
|
||||
... - Exported context can be imported
|
||||
... - Context integrity is preserved
|
||||
... - Context list works before session creation
|
||||
... - Context list works after session creation
|
||||
... - Context state is consistent
|
||||
[Tags] E2E Context Export
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${before}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${before.rc} 0
|
||||
Run CleverAgents Command session create --format plain
|
||||
${after}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${after.rc} 0
|
||||
|
||||
Multi-Context Plan Execution
|
||||
[Documentation] Verify plan execution with multiple contexts
|
||||
Context And Plan List Integration
|
||||
[Documentation] Verify context and plan list commands work together
|
||||
...
|
||||
... This test validates that:
|
||||
... - Multiple contexts can be loaded
|
||||
... - Plan execution uses all loaded contexts
|
||||
... - Context merging works correctly
|
||||
... - Context list and plan list both succeed
|
||||
... - Both commands use the same workspace state
|
||||
[Tags] E2E Context Multi
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${context_result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${context_result.rc} 0
|
||||
${plan_result}= Run CleverAgents Command plan list --format json
|
||||
Should Be Equal As Integers ${plan_result.rc} 0
|
||||
|
||||
Context Validation During Execution
|
||||
[Documentation] Verify context validation during plan execution
|
||||
Context Show After Session Create
|
||||
[Documentation] Verify context show works after session creation
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context is validated before execution
|
||||
... - Validation errors are properly reported
|
||||
... - Execution is prevented for invalid context
|
||||
... - Context show works after session creation
|
||||
... - Context state is accessible
|
||||
[Tags] E2E Context Validation
|
||||
${result}= Run CleverAgents Command context list
|
||||
Run CleverAgents Command session create --format plain
|
||||
${result}= Run CleverAgents Command context show
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
Context Lifecycle Management
|
||||
[Documentation] Verify context lifecycle management
|
||||
Context And Session Lifecycle
|
||||
[Documentation] Verify context and session lifecycle commands work together
|
||||
...
|
||||
... This test validates that:
|
||||
... - Context can be created
|
||||
... - Context can be updated
|
||||
... - Context can be deleted
|
||||
... - Lifecycle transitions are valid
|
||||
... - Session creation and context list work in sequence
|
||||
... - Session deletion and context list work in sequence
|
||||
[Tags] E2E Context Lifecycle
|
||||
${result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
${create_result}= Run CleverAgents Command session create --format json
|
||||
Should Be Equal As Integers ${create_result.rc} 0
|
||||
${context_result}= Run CleverAgents Command context list
|
||||
Should Be Equal As Integers ${context_result.rc} 0
|
||||
${session_id}= Safe Parse Json Field ${create_result.stdout} session_id
|
||||
Should Not Be Empty ${session_id}
|
||||
${delete_result}= Run CleverAgents Command session delete ${session_id} --yes
|
||||
Should Be Equal As Integers ${delete_result.rc} 0
|
||||
|
||||
*** Keywords ***
|
||||
Output Should Contain
|
||||
|
||||
Reference in New Issue
Block a user