fix(e2e): add timeout, setup keyword, and deduplicate temp directory in workflow tests
CI / lint (pull_request) Successful in 39s
CI / helm (pull_request) Successful in 39s
CI / push-validation (pull_request) Successful in 38s
CI / typecheck (pull_request) Successful in 58s
CI / quality (pull_request) Successful in 1m1s
CI / unit_tests (pull_request) Successful in 5m41s
CI / integration_tests (pull_request) Failing after 9m57s
CI / build (pull_request) Failing after 15m5s
CI / security (pull_request) Failing after 15m6s
CI / coverage (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled

Address PR #10614 review findings:
- Added [Timeout] 30 minutes to all 9 test cases to prevent CI hangs
- Removed duplicate Create Temp Directory keyword from each file (moved to common_e2e.resource)
- Added Set Up E2E Project Test keyword for consistent temp dir creation via TEST NAME variable
- Added msg= parameters to Should Not Be Empty assertions for better debugging

Signed-off-by: HAL9000 <hal9000@noreply.git.cleverthis.com>
This commit is contained in:
2026-05-13 08:29:23 +00:00
committed by drew
parent 59f8f73f70
commit 77bddecedd
4 changed files with 69 additions and 60 deletions
+18
View File
@@ -292,6 +292,24 @@ Create Synthetic Codebase
${large_content}= Evaluate "# auto-generated large file\\n" + ("x = 1\\n" * 250)
Create File ${base_dir}${/}large_file.py ${large_content}
Create Temp Directory
[Documentation] Create a temporary directory for testing.
...
... Uses Python's tempfile module to create an isolated
... temp directory with the given name prefix. Ideal for
... project creation tests that need their own workspace.
[Arguments] ${name}=${EMPTY}
${temp_dir}= Evaluate __import__("tempfile").mkdtemp(prefix="${name}-")
RETURN ${temp_dir}
Remove Temp Directory
[Documentation] Remove a temporary directory created for testing.
...
... Safely removes the directory and all contents recursively,
... ignoring errors if the directory does not exist.
[Arguments] ${dir}
Run Keyword And Ignore Error Remove Directory ${dir} recursive=True
Create Temp Git Repo
[Documentation] Create a temporary git repository for E2E testing.
...
+17 -20
View File
@@ -14,6 +14,7 @@ Suite Teardown E2E Suite Teardown
Force Tags E2E
*** Test Cases ***
Correction Revert Mode Workflow
[Documentation] Test plan correction in revert mode.
...
@@ -21,15 +22,14 @@ Correction Revert Mode Workflow
... then corrects it using revert mode to restore
... the plan to a previous state.
[Tags] correction-revert
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory correction-revert-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name correction-revert-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name correction-revert-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} correction-revert-project
Correction Append Mode Workflow
@@ -39,15 +39,14 @@ Correction Append Mode Workflow
... then corrects it using append mode to add corrections
... to the existing plan.
[Tags] correction-append
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory correction-append-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name correction-append-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name correction-append-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} correction-append-project
Correction State Transition Validation
@@ -56,20 +55,18 @@ Correction State Transition Validation
... Verifies that correction workflows properly transition
... plan state through correction phases.
[Tags] correction-state
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory correction-state-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name correction-state-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name correction-state-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} correction-state-project
*** Keywords ***
Create Temp Directory
[Documentation] Create a temporary directory for testing.
[Arguments] ${name}
${temp_dir}= Evaluate __import__("tempfile").mkdtemp(prefix="${name}-")
RETURN ${temp_dir}
Set Up E2E Project Test
[Documentation] Set up a fresh temp directory for the test case.
${project_dir}= Create Temp Directory ${TEST NAME}
Set Suite Variable ${PROJECT_DIR} ${project_dir}
+17 -20
View File
@@ -15,22 +15,22 @@ Suite Teardown E2E Suite Teardown
Force Tags E2E
*** Test Cases ***
Project Creation Workflow
[Documentation] Test complete project creation workflow.
...
... Creates a new project via CLI, verifies project
... is created successfully, and validates output format.
[Tags] project-creation
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory project-creation-test
# Initialize a new project
${result}= Run CleverAgents Command project create --name test-project --path ${project_dir}
Should Not Be Empty ${result.stdout}
Should Not Be Empty ${result.stdout} msg=project create returned empty output
Output Should Contain ${result} test-project
# Verify project was created
${list_result}= Run CleverAgents Command project list
Should Not Be Empty ${list_result.stdout}
Should Not Be Empty ${list_result.stdout} msg=project list returned empty output
Output Should Contain ${list_result} test-project
Actor Setup Workflow
@@ -39,15 +39,14 @@ Actor Setup Workflow
... Creates a project, adds actors to it, and verifies
... actors are properly registered.
[Tags] actor-setup
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory actor-setup-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name actor-test-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# List actors in the project
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${list_result}= Run CleverAgents Command actor list
Should Not Be Empty ${list_result.stdout}
Should Not Be Empty ${list_result.stdout} msg=actor list returned empty output
Plan Execution Workflow
[Documentation] Test complete plan execution workflow.
@@ -55,20 +54,18 @@ Plan Execution Workflow
... Creates a project, generates a plan, and executes it
... through the complete workflow.
[Tags] plan-execution
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory plan-execution-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name plan-test-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project creation
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name plan-test-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} plan-test-project
*** Keywords ***
Create Temp Directory
[Documentation] Create a temporary directory for testing.
[Arguments] ${name}
${temp_dir}= Evaluate __import__("tempfile").mkdtemp(prefix="${name}-")
RETURN ${temp_dir}
Set Up E2E Project Test
[Documentation] Set up a fresh temp directory for the test case.
${project_dir}= Create Temp Directory ${TEST NAME}
Set Suite Variable ${PROJECT_DIR} ${project_dir}
+17 -20
View File
@@ -14,21 +14,21 @@ Suite Teardown E2E Suite Teardown
Force Tags E2E
*** Test Cases ***
Subplan Spawning Workflow
[Documentation] Test subplan spawning from parent plan.
...
... Creates a project, generates a plan, and spawns
... subplans from the parent plan.
[Tags] subplan-spawn
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory subplan-spawn-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name subplan-spawn-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name subplan-spawn-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} subplan-spawn-project
Three-Way Merge Workflow
@@ -37,15 +37,14 @@ Three-Way Merge Workflow
... Creates a project, generates a plan with subplans,
... executes them, and performs three-way merge.
[Tags] three-way-merge
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory three-way-merge-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name three-way-merge-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name three-way-merge-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} three-way-merge-project
Merge Result Validation
@@ -54,20 +53,18 @@ Merge Result Validation
... Verifies that merge results are properly validated
... and conform to specification requirements.
[Tags] merge-validation
[Timeout] 30 minutes
[Setup] Set Up E2E Project Test
Skip If No LLM Keys
# Create a temporary directory for the project
${project_dir}= Create Temp Directory merge-validation-test
# Initialize a new project
${create_result}= Run CleverAgents Command project create --name merge-validation-project --path ${project_dir}
Should Not Be Empty ${create_result.stdout}
# Verify project was created
Should Not Be Empty ${create_result.stdout} msg=project create returned empty output
${show_result}= Run CleverAgents Command project show --name merge-validation-project
Should Not Be Empty ${show_result.stdout}
Should Not Be Empty ${show_result.stdout} msg=project show returned empty output
Output Should Contain ${show_result} merge-validation-project
*** Keywords ***
Create Temp Directory
[Documentation] Create a temporary directory for testing.
[Arguments] ${name}
${temp_dir}= Evaluate __import__("tempfile").mkdtemp(prefix="${name}-")
RETURN ${temp_dir}
Set Up E2E Project Test
[Documentation] Set up a fresh temp directory for the test case.
${project_dir}= Create Temp Directory ${TEST NAME}
Set Suite Variable ${PROJECT_DIR} ${project_dir}