forked from cleveragents/cleveragents-core
brent.edwards
3eecb79003
TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839
150 lines
8.1 KiB
Plaintext
150 lines
8.1 KiB
Plaintext
*** Settings ***
|
|
Documentation M4 end-to-end verification: subplan spawning, parallel execution,
|
|
... plan tree viewing, three-way merge, conflict surfacing,
|
|
... parent plan subplan status tracking, and CLI error paths.
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment With Database Isolation
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_m4_e2e_verification.py
|
|
|
|
*** Test Cases ***
|
|
Plan Spawns Multiple Subplans During Execute
|
|
[Documentation] Execute a parent plan that spawns multiple subplans.
|
|
... Verifies subplans are created with correct parent_plan_id
|
|
... and that the parent's subplan_statuses list is populated.
|
|
${result}= Run Process ${PYTHON} ${HELPER} spawn-subplans cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-spawn-subplans-ok
|
|
|
|
View Subplan Tree Via Plan Tree
|
|
[Documentation] Verify SubplanStatus field completeness and parent-child
|
|
... linkage for the plan tree data model.
|
|
${result}= Run Process ${PYTHON} ${HELPER} plan-tree cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-plan-tree-ok
|
|
|
|
Verify Merged Results Via Plan Diff
|
|
[Documentation] View merged results via agents plan diff.
|
|
... Verifies the diff service returns unified diff output
|
|
... after subplan execution and merging.
|
|
${result}= Run Process ${PYTHON} ${HELPER} plan-diff cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-plan-diff-ok
|
|
|
|
Parallel Subplan Execution With Max Parallel
|
|
[Documentation] Verify parallel subplan execution respects max_parallel.
|
|
... Creates a SubplanConfig with PARALLEL mode and max_parallel=3
|
|
... and verifies concurrent scheduling constraints.
|
|
${result}= Run Process ${PYTHON} ${HELPER} parallel-max cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-parallel-max-ok
|
|
|
|
Three Way Merge Combines Non Conflicting Changes
|
|
[Documentation] Verify three-way merge combines non-conflicting changes
|
|
... from parallel subplans using GitMergeStrategy.
|
|
${result}= Run Process ${PYTHON} ${HELPER} merge-clean cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-merge-clean-ok
|
|
|
|
Merge Conflicts Are Surfaced Correctly
|
|
[Documentation] Verify merge conflicts from parallel subplans are
|
|
... detected and reported with conflict markers.
|
|
${result}= Run Process ${PYTHON} ${HELPER} merge-conflict cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-merge-conflict-ok
|
|
|
|
Parent Plan Tracks All Subplan Statuses
|
|
[Documentation] Verify parent plan tracks all subplan statuses through
|
|
... lifecycle transitions including completion, failure,
|
|
... and retry attempts.
|
|
${result}= Run Process ${PYTHON} ${HELPER} parent-tracking cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-parent-tracking-ok
|
|
|
|
CLI Plan Use Creates Plan With Subplan Config
|
|
[Documentation] Invoke agents plan use local/refactor-action local/monorepo
|
|
... via the actual CLI (Typer CliRunner) with a mocked lifecycle
|
|
... service and verify the plan is created with subplan config.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-use cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-use-ok
|
|
|
|
CLI Plan Execute Transitions With Subplans
|
|
[Documentation] Invoke agents plan execute <plan_id> via the actual CLI
|
|
... (Typer CliRunner) with a mocked lifecycle service and
|
|
... verify the plan transitions to Execute phase with subplans.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-execute cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-execute-ok
|
|
|
|
CLI Plan Tree Displays Subplan Hierarchy
|
|
[Documentation] Invoke agents plan tree <plan_id> --format json via the
|
|
... actual CLI (Typer CliRunner) with mocked Decision objects
|
|
... forming a subplan tree. Verify JSON output contains
|
|
... subplan_spawn and subplan_parallel_spawn decision types.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-tree cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-tree-ok
|
|
|
|
CLI Plan Execute Aborts On Read Only Plan
|
|
[Documentation] Invoke agents plan execute on a read-only plan.
|
|
... Verifies the CLI aborts with an appropriate error message
|
|
... and does not call execute_plan.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-execute-readonly cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-execute-readonly-ok
|
|
|
|
CLI Plan Use Aborts On Unavailable Action
|
|
[Documentation] Invoke agents plan use with a nonexistent/unavailable action.
|
|
... Verifies the CLI aborts with an appropriate error message
|
|
... and does not call use_action.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-use-not-found cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-use-not-found-ok
|
|
|
|
CLI Plan Diff Aborts On Missing Changeset
|
|
[Documentation] Invoke agents plan diff when the plan has no changeset.
|
|
... Verifies the CLI catches PlanError and aborts with a
|
|
... user-friendly error message.
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-diff-no-changeset cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-diff-no-changeset-ok
|
|
|
|
CLI Plan Tree Handles Zero Decisions Gracefully
|
|
[Documentation] Invoke agents plan tree when no decisions exist.
|
|
... Verifies the CLI prints an informational message and
|
|
... exits cleanly (exit code 0).
|
|
${result}= Run Process ${PYTHON} ${HELPER} cli-plan-tree-empty cwd=${WORKSPACE} timeout=120s on_timeout=kill
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} m4-cli-plan-tree-empty-ok
|