cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

test(e2e): workflow example 16 — devcontainer-driven development (supervised profile) #818

Merged
hurui200320 merged 1 commits from test/e2e-wf16-devcontainer into master 2026-03-30 11:13:30 +00:00
Conversation 7 Commits 1 Files Changed 3
+496 -9
3 changed files with 496 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+7
View File
@@ -296,6 +296,13 @@
violation reporting. Pipeline integration in `ACMSPipeline.assemble()`
applies enforcement as a pre-filter when a `context_view` is
provided. (#847)
- Added E2E test for Workflow Example 16: devcontainer-driven development
with supervised automation profile. Exercises devcontainer auto-detection
during resource registration, lazy container build during plan execution,
tool invocation routing to container workspace, and apply writing changes
back to host filesystem via bind mount. Uses dynamic actor selection
(Anthropic/OpenAI) and UUID-suffixed names for parallel CI safety.
(`robot/e2e/wf16_devcontainer.robot`) (#762)
- **Breaking (behavioral):** `SandboxManager.commit_all()` is now an
all-or-nothing atomic operation per specification line 45938. (#925)
- On partial failure, already-committed sandboxes are rolled back in
robot/e2e/wf16_devcontainer.robot
+473
View File
@@ -0,0 +1,473 @@
*** Settings ***
Documentation E2E test for Workflow Example 16: Devcontainer-Driven Development
... (supervised automation profile).
...
... Exercises the devcontainer-specific plan lifecycle:
... devcontainer auto-detection during resource registration,
... lazy container build during plan execution, tool invocation
... routing to the container workspace, and apply writing changes
... back to the host filesystem via bind mount.
...
... **Devcontainer-specific assertions** are strict and contribute
... to AC validation. If one or more AC indicators are missing,
... the test records each unmet AC and fails explicitly rather
... than skipping or passing silently.
...
... The test is tagged ``tdd_expected_fail`` because devcontainer
... features are not yet fully wired. The
... ``tdd_expected_fail_listener`` inverts the failure to a pass
... in CI until all AC indicators are present.
...
... Zero mocking — real CLI, real LLM API keys.
Resource common_e2e.resource
Suite Setup WF16 Suite Setup
Suite Teardown E2E Suite Teardown
Force Tags E2E
*** Variables ***
${ACTION_PREFIX} local/wf16-devcontainer-action
${RESOURCE_PREFIX} local/wf16-devcontainer-repo
${PROJECT_PREFIX} local/wf16-devcontainer-project
*** Keywords ***
WF16 Suite Setup
[Documentation] E2E Suite Setup plus database initialisation, unique suffix
... generation, and dynamic actor selection for WF16 tests.
E2E Suite Setup
# Initialise the database so CLI commands work in all tests.
# Use --force because the workspace may already contain an initialised project.
${init}= Run CleverAgents Command init --force --yes
Should Be Equal As Integers ${init.rc} 0
# Generate a unique suffix for resource/project names to avoid UNIQUE
# constraint collisions when tests run in parallel or are re-run
# against the same database (uuid4 provides ~4 billion possibilities).
${suffix}= Evaluate __import__('uuid').uuid4().hex[:12]
Set Suite Variable ${RUN_SUFFIX} ${suffix}
Set Suite Variable ${ACTION_NAME} ${ACTION_PREFIX}-${suffix}
Set Suite Variable ${RESOURCE_NAME} ${RESOURCE_PREFIX}-${suffix}
Set Suite Variable ${PROJECT_NAME} ${PROJECT_PREFIX}-${suffix}
# Pick an actor that matches available API keys.
# Prefer OpenAI first to reduce Anthropic credit-quota flakiness.
${has_openai}= Evaluate bool(__import__('os').environ.get('OPENAI_API_KEY', ''))
${has_anthropic}= Evaluate bool(__import__('os').environ.get('ANTHROPIC_API_KEY', ''))
IF ${has_openai}
# Use gpt-4o-mini for cost optimization — WF16 exercises plan lifecycle
# mechanics (not LLM quality), so a smaller model suffices.
${actor}= Set Variable openai/gpt-4o-mini
ELSE IF ${has_anthropic}
${actor}= Set Variable anthropic/claude-sonnet-4-20250514
ELSE
${actor}= Set Variable openai/gpt-4o-mini
END
Set Suite Variable ${SELECTED_ACTOR} ${actor}
Create Devcontainer Repo
[Documentation] Create a temp git repo with devcontainer configuration.
... Returns the path to the created repository.
${repo}= Create Temp Git Repo wf16-devcontainer-${RUN_SUFFIX}
Create Directory ${repo}${/}.devcontainer
Create Directory ${repo}${/}src
${devcontainer_json}= Catenate SEPARATOR=\n
... {
... ${SPACE}${SPACE}"name": "wf16-dev",
... ${SPACE}${SPACE}"image": "mcr.microsoft.com/devcontainers/python:3.12@sha256:3de8f04c3748897ff3aa32b11ee18306d6c56556f4d548a276d0ff397b28b9da",
... ${SPACE}${SPACE}"customizations": {
... ${SPACE}${SPACE}${SPACE}${SPACE}"vscode": {
... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}"extensions": ["ms-python.python"]
... ${SPACE}${SPACE}${SPACE}${SPACE}}
... ${SPACE}${SPACE}}
... }
Create File ${repo}${/}.devcontainer${/}devcontainer.json ${devcontainer_json}
${app_content}= Catenate SEPARATOR=\n
... """Main application module."""
... ${EMPTY}
... ${EMPTY}
... def main():
... ${SPACE}${SPACE}${SPACE}${SPACE}"""Entry point."""
... ${SPACE}${SPACE}${SPACE}${SPACE}print("Hello from devcontainer app")
... ${SPACE}${SPACE}${SPACE}${SPACE}return 0
Create File ${repo}${/}src${/}app.py ${app_content}
${test_content}= Catenate SEPARATOR=\n
... """Tests for the application."""
... from src.app import main
... ${EMPTY}
... ${EMPTY}
... def test_main():
... ${SPACE}${SPACE}${SPACE}${SPACE}assert main() == 0
Create File ${repo}${/}src${/}test_app.py ${test_content}
Create File ${repo}${/}src${/}__init__.py \n
Create File ${repo}${/}requirements.txt pytest>=7.0\n
${r_add}= Run Process git add . cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${r_add.rc} 0 msg=git add failed (rc=${r_add.rc})
${r_commit}= Run Process git commit -m Initial devcontainer project cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${r_commit.rc} 0 msg=git commit failed (rc=${r_commit.rc})
RETURN ${repo}
WF16 Test Teardown
[Documentation] Log diagnostic context on failure for debugging.
... Captures plan status when a plan ID is available.
${plan_id}= Get Variable Value ${WF16_PLAN_ID} ${EMPTY}
IF '${plan_id}' != ''
${status} ${result}= Run Keyword And Ignore Error
... Run CleverAgents Command plan status ${plan_id} --format json expected_rc=None timeout=30s
IF '${status}' == 'PASS'
Log Teardown plan status: ${result.stdout} WARN
END
END
*** Test Cases ***
WF16 Devcontainer Driven Development Supervised Profile
[Documentation] Supervised-profile workflow exercising devcontainer-specific
... behaviors: auto-detection during resource registration,
... lazy container build during execution, tool invocation
... routing to the container workspace, and apply writing
... changes back to the host filesystem via bind mount.
...
... Devcontainer-specific assertions are enforced via
... explicit AC checks. Missing AC indicators are collected
... and reported as explicit test failures.
...
... Tagged ``tdd_expected_fail`` because devcontainer features
... are not yet fully wired; the listener inverts the failure
... to a CI pass until all AC indicators are present.
[Tags] tdd_expected_fail tdd_issue tdd_issue_1208
# Timeout budget: 35 minutes test-level timeout. Per-step timeouts sum
# higher in theory, but retries and conditional paths are mutually
# exclusive — realistic worst-case is well under 35 minutes.
[Timeout] 35 minutes
[Teardown] WF16 Test Teardown
Skip If No LLM Keys
${ac_checks_missing}= Evaluate []
Set Test Variable ${WF16_PLAN_ID} ${EMPTY}
# ---- Create fixture repo with devcontainer ----
${repo}= Create Devcontainer Repo
${branch_result}= Run Process git rev-parse --abbrev-ref HEAD cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${branch_result.rc} 0 msg=git rev-parse failed (rc=${branch_result.rc})
${branch}= Strip String ${branch_result.stdout}
Log Fixture repo created at ${repo} on branch ${branch}
# ---- Register resource (AC-3: devcontainer auto-detection) ----
${r_res}= Run CleverAgents Command
... resource add git-checkout ${RESOURCE_NAME}
... --path ${repo} --branch ${branch}
Should Not Contain ${r_res.stdout} Traceback
Should Not Contain ${r_res.stderr} Traceback
Should Not Contain ${r_res.stdout} INTERNAL
Should Not Contain ${r_res.stderr} INTERNAL
Output Should Contain ${r_res} ${RESOURCE_NAME}
# AC-3: Verify devcontainer auto-detection indicators from resource add.
# Spec Example 16 shows "devcontainer" plus "detected (not built)".
${res_combined}= Set Variable ${r_res.stdout}\n${r_res.stderr}
${res_lower}= Evaluate ($res_combined).lower()
${has_devcontainer}= Evaluate 'devcontainer' in $res_lower
${has_detected}= Evaluate 'detected' in $res_lower
${has_not_built}= Evaluate 'not built' in $res_lower
${has_ac3}= Evaluate $has_devcontainer and $has_detected and $has_not_built
IF ${has_ac3}
Log AC-3 verified: resource output shows devcontainer detected (not built) state
ELSE
${ac_checks_missing}= Evaluate $ac_checks_missing + ['AC-3: resource add output missing explicit detected (not built) devcontainer state']
Log AC-3 unmet: missing explicit detected (not built) devcontainer state in resource output. AC-4, AC-5, and AC-6 are dependent on AC-3 and are likely to also fail. WARN
END
Log Resource registered: ${RESOURCE_NAME}
CoreRasurae commented 2026-03-24 11:30:31 +00:00
Outdated
Review
Copy Link
Copy Source

[LOW] Consider adding Output Should Contain ${r_action} ${ACTION_NAME} after this block for parity with m1_acceptance.robot:43, which verifies the action name in the output.

**[LOW]** Consider adding `Output Should Contain ${r_action} ${ACTION_NAME}` after this block for parity with `m1_acceptance.robot:43`, which verifies the action name in the output.
# ---- Create project ----
${r_proj}= Run CleverAgents Command
... project create ${PROJECT_NAME}
... --resource ${RESOURCE_NAME}
Should Not Contain ${r_proj.stdout} Traceback
Should Not Contain ${r_proj.stderr} Traceback
Should Not Contain ${r_proj.stdout} INTERNAL
Should Not Contain ${r_proj.stderr} INTERNAL
Output Should Contain ${r_proj} ${PROJECT_NAME}
Log Project created: ${PROJECT_NAME}
# ---- Create action (dynamic actor selection) ----
${action_yaml}= Catenate SEPARATOR=\n
... name: ${ACTION_NAME}
... description: Implement feature in devcontainer-enabled project
... definition_of_done: Feature implemented with tests passing
... strategy_actor: ${SELECTED_ACTOR}
... execution_actor: ${SELECTED_ACTOR}
... reusable: true
... read_only: false
${action_path}= Set Variable ${SUITE_HOME}${/}wf16_action.yaml
Create File ${action_path} ${action_yaml}
${r_action}= Run CleverAgents Command
... action create --config ${action_path}
Should Not Contain ${r_action.stdout} Traceback
Should Not Contain ${r_action.stderr} Traceback
Should Not Contain ${r_action.stdout} INTERNAL
Should Not Contain ${r_action.stderr} INTERNAL
Output Should Contain ${r_action} ${ACTION_NAME}
Log Action created: ${ACTION_NAME} with actor ${SELECTED_ACTOR}
# ---- Plan use with supervised automation profile ----
${r_use}= Run CleverAgents Command
... plan use ${ACTION_NAME} ${PROJECT_NAME}
... --automation-profile supervised
... --format json
... expected_rc=None timeout=180s
Should Be Equal As Integers ${r_use.rc} 0
... plan use failed (rc=${r_use.rc}); stderr redacted from assertion message
Should Not Contain ${r_use.stdout} Traceback
Should Not Contain ${r_use.stderr} Traceback
Should Not Contain ${r_use.stdout} INTERNAL
Should Not Contain ${r_use.stderr} INTERNAL
# Parse plan ID from structured JSON first to avoid selecting the wrong
# ULID when output contains multiple identifiers. Only use regex fallback
# if JSON parsing fails and the fallback result is unambiguous.
Output Should Contain ${r_use} plan_id
${plan_id}= Safe Parse Json Field ${r_use.stdout} plan_id
${plan_id}= Set Variable If $plan_id is None ${EMPTY} ${plan_id}
IF '${plan_id}' != ''
${plan_id}= Strip String ${plan_id}
Should Match Regexp ${plan_id} (?i)^[0-9A-HJKMNP-TV-Z]{26}$
ELSE
${plan_ids}= Get Regexp Matches ${r_use.stdout} [0-9A-HJKMNP-TV-Z]{26} flags=IGNORECASE
Should Not Be Empty ${plan_ids} msg=Expected plan ID in plan use output
${unique_plan_ids}= Evaluate list(dict.fromkeys($plan_ids))
${plan_id_count}= Evaluate len($unique_plan_ids)
Should Be Equal As Integers ${plan_id_count} 1
... msg=Ambiguous ULID fallback while extracting plan ID: ${unique_plan_ids}
${plan_id}= Set Variable ${unique_plan_ids}[0]
END
# Verify automation profile was accepted via JSON output
${resolved_profile}= Safe Parse Json Field ${r_use.stdout} automation_profile
${resolved_profile}= Set Variable If $resolved_profile is None ${EMPTY} ${resolved_profile}
IF '${resolved_profile}' != ''
Should Be Equal As Strings ${resolved_profile} supervised
ELSE
# Fallback: check combined output for "supervised" profile name
Output Should Contain ${r_use} supervised
END
Set Test Variable ${WF16_PLAN_ID} ${plan_id}
Log Plan created: ${plan_id} with supervised profile
# ---- Strategize + Execute ----
# The first plan execute call advances through all pending phases
# (Strategize → Execute) in a single invocation.
${r_strat_first}= Run CleverAgents Command
... plan execute ${plan_id}
... --format json
... expected_rc=None timeout=180s
${r_strat}= Set Variable ${r_strat_first}
IF ${r_strat_first.rc} != 0
Log First execute returned rc=${r_strat_first.rc}; retrying once WARN
${r_strat_retry}= Run CleverAgents Command
... plan execute ${plan_id}
... --format json
... expected_rc=None timeout=300s
IF ${r_strat_retry.rc} != 0
Fail WF16 execute instability after retry (first pass): rc1=${r_strat_first.rc}, rc2=${r_strat_retry.rc} (stderr redacted)
ELSE
${r_strat}= Set Variable ${r_strat_retry}
END
END
Should Not Contain ${r_strat.stdout} Traceback
Should Not Contain ${r_strat.stderr} Traceback
Should Not Contain ${r_strat.stdout} INTERNAL
Should Not Contain ${r_strat.stderr} INTERNAL
${strat_first_combined}= Set Variable ${r_strat_first.stdout}\n${r_strat_first.stderr}
${strat_first_lower}= Evaluate ($strat_first_combined).lower()
Log First execute completed: rc=${r_strat.rc}
# AC-4: Verify lazy devcontainer build indicators are present in the
# first execute call (proof of lazy build trigger on first execution).
${has_build_first}= Evaluate 'building' in $strat_first_lower and 'devcontainer' in $strat_first_lower
IF ${has_build_first}
Log AC-4 verified: first execute output shows lazy devcontainer build
ELSE
${ac_checks_missing}= Evaluate $ac_checks_missing + ['AC-4: first execute output missing lazy devcontainer build indicator']
CoreRasurae commented 2026-03-24 11:30:31 +00:00
Outdated
Review
Copy Link
Copy Source

[CRITICAL] Missing --yes flag. The specification Example 16 Step 5 explicitly uses agents plan apply --yes <PLAN_ID> (spec line 42536). All 24 plan apply examples in the spec use --yes. Without it, the command may prompt for confirmation, hanging the test until the 180s timeout.

Fix: Change to:

...    plan    apply    --yes    ${plan_id}
**[CRITICAL]** Missing `--yes` flag. The specification Example 16 Step 5 explicitly uses `agents plan apply --yes <PLAN_ID>` (spec line 42536). All 24 `plan apply` examples in the spec use `--yes`. Without it, the command may prompt for confirmation, hanging the test until the 180s timeout. **Fix:** Change to: ``` ... plan apply --yes ${plan_id} ```
Log AC-4 unmet: first execute output missing lazy devcontainer build indicator WARN
END
# Determine whether a second execute call is needed; only re-execute when
# the first execute did not reach a ready-for-apply state.
${r_status_after_first}= Run CleverAgents Command
... plan status ${plan_id}
... --format json
... expected_rc=None timeout=180s
Should Be Equal As Integers ${r_status_after_first.rc} 0
... plan status after first execute failed (rc=${r_status_after_first.rc}); stderr redacted from assertion message
${phase_after_first}= Safe Parse Json Field ${r_status_after_first.stdout} phase
${phase_after_first}= Set Variable If $phase_after_first is None ${EMPTY} ${phase_after_first}
${state_after_first}= Safe Parse Json Field ${r_status_after_first.stdout} processing_state
${state_after_first}= Set Variable If $state_after_first is None ${EMPTY} ${state_after_first}
${phase_after_first_norm}= Evaluate $phase_after_first.strip().lower()
${state_after_first_norm}= Evaluate $state_after_first.strip().lower()
${is_apply_phase}= Evaluate $phase_after_first_norm == 'apply'
${is_execute_complete}= Evaluate $phase_after_first_norm == 'execute' and $state_after_first_norm in ['complete', 'completed']
${is_already_applied}= Evaluate $state_after_first_norm == 'applied'
${ready_for_apply}= Evaluate $is_apply_phase or $is_execute_complete or $is_already_applied
${needs_second_execute}= Evaluate not $ready_for_apply
# ---- Defensive re-execute (AC-4: lazy container build, AC-5: container routing) ----
# The first call typically completes both Strategize and Execute phases.
# This second call is a defensive re-check — it is a no-op if both
# phases already completed, but ensures execution finishes if the first
# call only advanced through Strategize.
${r_exec}= Set Variable ${NONE}
IF ${needs_second_execute}
${r_exec}= Run CleverAgents Command
... plan execute ${plan_id}
... --format json
... expected_rc=None timeout=300s
IF ${r_exec.rc} != 0
Log Conditional re-check execute returned rc=${r_exec.rc}; retrying once WARN
${r_exec_retry}= Run CleverAgents Command
... plan execute ${plan_id}
... --format json
... expected_rc=None timeout=300s
IF ${r_exec_retry.rc} != 0
Fail WF16 execute instability after retry (conditional re-check): rc1=${r_exec.rc}, rc2=${r_exec_retry.rc} (stderr redacted)
ELSE
${r_exec}= Set Variable ${r_exec_retry}
END
END
Should Not Contain ${r_exec.stdout} Traceback
Should Not Contain ${r_exec.stderr} Traceback
Should Not Contain ${r_exec.stdout} INTERNAL
Should Not Contain ${r_exec.stderr} INTERNAL
Log Conditional second execute completed: rc=${r_exec.rc}
ELSE
Log Second execute skipped: first execute already reached ready-for-apply state
END
# Combine output from both execute calls — devcontainer indicators may
# appear in either invocation depending on which call runs each phase.
IF ${needs_second_execute}
${exec_combined}= Set Variable ${r_strat.stdout}\n${r_strat.stderr}\n${r_exec.stdout}\n${r_exec.stderr}
ELSE
${exec_combined}= Set Variable ${r_strat.stdout}\n${r_strat.stderr}
END
# AC-5: Require concrete routing evidence with both explicit devcontainer
# identity and explicit container workspace path evidence.
${has_dc_resource}= Evaluate bool(__import__('re').search(r'(?im)resource\s*:\s*[^\n]*\(devcontainer-instance\)', $exec_combined))
${has_dc_resolution}= Evaluate bool(__import__('re').search(r'(?im)resolved via\s*:\s*nearest-ancestor devcontainer', $exec_combined))
${has_dc_environment}= Evaluate bool(__import__('re').search(r'(?im)environment\s*:\s*devcontainer\b', $exec_combined))
${has_dc_identity}= Evaluate $has_dc_resource or ($has_dc_resolution and $has_dc_environment)
${has_workspace_header}= Evaluate bool(__import__('re').search(r'(?im)workspace\s*:\s*/workspaces?/', $exec_combined))
${has_workspace_tool_path}= Evaluate bool(__import__('re').search(r'(?im)in\s+container\s+/workspaces?/', $exec_combined))
${has_workspace_context}= Evaluate $has_workspace_header or $has_workspace_tool_path
${has_routing}= Evaluate $has_dc_identity and $has_workspace_context
IF ${has_routing}
Log AC-5 verified: devcontainer-specific routing indicators detected in execute output
ELSE
${ac_checks_missing}= Evaluate $ac_checks_missing + ['AC-5: execute output missing devcontainer-specific routing indicator']
Log AC-5 unmet: missing devcontainer-specific routing indicator in execute output WARN
END
IF ${needs_second_execute}
Log Execute completed: rc=${r_exec.rc}
ELSE
Log Execute completed in first pass (no second execute needed)
END
# ---- Diff (verify non-empty changeset) ----
${r_diff}= Run CleverAgents Command
... plan diff ${plan_id}
... --format json
... expected_rc=None timeout=180s
Should Be Equal As Integers ${r_diff.rc} 0
... plan diff failed (rc=${r_diff.rc}); stderr redacted from assertion message
Should Not Contain ${r_diff.stdout} Traceback
Should Not Contain ${r_diff.stderr} Traceback
Should Not Contain ${r_diff.stdout} INTERNAL
Should Not Contain ${r_diff.stderr} INTERNAL
Should Not Be Empty ${r_diff.stdout} msg=Plan diff produced no output — expected a changeset
Log Diff completed with non-empty output
# ---- Apply (AC-6: host filesystem write verification) ----
# Capture HEAD SHA before apply to detect new commits written by apply
${pre_apply_head}= Run Process git rev-parse HEAD cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${pre_apply_head.rc} 0 msg=git rev-parse HEAD failed before apply (rc=${pre_apply_head.rc})
${head_before}= Strip String ${pre_apply_head.stdout}
${pre_apply_status}= Run Process git status --porcelain cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${pre_apply_status.rc} 0 msg=git status --porcelain failed before apply (rc=${pre_apply_status.rc})
${worktree_before}= Strip String ${pre_apply_status.stdout}
Should Be Empty ${worktree_before} msg=Fixture repository must be clean before apply
Log HEAD before apply: ${head_before}
# Use ``plan apply`` (not ``lifecycle-apply``) so the plan drives
# through all three Apply sub-transitions synchronously:
# Execute/complete → Apply/queued → Apply/processing → Apply/applied.
# ``lifecycle-apply`` only transitions to Apply/queued; ``plan apply``
# with a plan ID calls ``_lifecycle_apply_with_id`` which completes
# the full transition to the terminal Apply/applied state.
${r_apply}= Run CleverAgents Command
... plan apply --yes --format json ${plan_id}
... expected_rc=None timeout=180s
Should Be Equal As Integers ${r_apply.rc} 0
... plan apply failed (rc=${r_apply.rc}); stderr redacted from assertion message
Should Not Contain ${r_apply.stdout} Traceback
Should Not Contain ${r_apply.stderr} Traceback
Should Not Contain ${r_apply.stdout} INTERNAL
Should Not Contain ${r_apply.stderr} INTERNAL
# Verify apply output contains the plan ID (meaningful verification
# that apply processed the correct plan).
Output Should Contain ${r_apply} ${plan_id}
# Verify the apply phase in JSON output (parse success + phase presence
# are mandatory before phase assertion).
${apply_phase}= Safe Parse Json Field ${r_apply.stdout} phase
${apply_phase}= Set Variable If $apply_phase is None ${EMPTY} ${apply_phase}
Should Not Be Empty ${apply_phase} msg=Could not parse non-empty phase from plan apply JSON
Should Contain ${apply_phase.lower()} apply Plan phase should indicate apply after plan apply
${post_apply_head}= Run Process git rev-parse HEAD cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${post_apply_head.rc} 0 msg=git rev-parse HEAD failed after apply (rc=${post_apply_head.rc})
${head_after}= Strip String ${post_apply_head.stdout}
${post_apply_status}= Run Process git status --porcelain cwd=${repo} timeout=60s on_timeout=kill
Should Be Equal As Integers ${post_apply_status.rc} 0 msg=git status --porcelain failed after apply (rc=${post_apply_status.rc})
${worktree_after}= Strip String ${post_apply_status.stdout}
Log HEAD after apply: ${head_after}
${apply_combined}= Set Variable ${r_apply.stdout}\n${r_apply.stderr}
${apply_lower}= Evaluate ($apply_combined).lower()
${has_bind_mount_mechanism}= Evaluate 'bind mount' in $apply_lower or 'bind-mount' in $apply_lower or 'bind_mount' in $apply_lower
${has_head_advance}= Evaluate $head_before != $head_after
${has_worktree_changes}= Evaluate bool($worktree_after.strip())
${has_host_mutation}= Evaluate $has_head_advance or $has_worktree_changes
${has_ac6}= Evaluate $has_bind_mount_mechanism and $has_host_mutation
IF ${has_ac6}
IF ${has_worktree_changes}
Log AC-6 host mutation evidence (git status --porcelain): ${worktree_after}
END
Log AC-6 verified: bind-mount mechanism signal and concrete host mutation both detected
ELSE
${ac_checks_missing}= Evaluate $ac_checks_missing + ['AC-6: missing bind-mount mechanism signal and/or host mutation evidence after apply']
Log AC-6 unmet: missing bind-mount mechanism signal and/or host mutation evidence after apply WARN
END
Log Apply completed: plan transitioned to apply phase
# ---- Status (terminal state verification) ----
${r_status}= Run CleverAgents Command
... plan status ${plan_id}
... --format json
... expected_rc=None timeout=180s
Should Be Equal As Integers ${r_status.rc} 0
... plan status failed (rc=${r_status.rc}); stderr redacted from assertion message
Should Not Contain ${r_status.stdout} Traceback
Should Not Contain ${r_status.stderr} Traceback
Should Not Contain ${r_status.stdout} INTERNAL
Should Not Contain ${r_status.stderr} INTERNAL
Should Not Be Empty ${r_status.stdout}
Output Should Contain ${r_status} ${plan_id}
# Verify success-only terminal state after apply.
${status_phase}= Safe Parse Json Field ${r_status.stdout} phase
${status_phase}= Set Variable If $status_phase is None ${EMPTY} ${status_phase}
Should Not Be Empty ${status_phase} msg=Could not parse non-empty phase from plan status JSON
Should Contain ${status_phase.lower()} apply
... Plan phase should remain apply after successful plan apply
${status_state}= Safe Parse Json Field ${r_status.stdout} processing_state
${status_state}= Set Variable If $status_state is None ${EMPTY} ${status_state}
IF '${status_state}' != ''
Log Final processing_state: ${status_state}
${state_lower}= Evaluate ($status_state).lower()
Should Be Equal As Strings ${state_lower} applied
... Expected success-only terminal state 'applied' but found: ${status_state}
ELSE
Fail Could not parse processing_state from plan status JSON
END
${missing_ac_count}= Evaluate len($ac_checks_missing)
IF ${missing_ac_count} > 0
${missing_ac_summary}= Evaluate '; '.join($ac_checks_missing)
Fail WF16 AC verification incomplete: ${missing_ac_summary}
END
Log Plan status verified: ${plan_id} in state ${status_state}
robot/resource_dag.robot
+16 -9
View File
@@ -23,8 +23,9 @@ Link Child And Verify Tree
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
... Base.metadata.create_all(engine)
... factory = sessionmaker(bind=engine)
... rt_repo = ResourceTypeRepository(factory)
... res_repo = ResourceRepository(factory)
... shared_session = factory()
... rt_repo = ResourceTypeRepository(lambda: shared_session)
... res_repo = ResourceRepository(lambda: shared_session)
... parent_spec = ResourceTypeSpec(name="robot/dag-parent", description="Parent", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/dag-child"], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
... child_spec = ResourceTypeSpec(name="robot/dag-child", description="Child", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=[], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
... rt_repo.create(parent_spec)
@@ -37,8 +38,9 @@ Link Child And Verify Tree
... children = res_repo.get_children("01HDAGR0B0T0000000PARENT01")
... assert len(children) == 1, f"Expected 1 child, got {len(children)}"
... assert children[0].resource_id == "01HDAGR0B0T00000CHXND00001"
... shared_session.close()
... print("Link child and verify tree passed")
${result}= Run Process ${PYTHON} -c ${script}
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
Should Be Equal As Integers ${result.rc} 0 Link test failed: ${result.stderr}
Should Contain ${result.stdout} Link child and verify tree passed
@@ -57,8 +59,9 @@ Cycle Detection Rejects A To B To A
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
... Base.metadata.create_all(engine)
... factory = sessionmaker(bind=engine)
... rt_repo = ResourceTypeRepository(factory)
... res_repo = ResourceRepository(factory)
... shared_session = factory()
... rt_repo = ResourceTypeRepository(lambda: shared_session)
... res_repo = ResourceRepository(lambda: shared_session)
... spec = ResourceTypeSpec(name="robot/cycle-type", description="Cycle", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/cycle-type"], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
... rt_repo.create(spec)
... a = Resource(resource_id="01HDAGCYC000000000000000A1", name=None, resource_type_name="robot/cycle-type", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
@@ -71,7 +74,9 @@ Cycle Detection Rejects A To B To A
... ${SPACE * 4}assert False, "Should have raised CycleDetectedError"
... except CycleDetectedError:
... ${SPACE * 4}print("Cycle detection passed")
${result}= Run Process ${PYTHON} -c ${script}
... finally:
... ${SPACE * 4}shared_session.close()
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
Should Be Equal As Integers ${result.rc} 0 Cycle test failed: ${result.stderr}
Should Contain ${result.stdout} Cycle detection passed
@@ -91,8 +96,9 @@ Auto Discover Children
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
... Base.metadata.create_all(engine)
... factory = sessionmaker(bind=engine)
... rt_repo = ResourceTypeRepository(factory)
... res_repo = ResourceRepository(factory)
... shared_session = factory()
... rt_repo = ResourceTypeRepository(lambda: shared_session)
... res_repo = ResourceRepository(lambda: shared_session)
... parent_spec = ResourceTypeSpec(name="robot/disc-parent", description="Discoverer", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/disc-child"], auto_discovery={"enabled": True, "rules": [{"type": "robot/disc-child", "pattern": "*"}]}, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
... child_spec = ResourceTypeSpec(name="robot/disc-child", description="Discovered", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=[], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
... rt_repo.create(parent_spec)
@@ -104,8 +110,9 @@ Auto Discover Children
... assert created[0].resource_type_name == "robot/disc-child"
... children = res_repo.get_children("01HDAGR0B0TDSC00PARENT0001")
... assert len(children) >= 1, f"Expected >=1 linked children, got {len(children)}"
... shared_session.close()
... print("Auto discover children passed")
${result}= Run Process ${PYTHON} -c ${script}
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
Should Be Equal As Integers ${result.rc} 0 Auto discover test failed: ${result.stderr}
Should Contain ${result.stdout} Auto discover children passed