*** Settings *** Documentation E2E test for Workflow Example 18: Container with Remote Repo ... Clone (trusted profile). ... ... Exercises resource registration with container-instance and ... --clone-into flag for remote repo clone, two-step project ... creation and linking, action with trusted automation profile, ... plan-level execution environment with fallback priority, and ... full plan lifecycle including apply with container commit/push. ... ... Zero mocking — real CLI, real LLM API keys. Resource common_e2e.resource Suite Setup WF18 Suite Setup Suite Teardown E2E Suite Teardown Force Tags E2E *** Variables *** ${ACTION_PREFIX} local/wf18-clone-action ${RESOURCE_PREFIX} local/wf18-clone-res ${PROJECT_PREFIX} local/wf18-clone-proj *** Keywords *** WF18 Suite Setup [Documentation] E2E Suite Setup plus unique suffix generation and dynamic ... actor selection for WF18 tests. E2E Suite Setup # Initialise the workspace so CLI commands work. ${init}= Run CleverAgents Command init --force --yes Should Be Equal As Integers ${init.rc} 0 # Generate a unique suffix for resource/project names to avoid UNIQUE # constraint collisions on repeated E2E runs or parallel CI. ${suffix}= Evaluate __import__('uuid').uuid4().hex[:12] Set Suite Variable ${RUN_SUFFIX} ${suffix} # Pick an actor that matches the available API key. ${has_anthropic}= Evaluate bool(__import__('os').environ.get('ANTHROPIC_API_KEY', '')) IF ${has_anthropic} ${actor}= Set Variable anthropic/claude-sonnet-4-20250514 ELSE ${actor}= Set Variable openai/gpt-4o END Set Suite Variable ${LLM_ACTOR} ${actor} # Compute unique action name using the run suffix for parallel CI safety. ${action_name}= Set Variable ${ACTION_PREFIX}-${RUN_SUFFIX} Set Suite Variable ${ACTION_NAME} ${action_name} Create Remote Clone Repo [Documentation] Create a temp git repo simulating a project that would ... be cloned from a remote repository into a container. ${repo}= Create Temp Git Repo wf18-remote-clone-${RUN_SUFFIX} Create Directory ${repo}${/}src Create Directory ${repo}${/}deploy ${app_content}= Catenate SEPARATOR=\n ... """Application for remote deployment via container clone.""" ... ${EMPTY} ... ${EMPTY} ... class DeploymentManager: ... ${SPACE}${SPACE}${SPACE}${SPACE}"""Manages deployment lifecycle.""" ... ${EMPTY} ... ${SPACE}${SPACE}${SPACE}${SPACE}def __init__(self, repo_url, branch="main"): ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}self.repo_url = repo_url ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}self.branch = branch ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}self.deployed = False ... ${EMPTY} ... ${SPACE}${SPACE}${SPACE}${SPACE}def deploy(self): ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}"""Execute deployment.""" ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}self.deployed = True ... ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}return {"status": "deployed", "branch": self.branch} Create File ${repo}${/}src${/}deploy_manager.py ${app_content} ${deploy_config}= Catenate SEPARATOR=\n ... # Deployment configuration ... CONTAINER_IMAGE=python:3.12-slim ... CLONE_DEPTH=1 ... DEPLOY_TIMEOUT=300 ... HEALTH_CHECK_INTERVAL=10 Create File ${repo}${/}deploy${/}config.env ${deploy_config} # NOTE: This Dockerfile is fixture data only — never built by this test. # It simulates a project that would be cloned into a container. ${dockerfile}= Catenate SEPARATOR=\n ... FROM python:3.12-slim ... RUN apt-get update && apt-get install -y git ... WORKDIR /workspace ... ARG REPO_URL ... RUN git clone --depth 1 \${REPO_URL} . ... CMD ["python", "-m", "src.deploy_manager"] Create File ${repo}${/}Dockerfile ${dockerfile} Create File ${repo}${/}src${/}__init__.py \n ${git_add}= Run Process git add . cwd=${repo} Should Be Equal As Integers ${git_add.rc} 0 ... Fixture git add failed: ${git_add.stderr} ${git_commit}= Run Process git commit -m Initial remote-clone container project cwd=${repo} Should Be Equal As Integers ${git_commit.rc} 0 ... Fixture git commit failed: ${git_commit.stderr} RETURN ${repo} *** Test Cases *** WF18 Container With Remote Repo Clone Trusted Profile [Documentation] Trusted-profile workflow: container-instance resource with ... --clone-into for remote repo clone, two-step project setup, ... plan-level execution environment with fallback priority, and ... full plan lifecycle including apply with container commit/push. [Timeout] 20 minutes [Teardown] Log WF18 Container Clone test completed. Skip If No LLM Keys # ---- Create fixture repo ---- ${repo}= Create Remote Clone Repo # ---- Register container-instance resource with --clone-into ---- ${resource_name}= Set Variable ${RESOURCE_PREFIX}-${RUN_SUFFIX} ${r_resource}= Run CleverAgents Command ... resource add container-instance ${resource_name} ... --clone-into ${repo}:/workspace ... --format plain Should Not Contain ${r_resource.stdout}${r_resource.stderr} Traceback Should Not Contain ${r_resource.stdout}${r_resource.stderr} INTERNAL Output Should Contain ${r_resource} ${resource_name} Output Should Contain ${r_resource} container-instance # ---- Create project (two-step: create then link-resource per spec) ---- ${project_name}= Set Variable ${PROJECT_PREFIX}-${RUN_SUFFIX} ${r_project_create}= Run CleverAgents Command ... project create ${project_name} ... --format plain Should Not Contain ${r_project_create.stdout}${r_project_create.stderr} Traceback Should Not Contain ${r_project_create.stdout}${r_project_create.stderr} INTERNAL Output Should Contain ${r_project_create} ${project_name} ${r_link}= Run CleverAgents Command ... project link-resource ${project_name} ${resource_name} ... --format plain Should Not Contain ${r_link.stdout}${r_link.stderr} Traceback Should Not Contain ${r_link.stdout}${r_link.stderr} INTERNAL Output Should Contain ${r_link} ${resource_name} # ---- Create action with dynamic actor selection ---- ${action_yaml}= Catenate SEPARATOR=\n ... name: ${ACTION_NAME} ... description: Implement container-based remote deployment with repo cloning ... definition_of_done: Deployment pipeline configured and validated ... strategy_actor: ${LLM_ACTOR} ... execution_actor: ${LLM_ACTOR} ${action_path}= Set Variable ${SUITE_HOME}${/}wf18_action.yaml Create File ${action_path} ${action_yaml} ${r_action}= Run CleverAgents Command ... action create --config ${action_path} ... --format plain Should Not Contain ${r_action.stdout}${r_action.stderr} Traceback Should Not Contain ${r_action.stdout}${r_action.stderr} INTERNAL Output Should Contain ${r_action} ${ACTION_NAME} # ---- Plan use with trusted profile, execution environment, and fallback priority ---- # NOTE: The spec Example 18 shows --execution-environment cloud/build-env # (a resource name). The current CLI implementation validates against the # ExecutionEnvironment enum which only accepts "host" or "container". # Resource-name-based resolution is future work; we use "container" here to # exercise the plan-level fallback routing path (precedence level 4). ${r_use}= Run CleverAgents Command ... plan use ${ACTION_NAME} ${project_name} ... --automation-profile trusted ... --execution-environment container ... --execution-env-priority fallback ... --format plain ... timeout=120s Should Not Contain ${r_use.stdout}${r_use.stderr} Traceback Should Not Contain ${r_use.stdout}${r_use.stderr} INTERNAL Should Not Be Empty ${r_use.stdout} Plan use produced no output ${plan_ids}= Get Regexp Matches ${r_use.stdout} [0-9A-HJKMNP-TV-Z]{26} Should Not Be Empty ${plan_ids} msg=Expected ULID plan ID in plan use output ${plan_id}= Set Variable ${plan_ids}[0] Log Plan ID: ${plan_id} # ---- Strategize ---- # The plan lifecycle uses two sequential ``plan execute`` calls: # the first advances the plan through the Strategize phase, and # the second advances it through the Execute phase. ${r_strat}= Run CleverAgents Command ... plan execute ${plan_id} ... --format plain ... timeout=180s Should Not Contain ${r_strat.stdout}${r_strat.stderr} Traceback Should Not Contain ${r_strat.stdout}${r_strat.stderr} INTERNAL Should Not Be Empty ${r_strat.stdout} Strategize produced no output Output Should Contain ${r_strat} ${plan_id} # ---- Execute ---- ${r_exec}= Run CleverAgents Command ... plan execute ${plan_id} ... --format plain ... timeout=300s Should Not Contain ${r_exec.stdout}${r_exec.stderr} Traceback Should Not Contain ${r_exec.stdout}${r_exec.stderr} INTERNAL Should Not Be Empty ${r_exec.stdout} Execute produced no output Output Should Contain ${r_exec} ${plan_id} # Verify container start and remote repo clone evidence (AC-4). # Soft assertion: LLM output is non-deterministic so we warn rather than fail. ${exec_combined}= Set Variable ${r_exec.stdout}\n${r_exec.stderr} ${exec_lower}= Evaluate ($exec_combined).lower() ${has_container_evidence}= Evaluate 'container' in $exec_lower or 'clone' in $exec_lower or 'workspace' in $exec_lower or 'starting' in $exec_lower Log Container/clone evidence in execute output: ${has_container_evidence} Run Keyword And Warn On Failure Should Be True ${has_container_evidence} ... Execute output should contain evidence of container start or clone operation # Verify execution environment resolution — plan fallback, precedence level 4 (AC-5). ${has_env_resolution}= Evaluate 'fallback' in $exec_lower or 'execution' in $exec_lower or 'environment' in $exec_lower or 'precedence' in $exec_lower or 'resolved' in $exec_lower Log Execution environment resolution evidence: ${has_env_resolution} Run Keyword And Warn On Failure Should Be True ${has_env_resolution} ... Execute output should contain execution environment resolution evidence (fallback/precedence) # ---- Diff ---- ${r_diff}= Run CleverAgents Command ... plan diff ${plan_id} ... --format plain ... timeout=60s Should Not Contain ${r_diff.stdout}${r_diff.stderr} Traceback Should Not Contain ${r_diff.stdout}${r_diff.stderr} INTERNAL Should Not Be Empty ${r_diff.stdout} Plan diff produced no output # ---- Capture pre-apply commit count ---- ${pre_log}= Run Process git log --oneline cwd=${repo} Should Be Equal As Integers ${pre_log.rc} 0 ... Pre-apply git log failed: ${pre_log.stderr} ${pre_commit_count}= Get Line Count ${pre_log.stdout} Log Pre-apply commit count: ${pre_commit_count} # ---- Apply ---- ${r_apply}= Run CleverAgents Command ... plan apply --yes ${plan_id} ... --format plain ... timeout=120s Should Not Contain ${r_apply.stdout}${r_apply.stderr} Traceback Should Not Contain ${r_apply.stdout}${r_apply.stderr} INTERNAL Should Not Be Empty ${r_apply.stdout} Apply produced no output Output Should Contain ${r_apply} ${plan_id} # Verify apply output references container commit and push (AC-6). # Soft assertion: LLM output is non-deterministic so we warn rather than fail. ${apply_combined}= Set Variable ${r_apply.stdout}\n${r_apply.stderr} ${apply_lower}= Evaluate ($apply_combined).lower() ${has_apply_evidence}= Evaluate 'commit' in $apply_lower or 'push' in $apply_lower or 'applied' in $apply_lower or 'container' in $apply_lower or 'origin' in $apply_lower Log Apply container commit/push evidence: ${has_apply_evidence} Run Keyword And Warn On Failure Should Be True ${has_apply_evidence} ... Apply output should contain evidence of commit/push from container # ---- Status — verify terminal state ---- ${r_status}= Run CleverAgents Command ... plan status ${plan_id} ... --format plain ... timeout=60s Should Not Contain ${r_status.stdout}${r_status.stderr} Traceback Should Not Contain ${r_status.stdout}${r_status.stderr} INTERNAL Should Not Be Empty ${r_status.stdout} Plan status produced no output Output Should Contain ${r_status} ${plan_id} # Verify plan reached a terminal state (applied phase) ${status_lower}= Evaluate ($r_status.stdout).lower() ${is_terminal}= Evaluate 'applied' in $status_lower or 'complete' in $status_lower or 'done' in $status_lower or 'finished' in $status_lower or 'apply' in $status_lower Should Be True ${is_terminal} ... Plan status should indicate a terminal/applied state after apply # ---- Verify repo state after apply ---- ${post_log}= Run Process git log --oneline cwd=${repo} Should Be Equal As Integers ${post_log.rc} 0 ... Post-apply git log failed: ${post_log.stderr} Log Git log after apply: ${post_log.stdout} ${post_commit_count}= Get Line Count ${post_log.stdout} # Apply may or may not create new commits depending on whether the LLM # generated file changes. Log the delta for diagnostics but only # hard-assert that the repo is still valid (at least the fixture commits). ${delta}= Evaluate ${post_commit_count} - ${pre_commit_count} Log Commit count delta after apply: ${delta} (pre=${pre_commit_count}, post=${post_commit_count}) Should Be True ${post_commit_count} >= ${pre_commit_count} ... Repo commit count should not decrease after apply (pre=${pre_commit_count}, post=${post_commit_count}) # Verify fixture files are still present in the repo after the full lifecycle. File Should Exist ${repo}${/}src${/}deploy_manager.py File Should Exist ${repo}${/}Dockerfile