*** Settings *** Documentation E2E acceptance test for M2 (v3.1.0): Actor Compiler + Full LLM Integration. ... ... Exercises actor YAML compilation into functional graphs, skill registry, ... tool lifecycle, and plan execution with a custom actor using real LLM keys. ... Zero mocking — all CLI invocations hit the real CleverAgents binary with ... real provider keys. Resource common_e2e.resource Suite Setup E2E Suite Setup Suite Teardown E2E Suite Teardown *** Variables *** ${ACTOR_NAME} local/m2-e2e-actor ${ACTION_NAME} local/m2-e2e-action ${RESOURCE_NAME} local/m2-e2e-repo ${PROJECT_NAME} local/m2-e2e-project *** Test Cases *** M2 Full Actor Compiler And LLM Integration [Documentation] End-to-end acceptance test for the M2 milestone. ... ... Exercises actor YAML compilation, registration via CLI, ... resource and project setup, action creation referencing a ... custom actor, and the full plan lifecycle (use, execute ... strategize, execute, diff, apply) with real LLM API keys. [Tags] E2E Skip If No LLM Keys # ---- Step 1: Create temp git repo with sample project files ---- ${repo_dir}= Create Temp Git Repo m2-e2e-repo Create Directory ${repo_dir}${/}src Create File ${repo_dir}${/}src${/}main.py print("hello world")\n ${r_add}= Run Process git add . cwd=${repo_dir} timeout=60s on_timeout=kill Should Be Equal As Integers ${r_add.rc} 0 msg=git add failed (rc=${r_add.rc}). Check DEBUG logs above. ${r_commit}= Run Process git commit -m Add source files cwd=${repo_dir} timeout=60s on_timeout=kill Should Be Equal As Integers ${r_commit.rc} 0 msg=git commit failed (rc=${r_commit.rc}). Check DEBUG logs above. # Detect the default branch name created by git init ${branch_result}= Run Process git rev-parse --abbrev-ref HEAD cwd=${repo_dir} timeout=60s on_timeout=kill ${branch}= Strip String ${branch_result.stdout} Log Detected branch: ${branch} # ---- Resolve LLM actor (OpenAI if available, Anthropic fallback) ---- ${llm_actor}= Resolve LLM Actor ${llm_parts}= Evaluate $llm_actor.split('/', 1) ${llm_provider}= Set Variable ${llm_parts}[0] ${llm_model}= Set Variable ${llm_parts}[1] # ---- Step 2: Create and register custom actor YAML ---- ${actor_yaml}= Catenate SEPARATOR=\n ... name: ${ACTOR_NAME} ... type: llm ... description: M2 E2E acceptance actor for compiler and LLM integration ... version: "1.0" ... model: ${llm_model} ${actor_yaml_path}= Set Variable ${SUITE_HOME}${/}m2_actor.yaml Create File ${actor_yaml_path} ${actor_yaml} ${actor_config}= Catenate SEPARATOR=\n ... { ... "name": "${ACTOR_NAME}", ... "provider": "${llm_provider}", ... "model": "${llm_model}", ... "options": {"temperature": 0.2} ... } ${config_path}= Set Variable ${SUITE_HOME}${/}actor_config.json Create File ${config_path} ${actor_config} ${r_actor}= Run CleverAgents Command ... actor add --config ${config_path} --format plain Should Be Equal As Integers ${r_actor.rc} 0 msg=Actor add failed (rc=${r_actor.rc}). Check DEBUG logs above. Should Not Contain ${r_actor.stdout}${r_actor.stderr} Traceback Log Actor registration: ${r_actor.stdout} # ---- Step 3: Register resource and create project ---- ${r_resource}= Run CleverAgents Command ... resource add git-checkout ${RESOURCE_NAME} ... --path ${repo_dir} --branch ${branch} --format plain Output Should Contain ${r_resource} ${RESOURCE_NAME} ${r_project}= Run CleverAgents Command ... project create ${PROJECT_NAME} ... --description M2 E2E acceptance project ... --resource ${RESOURCE_NAME} --format plain Output Should Contain ${r_project} ${PROJECT_NAME} # ---- Step 4: Create action referencing the custom actor ---- ${action_yaml}= Catenate SEPARATOR=\n ... name: ${ACTION_NAME} ... description: M2 acceptance test action for actor compiler and LLM integration ... definition_of_done: Generate or modify at least one source file ... strategy_actor: ${llm_actor} ... execution_actor: ${llm_actor} ${action_yaml_path}= Set Variable ${SUITE_HOME}${/}action.yaml Create File ${action_yaml_path} ${action_yaml} ${r_action}= Run CleverAgents Command ... action create --config ${action_yaml_path} --format plain Output Should Contain ${r_action} ${ACTION_NAME} # ---- Step 5: Plan use ---- ${r_use}= Run CleverAgents Command ... plan use ${ACTION_NAME} ${PROJECT_NAME} --format plain Should Not Be Empty ${r_use.stdout} ${plan_ids}= Get Regexp Matches ${r_use.stdout} [0-9A-Z]{26} Should Not Be Empty ${plan_ids} msg=Expected a ULID plan ID in plan use output ${plan_id}= Set Variable ${plan_ids}[0] Log Extracted plan_id: ${plan_id} # ---- Step 6: Plan execute — strategize phase ---- ${r_strategize}= Run CleverAgents Command ... plan execute ${plan_id} --format plain ... timeout=180s Should Not Contain ${r_strategize.stdout}${r_strategize.stderr} INTERNAL Should Not Contain ${r_strategize.stdout}${r_strategize.stderr} Traceback Log Strategize phase output: ${r_strategize.stdout} # ---- Step 7: Plan execute — execute phase ---- ${r_execute}= Run CleverAgents Command ... plan execute ${plan_id} --format plain ... timeout=180s Should Not Contain ${r_execute.stdout}${r_execute.stderr} INTERNAL Should Not Contain ${r_execute.stdout}${r_execute.stderr} Traceback Log Execute phase output: ${r_execute.stdout} # ---- Step 8: Plan diff ---- ${r_diff}= Run CleverAgents Command ... plan diff ${plan_id} --format plain Should Not Contain ${r_diff.stdout}${r_diff.stderr} INTERNAL Should Not Contain ${r_diff.stdout}${r_diff.stderr} Traceback Log Diff output: ${r_diff.stdout} # ---- Step 9: Plan apply ---- ${r_apply}= Run CleverAgents Command ... plan apply --yes ${plan_id} --format plain Should Not Contain ${r_apply.stdout}${r_apply.stderr} INTERNAL Should Not Contain ${r_apply.stdout}${r_apply.stderr} Traceback Log Apply output: ${r_apply.stdout} # ---- Step 10: Verify actor compilation and plan integrity ---- ${r_status}= Run CleverAgents Command ... plan status ${plan_id} --format plain Should Not Be Empty ${r_status.stdout} Output Should Contain ${r_status} ${plan_id} Log Final plan status: ${r_status.stdout}