freemo
d086f6a85e
CI / push-validation (pull_request) Failing after 18s
CI / lint (pull_request) Successful in 32s
CI / helm (pull_request) Successful in 42s
CI / build (pull_request) Successful in 45s
CI / typecheck (pull_request) Successful in 53s
CI / security (pull_request) Successful in 1m9s
CI / e2e_tests (pull_request) Failing after 3m36s
CI / quality (pull_request) Successful in 3m44s
CI / coverage (pull_request) Failing after 3m7s
CI / unit_tests (pull_request) Has been cancelled
CI / integration_tests (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / benchmark-regression (pull_request) Has been cancelled
CI / benchmark-publish (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
The CleverAgents CLI now returns JSON responses with a nested 'data' field wrapper. Updated all e2e tests to handle this new structure: - M5: Fixed fragment_count, tier_budget, budget_used, plan_id field access - M6: Updated automation profile and phase/state field access - WF04: Fixed plan_id extraction from plan use response - WF05: Updated all Safe Parse Json Field calls to use nested support - WF07: Updated remaining Safe Parse Json Field call - WF12: Fixed phase/processing_state field access - WF17: Updated all JSON field parsing for phase and execution environment All tests now use 'Safe Parse Json Field With Nested Support' which handles the data.fieldname path correctly for the new JSON structure.
450 lines
24 KiB
Plaintext
450 lines
24 KiB
Plaintext
*** Settings ***
|
|
Documentation E2E test for Workflow Example 17: Explicit Container with
|
|
... Directory Mount (trusted profile).
|
|
...
|
|
... Exercises resource registration (git-checkout and
|
|
... container-instance), project creation with linked container
|
|
... resource, execution environment override via project context
|
|
... set, action creation with trusted automation profile, and
|
|
... full plan lifecycle.
|
|
...
|
|
... Zero mocking — real CLI, real LLM API keys.
|
|
Resource common_e2e.resource
|
|
Suite Setup WF17 Suite Setup
|
|
Suite Teardown E2E Suite Teardown
|
|
Force Tags E2E
|
|
|
|
*** Variables ***
|
|
${ACTION_BASE} local/wf17-container-mount
|
|
${RESOURCE_BASE} local/wf17-mount-repo
|
|
${CONTAINER_BASE} local/wf17-container
|
|
${PROJECT_BASE} local/wf17-mount-project
|
|
|
|
*** Keywords ***
|
|
WF17 Suite Setup
|
|
[Documentation] E2E Suite Setup plus workspace initialisation and
|
|
... dynamic actor selection for WF17 container tests.
|
|
E2E Suite Setup
|
|
# Initialise the workspace so all CLI commands work.
|
|
${init}= Run CleverAgents Command init --force --yes
|
|
Should Be Equal As Integers ${init.rc} 0
|
|
# Generate a unique suffix for resource/project names to avoid UNIQUE
|
|
# constraint collisions on repeated or parallel E2E runs.
|
|
${suffix}= Evaluate __import__('uuid').uuid4().hex[:12]
|
|
Set Suite Variable ${RUN_SUFFIX} ${suffix}
|
|
# Compute run-unique names from base names + suffix.
|
|
Set Suite Variable ${ACTION_NAME} ${ACTION_BASE}-${suffix}
|
|
Set Suite Variable ${RESOURCE_NAME} ${RESOURCE_BASE}-${suffix}
|
|
Set Suite Variable ${CONTAINER_NAME} ${CONTAINER_BASE}-${suffix}
|
|
Set Suite Variable ${PROJECT_NAME} ${PROJECT_BASE}-${suffix}
|
|
# Dynamically select actor based on available API key.
|
|
${has_anthropic}= Evaluate bool(__import__('os').environ.get('ANTHROPIC_API_KEY', ''))
|
|
IF ${has_anthropic}
|
|
${actor}= Set Variable anthropic/claude-sonnet-4-20250514
|
|
ELSE
|
|
${actor}= Set Variable openai/gpt-4o
|
|
END
|
|
Set Suite Variable ${ACTOR} ${actor}
|
|
|
|
Create Mountable Repo
|
|
[Documentation] Create a temp git repo simulating a project that would
|
|
... be mounted into a container. Kept minimal (README +
|
|
... initial commit) — the fixture's purpose is to provide
|
|
... a valid git-checkout resource, not specific file content.
|
|
... ``Create Temp Git Repo`` already creates README.md and
|
|
... commits it, so no additional git operations are needed.
|
|
${repo}= Create Temp Git Repo wf17-mount-project-${RUN_SUFFIX}
|
|
RETURN ${repo}
|
|
|
|
*** Test Cases ***
|
|
WF17 Explicit Container With Directory Mount Trusted Profile
|
|
[Documentation] Trusted-profile workflow: register git-checkout and
|
|
... container-instance resources, create project, link
|
|
... container resource, set execution environment with
|
|
... plan-level override (precedence level 1), and run full
|
|
... plan lifecycle with trusted automation.
|
|
...
|
|
... The ``trusted`` automation profile grants auto-execution
|
|
... of tool invocations without user confirmation. This is
|
|
... intentional for E2E testing of the WF17 scenario where
|
|
... tool invocations should route to the container.
|
|
[Timeout] 20 minutes
|
|
|
|
Skip If No LLM Keys
|
|
|
|
# ---- Create fixture repo ----
|
|
${repo}= Create Mountable Repo
|
|
${branch_result}= Run Process git rev-parse --abbrev-ref HEAD
|
|
... cwd=${repo} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${branch_result.rc} 0 git rev-parse failed: ${branch_result.stderr}
|
|
${branch}= Strip String ${branch_result.stdout}
|
|
|
|
# ---- Register git-checkout resource ----
|
|
${r_res}= Run CleverAgents Command
|
|
... resource add git-checkout ${RESOURCE_NAME}
|
|
... --path ${repo} --branch ${branch} --format json
|
|
Should Not Contain ${r_res.stdout}${r_res.stderr} Traceback
|
|
Should Not Contain ${r_res.stdout}${r_res.stderr} INTERNAL
|
|
Output Should Contain ${r_res} ${RESOURCE_NAME}
|
|
|
|
# ---- Register container-instance resource (WF17 Step 1) ----
|
|
# NOTE: --mount flags are not yet available on the CLI. The spec's WF17
|
|
# shows dual mounts (resource-ref rw + host-path ro), but the current
|
|
# 'resource add container-instance' command only supports --image.
|
|
# Dual-mount testing is deferred until CLI support is implemented.
|
|
# Tracked in #1078.
|
|
Log WARN: --mount flags not yet available on CLI — dual-mount test deferred (AC #2, tracked in #1078) WARN
|
|
${r_container}= Run CleverAgents Command
|
|
... resource add container-instance ${CONTAINER_NAME}
|
|
... --image python:3.12-slim --format json
|
|
Should Not Contain ${r_container.stdout}${r_container.stderr} Traceback
|
|
Should Not Contain ${r_container.stdout}${r_container.stderr} INTERNAL
|
|
Output Should Contain ${r_container} ${CONTAINER_NAME}
|
|
|
|
# ---- Create project and link resources (WF17 Step 2) ----
|
|
# Following the spec's prescribed sequence: project create without --resource,
|
|
# then separate link-resource calls for each resource.
|
|
${r_proj}= Run CleverAgents Command
|
|
... project create ${PROJECT_NAME} --format json
|
|
Should Not Contain ${r_proj.stdout}${r_proj.stderr} Traceback
|
|
Should Not Contain ${r_proj.stdout}${r_proj.stderr} INTERNAL
|
|
Output Should Contain ${r_proj} ${PROJECT_NAME}
|
|
|
|
# Link git-checkout resource to the project
|
|
${r_link_repo}= Run CleverAgents Command
|
|
... project link-resource ${PROJECT_NAME} ${RESOURCE_NAME}
|
|
... --format json
|
|
Should Not Contain ${r_link_repo.stdout}${r_link_repo.stderr} Traceback
|
|
Should Not Contain ${r_link_repo.stdout}${r_link_repo.stderr} INTERNAL
|
|
Output Should Contain ${r_link_repo} ${RESOURCE_NAME}
|
|
|
|
# Link container resource to the project
|
|
${r_link}= Run CleverAgents Command
|
|
... project link-resource ${PROJECT_NAME} ${CONTAINER_NAME}
|
|
... --format json
|
|
Should Not Contain ${r_link.stdout}${r_link.stderr} Traceback
|
|
Should Not Contain ${r_link.stdout}${r_link.stderr} INTERNAL
|
|
Output Should Contain ${r_link} ${CONTAINER_NAME}
|
|
|
|
# Set execution environment to container via project context set (WF17 Step 2)
|
|
# NOTE: --execution-environment accepts enum values (host/container), not resource
|
|
# names. The spec shows resource names, but the CLI's ExecutionEnvironment enum only
|
|
# validates enum values. This is a known spec divergence that is not yet tracked in
|
|
# a dedicated issue (#1079 covers --execution-env-priority, not this enum issue).
|
|
${r_ctx}= Run CleverAgents Command
|
|
... project context set
|
|
... --execution-environment container
|
|
... ${PROJECT_NAME} --format json
|
|
Should Not Contain ${r_ctx.stdout}${r_ctx.stderr} Traceback
|
|
Should Not Contain ${r_ctx.stdout}${r_ctx.stderr} INTERNAL
|
|
|
|
# Verify execution environment setting persisted via project context show.
|
|
# NOTE: 'project context show' returns views/ACMS config but may not expose
|
|
# the execution_environment field directly. We attempt field extraction and
|
|
# fall back to logging the output if the field is absent.
|
|
${r_ctx_show}= Run CleverAgents Command
|
|
... project context show ${PROJECT_NAME} --format json
|
|
... expected_rc=None
|
|
IF ${r_ctx_show.rc} == 0
|
|
${persisted_env}= Safe Parse Json Field With Nested Support ${r_ctx_show.stdout} execution_environment
|
|
IF "${persisted_env}" != ""
|
|
Should Be Equal As Strings ${persisted_env.lower()} container
|
|
... Execution environment should be persisted as 'container' but got '${persisted_env}'
|
|
Log Execution environment persisted: ${persisted_env}
|
|
ELSE
|
|
Log execution_environment field not present in project context show output — persistence verification deferred (field not exposed by current CLI) WARN
|
|
END
|
|
Log project context show output: ${r_ctx_show.stdout}
|
|
ELSE
|
|
Log project context show not available or failed (rc=${r_ctx_show.rc}) — verification skipped WARN
|
|
END
|
|
|
|
# ---- Create action with trusted profile (dynamic actor) ----
|
|
${action_yaml}= Catenate SEPARATOR=\n
|
|
... name: ${ACTION_NAME}
|
|
... description: "Implement container-based deployment with directory mounts"
|
|
... definition_of_done: "Container config and mount paths validated"
|
|
... strategy_actor: ${ACTOR}
|
|
... execution_actor: ${ACTOR}
|
|
${action_path}= Set Variable ${SUITE_HOME}${/}wf17_action_${RUN_SUFFIX}.yaml
|
|
Create File ${action_path} ${action_yaml}
|
|
${r_action}= Run CleverAgents Command
|
|
... action create --config ${action_path} --format json
|
|
Should Not Contain ${r_action.stdout}${r_action.stderr} Traceback
|
|
Should Not Contain ${r_action.stdout}${r_action.stderr} INTERNAL
|
|
Output Should Contain ${r_action} ${ACTION_NAME}
|
|
|
|
# ---- Plan use with execution-env-priority override (WF17 Step 3) ----
|
|
# NOTE: Uses plan-level override (precedence level 1) as a workaround.
|
|
# The spec describes project-level override (precedence level 2) via
|
|
# 'project context set --execution-env-priority override', but the CLI
|
|
# only implements this flag on 'plan use'. Project-level override testing
|
|
# is deferred until CLI support is added. Tracked in #1079 and #1080.
|
|
# Security note: trusted profile grants auto-execution of tool invocations
|
|
# without user confirmation. This is intentional for E2E validation.
|
|
${r_use}= Run CleverAgents Command
|
|
... plan use ${ACTION_NAME} ${PROJECT_NAME}
|
|
... --automation-profile trusted
|
|
... --execution-environment container
|
|
... --execution-env-priority override
|
|
... --format json
|
|
Should Not Contain ${r_use.stdout}${r_use.stderr} Traceback
|
|
Should Not Contain ${r_use.stdout}${r_use.stderr} INTERNAL
|
|
${plan_id}= Safe Parse Json Field With Nested Support With Nested Support ${r_use.stdout} data.plan_id
|
|
Should Not Be Empty ${plan_id} msg=Expected plan_id in plan use JSON output
|
|
Should Match Regexp ${plan_id} ^[0-9A-HJKMNP-TV-Z]{26}$
|
|
... Plan ID should be a valid Crockford Base32 ULID: ${plan_id}
|
|
Log Plan ID: ${plan_id}
|
|
|
|
# ---- Strategize ----
|
|
${r_strat}= Run CleverAgents Command
|
|
... plan execute ${plan_id}
|
|
... --format json
|
|
... expected_rc=None
|
|
... timeout=180s
|
|
IF ${r_strat.rc} != 0
|
|
Fail plan execute (strategize) failed (rc=${r_strat.rc}) stdout=${r_strat.stdout} stderr=${r_strat.stderr}
|
|
END
|
|
Should Not Contain ${r_strat.stdout}${r_strat.stderr} Traceback
|
|
Should Not Contain ${r_strat.stdout}${r_strat.stderr} INTERNAL
|
|
Output Should Contain ${r_strat} ${plan_id}
|
|
|
|
# ---- Execute ----
|
|
# NOTE: 300s timeout (vs M6's 180s) accounts for potential container image
|
|
# pull overhead on first execution in a clean CI environment.
|
|
${r_exec}= Run CleverAgents Command
|
|
... plan execute ${plan_id}
|
|
... --format json
|
|
... expected_rc=None
|
|
... timeout=300s
|
|
IF ${r_exec.rc} != 0
|
|
Fail plan execute (execute) failed (rc=${r_exec.rc}) stdout=${r_exec.stdout} stderr=${r_exec.stderr}
|
|
END
|
|
Should Not Contain ${r_exec.stdout}${r_exec.stderr} Traceback
|
|
Should Not Contain ${r_exec.stdout}${r_exec.stderr} INTERNAL
|
|
Output Should Contain ${r_exec} ${plan_id}
|
|
|
|
# ---- Diff ----
|
|
${r_diff}= Run CleverAgents Command
|
|
... plan diff ${plan_id}
|
|
... --format json
|
|
... timeout=120s
|
|
Should Not Contain ${r_diff.stdout}${r_diff.stderr} Traceback
|
|
Should Not Contain ${r_diff.stdout}${r_diff.stderr} INTERNAL
|
|
Should Not Be Empty ${r_diff.stdout} Plan diff produced no output
|
|
|
|
# Capture HEAD SHA before apply so we can verify the repository changed.
|
|
${pre_apply_head}= Run Process git rev-parse HEAD
|
|
... cwd=${repo} timeout=60s on_timeout=kill
|
|
${pre_apply_sha}= Strip String ${pre_apply_head.stdout}
|
|
|
|
# ---- Apply ----
|
|
${r_apply}= Run CleverAgents Command
|
|
... plan apply --yes ${plan_id}
|
|
... --format json
|
|
... timeout=120s
|
|
Should Not Contain ${r_apply.stdout}${r_apply.stderr} Traceback
|
|
Should Not Contain ${r_apply.stdout}${r_apply.stderr} INTERNAL
|
|
Output Should Contain ${r_apply} ${plan_id}
|
|
|
|
# ---- Post-apply repository observation (informational) ----
|
|
# Check whether apply produced new commits in the fixture
|
|
# repository. Because the plan's file modifications are LLM-driven,
|
|
# new commits are likely but not guaranteed on every run. We assert
|
|
# that HEAD changed when we can read it; if HEAD did not move we log
|
|
# a warning rather than fail, since the absence of new commits does
|
|
# not indicate a bug in the container workflow itself.
|
|
${post_apply_head}= Run Process git rev-parse HEAD
|
|
... cwd=${repo} timeout=60s on_timeout=kill
|
|
IF ${post_apply_head.rc} == 0
|
|
${post_apply_sha}= Strip String ${post_apply_head.stdout}
|
|
IF "${post_apply_sha}" == "${pre_apply_sha}"
|
|
Log Post-apply HEAD unchanged (${pre_apply_sha}) — apply may not have produced commits this run WARN
|
|
ELSE
|
|
Log Post-apply HEAD moved: ${pre_apply_sha} → ${post_apply_sha}
|
|
END
|
|
ELSE
|
|
Log Post-apply git rev-parse failed (rc=${post_apply_head.rc}) — observation skipped WARN
|
|
END
|
|
|
|
# ---- Status ----
|
|
${r_status}= Run CleverAgents Command
|
|
... plan status ${plan_id}
|
|
... --format json
|
|
... timeout=120s
|
|
Should Not Contain ${r_status.stdout}${r_status.stderr} Traceback
|
|
Should Not Contain ${r_status.stdout}${r_status.stderr} INTERNAL
|
|
Should Not Be Empty ${r_status.stdout}
|
|
Output Should Contain ${r_status} ${plan_id}
|
|
|
|
# ---- Terminal state assertion ----
|
|
# After full lifecycle (strategize → execute → diff → apply → status),
|
|
# the plan should have reached an apply-related phase, matching M6's
|
|
# Full Flow Apply Step pattern.
|
|
${final_phase}= Safe Parse Json Field With Nested Support ${r_status.stdout} phase
|
|
${final_state}= Safe Parse Json Field With Nested Support ${r_status.stdout} processing_state
|
|
Should Not Be Empty ${final_phase} Plan phase should be populated after full lifecycle
|
|
Should Contain ${final_phase.lower()} apply
|
|
... Plan should reach apply phase after apply (got: ${final_phase})
|
|
Log Final phase=${final_phase} processing_state=${final_state}
|
|
|
|
# ---- Container routing verification (best-effort) ----
|
|
# Attempt to verify that the plan resolved with execution_environment=container
|
|
# by inspecting the plan status JSON. If the field is present, assert its value;
|
|
# if absent, log the gap — direct container routing verification is not yet fully
|
|
# feasible at the E2E level because the CLI does not emit routing-specific
|
|
# indicators beyond echoed arguments.
|
|
# NOTE: Container routing verification gap is not yet tracked in a dedicated
|
|
# issue. A tracking issue should be created to cover CLI-level routing
|
|
# indicators for container execution.
|
|
${exec_env}= Safe Parse Json Field With Nested Support ${r_status.stdout} execution_environment
|
|
IF "${exec_env}" != ""
|
|
Should Be Equal As Strings ${exec_env.lower()} container
|
|
... Execution environment should be 'container' but got '${exec_env}'
|
|
Log Container routing verified: execution_environment=${exec_env}
|
|
ELSE
|
|
Log execution_environment field not present in plan status JSON — container routing verification gap (plan completed full lifecycle with execution-environment=container and execution-env-priority=override) WARN
|
|
END
|
|
|
|
WF17 TDD Dual Mount Registration
|
|
[Documentation] Regression guard for AC #2: Register container-instance
|
|
... resource with dual mounts (resource-ref rw + host-path ro).
|
|
... Bug #1078 is now fixed — --mount flag is implemented on
|
|
... ``resource add container-instance``.
|
|
... Tracked in #1078.
|
|
[Tags] tdd_issue tdd_issue_1078
|
|
[Timeout] 5 minutes
|
|
|
|
# No LLM keys needed — this exercises pure CLI flag parsing.
|
|
|
|
# Regression guard: --mount flag is now implemented on the CLI.
|
|
${r_container}= Run CleverAgents Command
|
|
... resource add container-instance ${CONTAINER_BASE}-mount-${RUN_SUFFIX}
|
|
... --image python:3.12-slim
|
|
... --mount ${RESOURCE_BASE}-${RUN_SUFFIX}:/workspace
|
|
... --mount /var/shared/config:/config:ro
|
|
... --format json
|
|
... expected_rc=None
|
|
Should Be Equal As Integers ${r_container.rc} 0
|
|
... resource add container-instance with --mount should succeed (bug #1078)
|
|
Output Should Contain ${r_container} ${CONTAINER_BASE}-mount-${RUN_SUFFIX}
|
|
# Verify mount information is present in the output
|
|
${combined}= Set Variable ${r_container.stdout}${r_container.stderr}
|
|
Should Contain ${combined} /workspace
|
|
... Mount path /workspace should appear in resource output
|
|
|
|
WF17 TDD Project Level Execution Env Priority Override
|
|
[Documentation] TDD test for AC #3: Set execution environment override priority
|
|
... at the project level via ``project context set
|
|
... --execution-env-priority override``.
|
|
... Regression test for #1079: --execution-env-priority flag
|
|
... is now implemented on ``project context set``.
|
|
[Tags] tdd_issue tdd_issue_1079
|
|
[Timeout] 5 minutes
|
|
|
|
# Create a lightweight project for this TDD test so it does not depend on
|
|
# the main test's side effects.
|
|
${tdd_proj}= Set Variable ${PROJECT_BASE}-tdd-ctx-${RUN_SUFFIX}
|
|
${r_proj}= Run CleverAgents Command
|
|
... project create ${tdd_proj} --format json
|
|
... expected_rc=None
|
|
IF ${r_proj.rc} != 0
|
|
Fail TDD setup: project create failed (rc=${r_proj.rc}): ${r_proj.stderr}
|
|
END
|
|
|
|
# Attempt to set execution-env-priority at the project level.
|
|
# This should succeed once #1079 is fixed.
|
|
${r_ctx}= Run CleverAgents Command
|
|
... project context set
|
|
... --execution-environment container
|
|
... --execution-env-priority override
|
|
... ${tdd_proj} --format json
|
|
... expected_rc=None
|
|
Should Be Equal As Integers ${r_ctx.rc} 0
|
|
... project context set --execution-env-priority should succeed (bug #1079)
|
|
# Verify the setting persisted via project context show
|
|
${r_show}= Run CleverAgents Command
|
|
... project context show ${tdd_proj} --format json
|
|
Should Be Equal As Integers ${r_show.rc} 0
|
|
${show_combined}= Set Variable ${r_show.stdout}${r_show.stderr}
|
|
Should Contain ${show_combined.lower()} override
|
|
... Project context should show execution-env-priority override
|
|
|
|
WF17 TDD Precedence Level 2 Project Override Resolution
|
|
[Documentation] TDD test for AC #5: Verify execution environment resolves via
|
|
... project-level override (precedence level 2) when no plan-level
|
|
... override is present.
|
|
... Regression test for #1079 and #1080: project-level
|
|
... execution-env-priority is now implemented and the
|
|
... resolution logic honours it.
|
|
[Tags] tdd_issue tdd_issue_1080
|
|
[Timeout] 10 minutes
|
|
|
|
# Create lightweight project and action for this TDD test so it does not
|
|
# depend on the main test's side effects.
|
|
${tdd_proj}= Set Variable ${PROJECT_BASE}-tdd-prec-${RUN_SUFFIX}
|
|
${r_proj}= Run CleverAgents Command
|
|
... project create ${tdd_proj} --format json
|
|
... expected_rc=None
|
|
IF ${r_proj.rc} != 0
|
|
Fail TDD setup: project create failed (rc=${r_proj.rc}): ${r_proj.stderr}
|
|
END
|
|
# Dynamically select actor based on available API key.
|
|
${has_anthropic}= Evaluate bool(__import__('os').environ.get('ANTHROPIC_API_KEY', ''))
|
|
IF ${has_anthropic}
|
|
${tdd_actor}= Set Variable anthropic/claude-sonnet-4-20250514
|
|
ELSE
|
|
${tdd_actor}= Set Variable openai/gpt-4o
|
|
END
|
|
${tdd_action_name}= Set Variable ${ACTION_BASE}-tdd-prec-${RUN_SUFFIX}
|
|
${tdd_action_yaml}= Catenate SEPARATOR=\n
|
|
... name: ${tdd_action_name}
|
|
... description: "TDD precedence level 2 test action"
|
|
... definition_of_done: "Verify project-level override resolution"
|
|
... strategy_actor: ${tdd_actor}
|
|
... execution_actor: ${tdd_actor}
|
|
${tdd_action_path}= Set Variable ${SUITE_HOME}${/}wf17_tdd_prec_action_${RUN_SUFFIX}.yaml
|
|
Create File ${tdd_action_path} ${tdd_action_yaml}
|
|
${r_action}= Run CleverAgents Command
|
|
... action create --config ${tdd_action_path} --format json
|
|
... expected_rc=None
|
|
IF ${r_action.rc} != 0
|
|
Fail TDD setup: action create failed (rc=${r_action.rc}): ${r_action.stderr}
|
|
END
|
|
|
|
# Set project-level execution environment with override priority.
|
|
# This depends on #1079 being fixed first.
|
|
${r_ctx}= Run CleverAgents Command
|
|
... project context set
|
|
... --execution-environment container
|
|
... --execution-env-priority override
|
|
... ${tdd_proj} --format json
|
|
... expected_rc=None
|
|
Should Be Equal As Integers ${r_ctx.rc} 0
|
|
... project context set --execution-env-priority should succeed (depends on #1079)
|
|
|
|
# Create a plan WITHOUT plan-level --execution-env-priority override.
|
|
# The execution environment should still resolve to container via
|
|
# project-level override (precedence level 2).
|
|
${r_use}= Run CleverAgents Command
|
|
... plan use ${tdd_action_name} ${tdd_proj}
|
|
... --automation-profile trusted
|
|
... --format json
|
|
... expected_rc=None
|
|
IF ${r_use.rc} != 0
|
|
Fail plan use failed (rc=${r_use.rc}) — cannot verify precedence level 2: ${r_use.stderr}
|
|
END
|
|
${plan_id}= Safe Parse Json Field With Nested Support With Nested Support ${r_use.stdout} data.plan_id
|
|
Should Not Be Empty ${plan_id} msg=Expected plan_id in plan use output
|
|
|
|
# Check plan status for execution environment resolution.
|
|
${r_status}= Run CleverAgents Command
|
|
... plan status ${plan_id} --format json
|
|
... expected_rc=None timeout=120s
|
|
Should Be Equal As Integers ${r_status.rc} 0
|
|
# The plan should show execution_environment=container resolved via
|
|
# project-level override (precedence level 2), not plan-level.
|
|
${exec_env}= Safe Parse Json Field With Nested Support ${r_status.stdout} execution_environment
|
|
Should Be Equal As Strings ${exec_env.lower()} container
|
|
... Execution environment should resolve to container via project-level override (precedence level 2, bug #1080)
|