cleveragents-core/robot/e2e/wf16_devcontainer.robot

*** Settings ***
Documentation    E2E test for Workflow Example 16: Devcontainer-Driven Development
...    (supervised automation profile).
...
...    Exercises the devcontainer-specific plan lifecycle:
...    devcontainer auto-detection during resource registration,
...    lazy container build during plan execution, tool invocation
...    routing to the container workspace, and apply writing changes
...    back to the host filesystem via bind mount.
...
...    **Devcontainer-specific assertions** are strict and contribute
...    to AC validation. If one or more AC indicators are missing,
...    the test records each unmet AC and fails explicitly rather
...    than skipping or passing silently.
...
...    The test is tagged ``tdd_expected_fail`` because devcontainer
...    features are not yet fully wired. The
...    ``tdd_expected_fail_listener`` inverts the failure to a pass
...    in CI until all AC indicators are present.
...
...    Zero mocking — real CLI, real LLM API keys.
Resource    common_e2e.resource
Suite Setup    WF16 Suite Setup
Suite Teardown    E2E Suite Teardown
Force Tags       E2E

*** Variables ***
${ACTION_PREFIX}    local/wf16-devcontainer-action
${RESOURCE_PREFIX}    local/wf16-devcontainer-repo
${PROJECT_PREFIX}    local/wf16-devcontainer-project

*** Keywords ***
WF16 Suite Setup
  [Documentation]    E2E Suite Setup plus database initialisation, unique suffix
  ...    generation, and dynamic actor selection for WF16 tests.
  [Tags]    tdd_issue tdd_issue_4188 tdd_expected_fail
  E2E Suite Setup
  # Initialise the database so CLI commands work in all tests.
  # Use --force because the workspace may already contain an initialised project.
  ${init}=    Run CleverAgents Command    init --force --yes
  Should Be Equal As Integers    ${init.rc}    0
  # Generate a unique suffix for resource/project names to avoid UNIQUE
  # constraint collisions when tests run in parallel or are re-run
  # against the same database (uuid4 provides ~4 billion possibilities).
  ${suffix}=    Evaluate    __import__('uuid').uuid4().hex[:12]
  Set Suite Variable    ${RUN_SUFFIX}    ${suffix}
  Set Suite Variable    ${ACTION_NAME}    ${ACTION_PREFIX}-${suffix}
  Set Suite Variable    ${RESOURCE_NAME}    ${RESOURCE_PREFIX}-${suffix}
  Set Suite Variable    ${PROJECT_NAME}    ${PROJECT_PREFIX}-${suffix}
  # Probe the OpenAI API to confirm the key is usable before selecting actor.
  # Falls back to Anthropic when the key is absent or quota-exhausted.
  # Uses gpt-4o-mini for cost optimization — WF16 exercises plan lifecycle
  # mechanics (not LLM quality), so a smaller model suffices.
  ${actor}=    Resolve LLM Actor    openai_model=openai/gpt-4o-mini
  Set Suite Variable    ${SELECTED_ACTOR}    ${actor}

Create Devcontainer Repo
  [Documentation]    Create a temp git repo with devcontainer configuration.
  ...    Returns the path to the created repository.
  ${repo}=    Create Temp Git Repo wf16-devcontainer-${RUN_SUFFIX}
  Create Directory    ${repo}${/}.devcontainer
  Create Directory    ${repo}${/}src
  ${devcontainer_json}=    Catenate    SEPARATOR=\n
  ...    {
  ...    ${SPACE}${SPACE}"name": "wf16-dev",
  ...    ${SPACE}${SPACE}"image": "mcr.microsoft.com/devcontainers/python:3.12@sha256:3de8f04c3748897ff3aa32b11ee18306d6c56556f4d548a276d0ff397b28b9da",
  ...    ${SPACE}${SPACE}"customizations": {
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}"vscode": {
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}${SPACE}"extensions": ["ms-python.python"]
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}}
  ...    ${SPACE}${SPACE}}
  ...    }
  Create File    ${repo}${/}.devcontainer${/}devcontainer.json    ${devcontainer_json}
  ${app_content}=    Catenate    SEPARATOR=\n
  ...    """Main application module."""
  ...    ${EMPTY}
  ...    ${EMPTY}
  ...    def main():
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}"""Entry point."""
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}print("Hello from devcontainer app")
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}return 0
  Create File    ${repo}${/}src${/}app.py    ${app_content}
  ${test_content}=    Catenate    SEPARATOR=\n
  ...    """Tests for the application."""
  ...    from src.app import main
  ...    ${EMPTY}
  ...    ${EMPTY}
  ...    def test_main():
  ...    ${SPACE}${SPACE}${SPACE}${SPACE}assert main() == 0
  Create File    ${repo}${/}src${/}test_app.py    ${test_content}
  Create File    ${repo}${/}src${/}__init__.py    \n
  Create File    ${repo}${/}requirements.txt    pytest>=7.0\n
  ${r_add}=    Run Process    git    add    .    cwd=${repo}    timeout=60s    on_timeout=kill
  Should Be Equal As Integers    ${r_add.rc}    0    msg=git add failed (rc=${r_add.rc})
  ${r_commit}=    Run Process    git    commit    -m    Initial devcontainer project    cwd=${repo}    timeout=60s    on_timeout=kill
  Should Be Equal As Integers    ${r_commit.rc}    0    msg=git commit failed (rc=${r_commit.rc})
  RETURN    ${repo}

WF16 Test Teardown
  [Documentation]    Log diagnostic context on failure for debugging.
  ...    Captures plan status when a plan ID is available.
  ${plan_id}=    Get Variable Value    ${WF16_PLAN_ID}    ${EMPTY}
  IF    '${plan_id}' != ''
  ${status}    ${result}=    Run Keyword And Ignore Error
  ...    Run CleverAgents Command    plan status ${plan_id} --format json    expected_rc=None    timeout=30s
  IF    '${status}' == 'PASS'
  Log    Teardown plan status: ${result.stdout}    WARN
  END
  END

*** Test Cases ***
WF16 Devcontainer Driven Development Supervised Profile
  [Documentation]    Supervised-profile workflow exercising devcontainer-specific
  ...    behaviors: auto-detection during resource registration,
  ...    lazy container build during execution, tool invocation
  ...    routing to the container workspace, and apply writing
  ...    changes back to the host filesystem via bind mount.
  ...
  ...    Devcontainer-specific assertions are enforced via
  ...    explicit AC checks. Missing AC indicators are collected
  ...    and reported as explicit test failures.
  ...
  ...    Tagged ``tdd_expected_fail`` because devcontainer features
  ...    are not yet fully wired; the listener inverts the failure
  ...    to a CI pass until all AC indicators are present.
  [Tags]    tdd_issue tdd_issue_4188 tdd_expected_fail tdd_issue_1208
  # Timeout budget: 35 minutes test-level timeout. Per-step timeouts sum
  # higher in theory, but retries and conditional paths are mutually
  # exclusive — realistic worst-case is well under 35 minutes.
  [Timeout]    35 minutes
  [Teardown]    WF16 Test Teardown
  Skip If No LLM Keys
  ${ac_checks_missing}=    Evaluate    []
  Set Test Variable    ${WF16_PLAN_ID}    ${EMPTY}