cleveragents-core/robot/e2e/common_e2e.resource

*** Settings ***
Documentation     Common resources and keywords for E2E Robot Framework tests.
...
...               E2E tests use **zero mocking** — they exercise the real
...               CleverAgents CLI with real LLM API keys.  This resource
...               provides shared setup/teardown, API key detection with
...               graceful skip, and flexible assertion helpers.
Library           OperatingSystem
Library           String
Library           Process

*** Variables ***
${WORKSPACE}          ${CURDIR}/../..
${SRC_DIR}            ${WORKSPACE}/src/cleveragents
${E2E_TEMP_ROOT}      ${TEMPDIR}${/}cleveragents_e2e

*** Keywords ***
E2E Suite Setup
    [Documentation]    Set up the E2E test environment with per-suite isolation.
    ...
    ...                Creates a unique CLEVERAGENTS_HOME directory.
    ...                Does NOT enable mock AI — E2E tests use real providers.
    ...                Propagates LLM API keys from the environment.
    Log    Setting up E2E test environment
    ${safe_suite}=    Replace String    ${SUITE NAME}    ${SPACE}    _
    ${safe_suite}=    Replace String    ${safe_suite}    .    _
    ${home}=    Set Variable    ${E2E_TEMP_ROOT}${/}${safe_suite}
    ${rm_status}    ${rm_msg}=    Run Keyword And Ignore Error    Remove Directory    ${home}    recursive=True
    IF    '${rm_status}' == 'FAIL'
        Log    Could not remove previous suite home ${home}: ${rm_msg}    WARN
    END
    Create Directory    ${home}
    Set Environment Variable    CLEVERAGENTS_HOME    ${home}
    Set Suite Variable    ${SUITE_HOME}    ${home}
    Set Environment Variable    CLEVERAGENTS_AUTO_APPLY_MIGRATIONS    true
    # Explicitly disable mock AI — E2E tests use real providers
    Remove Environment Variable    CLEVERAGENTS_TESTING_USE_MOCK_AI
    # Get the actual Python executable being used
    ${python_exec}=    Evaluate    sys.executable    sys
    Set Suite Variable    ${PYTHON}    ${python_exec}

E2E Suite Teardown
    [Documentation]    Clean up the E2E test environment.
    Log    Cleaning up E2E test environment
    Run Keyword And Ignore Error    Remove Directory    ${SUITE_HOME}    recursive=True
    Remove Environment Variable    CLEVERAGENTS_HOME
    Remove Environment Variable    CLEVERAGENTS_AUTO_APPLY_MIGRATIONS

Skip If No LLM Keys
    [Documentation]    Skip the current test if no LLM API keys are available.
    ...
    ...                Checks for ANTHROPIC_API_KEY and OPENAI_API_KEY.
    ...                If neither is set, the test is skipped gracefully.
    ...                Keys are evaluated inline to avoid storing raw secrets
    ...                in Robot Framework variables (which may be logged at DEBUG level).
    ${has_keys}=    Evaluate    bool(__import__('os').environ.get('ANTHROPIC_API_KEY', '')) or bool(__import__('os').environ.get('OPENAI_API_KEY', ''))
    IF    not ${has_keys}
        Skip    No LLM API keys available (ANTHROPIC_API_KEY / OPENAI_API_KEY). Skipping E2E test.
    END

Run CleverAgents Command
    [Documentation]    Run a CleverAgents CLI command and return the result.
    ...
    ...                Executes ``python -m cleveragents <args>`` using the
    ...                venv Python.  Runs in ``SUITE_HOME`` so the CLI creates
    ...                its ``.cleveragents`` workspace inside the per-suite
    ...                temp directory, avoiding cross-run database pollution.
    ...                Returns the Process result object.
    [Arguments]    @{args}    ${expected_rc}=${0}    ${timeout}=120s    ${cwd}=${SUITE_HOME}
    ${result}=    Run Process    ${PYTHON}    -m    cleveragents    @{args}
    ...    cwd=${cwd}
    ...    timeout=${timeout}
    ...    on_timeout=kill
    ...    env:CLEVERAGENTS_HOME=${SUITE_HOME}
    ...    env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true
    ...    env:NO_COLOR=1
    Log    STDOUT: ${result.stdout}    level=DEBUG
    Log    STDERR: ${result.stderr}    level=DEBUG
    IF    '${expected_rc}' != 'None'
        Should Be Equal As Integers    ${result.rc}    ${expected_rc}
        ...    CleverAgents command failed with rc=${result.rc}. Check DEBUG-level log entries above.
    END
    RETURN    ${result}

Output Should Contain
    [Documentation]    Assert that command output contains expected text (flexible).
    ...
    ...                Checks stdout first, then stderr.  Case-insensitive by default.
    [Arguments]    ${result}    ${expected}    ${case_insensitive}=${TRUE}
    ${combined}=    Set Variable    ${result.stdout}\n${result.stderr}
    IF    ${case_insensitive}
        Should Contain    ${combined.lower()}    ${expected.lower()}
    ELSE
        Should Contain    ${combined}    ${expected}
    END

Safe Parse Json Field
    [Documentation]    Safely extract a JSON field from CLI stdout.
    ...
    ...                Tries two strategies to locate the JSON object:
    ...                1. Outer-bracket: first ``{`` to last ``}`` (works for
    ...                   single-object output with optional non-JSON preamble).
    ...                2. Last-line fallback: iterates lines from the end looking
    ...                   for a parseable JSON line (handles multi-object output
    ...                   where each object is on its own line).
    ...                Returns empty string if no JSON is found, the field is
    ...                absent, or all parse attempts fail.
    [Arguments]    ${stdout}    ${field_name}
    # Strategy 1: outer-bracket extraction (first '{' to last '}')
    ${pos}=    Evaluate    $stdout.find('{')
    IF    ${pos} == -1
        RETURN    ${EMPTY}
    END
    ${end}=    Evaluate    $stdout.rfind('}')
    IF    ${end} == -1 or ${end} < ${pos}
        RETURN    ${EMPTY}
    END
    ${json_str}=    Evaluate    $stdout[${pos}:${end}+1]
    ${status}    ${value}=    Run Keyword And Ignore Error
    ...    Evaluate    __import__('json').loads($json_str).get($field_name, '')
    IF    '${status}' == 'PASS'
        RETURN    ${value}
    END
    # Strategy 2: last-line fallback — scan lines in reverse for a parseable
    # JSON object that contains the requested field.
    ${strategy1_err}=    Set Variable    ${value}
    ${strategy2_err}=    Set Variable    no JSON lines found
    ${lines}=    Evaluate    list(reversed([l.strip() for l in $stdout.splitlines() if l.strip().startswith('{') and l.strip().endswith('}')]))
    FOR    ${line}    IN    @{lines}
        ${ls}    ${lv}=    Run Keyword And Ignore Error
        ...    Evaluate    __import__('json').loads($line).get($field_name, '')
        IF    '${ls}' == 'PASS' and '${lv}' != ''
            RETURN    ${lv}
        END
        ${strategy2_err}=    Set Variable    ${lv}
    END
    Log    Safe Parse Json Field: JSON parse failed for field '${field_name}'. Strategy 1: ${strategy1_err}; Strategy 2: ${strategy2_err}    WARN
    RETURN    ${EMPTY}

Create Temp Git Repo
    [Documentation]    Create a temporary git repository for E2E testing.
    ...
    ...                Returns the path to the created repository.
    [Arguments]    ${name}=test-repo
    ${repo_dir}=    Set Variable    ${SUITE_HOME}${/}${name}
    Create Directory    ${repo_dir}
    ${r1}=    Run Process    git    init    cwd=${repo_dir}    timeout=60s    on_timeout=kill
    Should Be Equal As Integers    ${r1.rc}    0    msg=git init failed (rc=${r1.rc}). Check DEBUG logs above.
    ${r2}=    Run Process    git    config    user.name    E2E Test    cwd=${repo_dir}    timeout=60s    on_timeout=kill
    Should Be Equal As Integers    ${r2.rc}    0    msg=git config user.name failed
    ${r3}=    Run Process    git    config    user.email    e2e@test.local    cwd=${repo_dir}    timeout=60s    on_timeout=kill
    Should Be Equal As Integers    ${r3.rc}    0    msg=git config user.email failed
    # Create an initial commit so the repo has a HEAD
    Create File    ${repo_dir}${/}README.md    # Test Repository\n
    ${r4}=    Run Process    git    add    .    cwd=${repo_dir}    timeout=60s    on_timeout=kill
    Should Be Equal As Integers    ${r4.rc}    0    msg=git add failed (rc=${r4.rc}). Check DEBUG logs above.
    ${r5}=    Run Process    git    commit    -m    Initial commit    cwd=${repo_dir}    timeout=60s    on_timeout=kill
    Should Be Equal As Integers    ${r5.rc}    0    msg=git commit failed (rc=${r5.rc}). Check DEBUG logs above.
    RETURN    ${repo_dir}