cleveragents-core/robot/m2_e2e_verification.robot

*** Settings ***
Documentation    End-to-end verification of M2 success criteria:
...              actor compiler, tool routing, validation runner,
...              and multi-file ChangeSet generation.
Resource         ${CURDIR}/common.resource
Suite Setup      Setup Test Environment
Suite Teardown   Cleanup Test Environment

*** Variables ***
${HELPER}    ${CURDIR}/helper_m2_e2e_verification.py

*** Test Cases ***
Actor YAML Create And Load Via Loader
    [Documentation]    Create an actor YAML file and load it via ActorLoader.
    ...                Verifies that the YAML parses correctly and the actor
    ...                is discovered with the expected name and type.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    actor-yaml-create-load    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-actor-yaml-create-load-ok

Actor Add Via Config CLI
    [Documentation]    Create an actor YAML config and load it via the
    ...                ``agents actor add --config`` CLI command. Verifies
    ...                the actor is registered with the expected attributes.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    actor-add-config-cli    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-actor-add-config-cli-ok

Action Create From Config Via CLI
    [Documentation]    Create an action referencing a custom actor via
    ...                ``agents action create --config``. Verifies the
    ...                action is created with the correct name and actors.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    action-create    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-action-create-ok

Plan Use And Execute With Custom Actor
    [Documentation]    Run ``plan use`` and ``plan execute`` with a custom
    ...                actor. Verifies the plan transitions through the
    ...                expected phases (strategize -> execute).
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    plan-use-execute    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-plan-use-execute-ok

Actor YAML Parse And Validate
    [Documentation]    Verify actor YAML files parse and validate correctly.
    ...                Tests valid graph/LLM YAML and confirms invalid YAML
    ...                is rejected with appropriate errors.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    actor-yaml-parse-validate    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-actor-yaml-parse-validate-ok

Actor Compile To LangGraph StateGraph
    [Documentation]    Verify actors compile to LangGraph StateGraphs.
    ...                Compiles a graph actor from examples and asserts
    ...                nodes, edges, entry point, and metadata. Also
    ...                confirms LLM actors are rejected by the compiler.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    actor-compile-stategraph    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-actor-compile-stategraph-ok

Tool Router Resolves External Tools
    [Documentation]    Verify the tool router can resolve external tools
    ...                across OpenAI and Anthropic provider formats.
    ...                Tests format detection, normalization, and routing.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    tool-router-resolve-external    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-tool-router-resolve-external-ok

Validation Runner Executes Required And Informational
    [Documentation]    Verify the validation runner executes both required
    ...                and informational validations correctly. Required
    ...                validations must pass; informational failures are
    ...                reported but do not block execution.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    validation-runner-execute    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-validation-runner-execute-ok

Multi File Generation Produces Correct ChangeSet
    [Documentation]    Verify multi-file generation produces a correct
    ...                ChangeSet with the expected number of entries,
    ...                operations, and summary counts.
    ${result}=    Run Process    ${PYTHON}    ${HELPER}    changeset-multifile    cwd=${WORKSPACE}
    Log    ${result.stdout}
    Log    ${result.stderr}
    Should Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    m2-changeset-multifile-ok