cleveragents-core/robot/rxpy_route_validation.robot

*** Settings ***
Documentation     Integration tests for RxPY route validation in run command
Library           Process
Library           OperatingSystem
Library           String
Library           Collections
Resource    ${CURDIR}/v2_paths.resource
Suite Setup       Setup Test Environment
Suite Teardown    Cleanup Test Environment

*** Variables ***
${TEST_DIR}           ${TEMPDIR}/rxpy_validation_test
${RXPY_CONFIG}        ${TEST_DIR}/rxpy_config.yaml
${LANGGRAPH_CONFIG}   ${TEST_DIR}/langgraph_config.yaml
${CONTEXT_DIR}        ${TEST_DIR}/test_contexts
${UNIQUE_ID}          ${EMPTY}
${RXPY_RUN_MODE_ERROR}    RxPY stream routes are only supported in run mode unless allowed. Use --allow-rxpy-in-run-mode to bypass.

*** Test Cases ***
Test Run Command Rejects RxPY Routes
    [Documentation]    Verify run command rejects RxPY stream routes
    [Tags]    rxpy    validation    run

    Create RxPY Config File

    # Try to run with RxPY routes - should fail
    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${RXPY_CONFIG}
    ...    --unsafe
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    Should Not Be Equal As Integers    ${result.rc}    0    Command should have failed
    Should Contain    ${result.stdout}    ${RXPY_RUN_MODE_ERROR}
    Should Contain    ${result.stdout}    allow-rxpy-in-run-mode

Test Run Command With Context Does Not Create Context
    [Documentation]    Verify that no context is created when RxPY validation fails
    [Tags]    rxpy    validation    context

    ${context_name} =    Set Variable    rxpy_test_ctx_${UNIQUE_ID}

    Create RxPY Config File

    # Ensure context doesn't exist
    Delete Context If Exists    ${context_name}

    # Try to run with --context flag
    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${RXPY_CONFIG}
    ...    --unsafe
    ...    --context    ${context_name}
    ...    --context-dir    ${CONTEXT_DIR}
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    Should Not Be Equal As Integers    ${result.rc}    0    Command should have failed
    Should Contain    ${result.stdout}    ${RXPY_RUN_MODE_ERROR}

    # Verify context was NOT created
    ${context_exists} =    Context Exists    ${context_name}
    Should Not Be True    ${context_exists}    Context should not have been created

Test Run Command With LangGraph Routes Succeeds
    [Documentation]    Verify run command works with LangGraph routes
    [Tags]    langgraph    validation    run    slow

    Create LangGraph Config File

    # Run with LangGraph routes - should succeed
    ${context_name} =    Set Variable    rxpy_test_ctx_${UNIQUE_ID}
    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${LANGGRAPH_CONFIG}
    ...    --context-dir    ${CONTEXT_DIR}
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    # For now, might fail on actual execution but shouldn't have route validation error
    ${output} =    Set Variable    ${result.stdout}
    Should Not Contain    ${output}    ${RXPY_RUN_MODE_ERROR}

Test Run Command Accepts RxPY Routes When Allowed
    [Documentation]    Verify actor run works with RxPY routes when allowed
    [Tags]    rxpy    run    slow

    Create RxPY Config File

    ${context_name} =    Set Variable    rxpy_test_ctx_${UNIQUE_ID}
    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${RXPY_CONFIG}
    ...    --unsafe
    ...    --allow-rxpy-in-run-mode
    ...    --context    ${context_name}
    ...    --context-dir    ${CONTEXT_DIR}
    ...    test-actor    test
    ...    stderr=STDOUT
    ...    timeout=120s    on_timeout=kill

    Should Be Equal As Integers    ${result.rc}    0
    Should Not Contain    ${result.stdout}    ${RXPY_RUN_MODE_ERROR}

Test Error Message Content
    [Documentation]    Verify error message contains helpful information
    [Tags]    rxpy    validation    error-message

    Create RxPY Config File

    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${RXPY_CONFIG}
    ...    --unsafe
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    Should Not Be Equal As Integers    ${result.rc}    0

    # Check error message contains all important information
    ${output} =    Convert To Lower Case    ${result.stdout}
    Should Contain    ${output}    nested
    Should Contain    ${output}    completion detection
    Should Contain    ${output}    run mode
    Should Contain    ${output}    langgraph

Test Run With Nested Switch Operators
    [Documentation]    Verify run command rejects configs with nested switch operators
    [Tags]    rxpy    nested    validation

    Create Nested Switch Config File

    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${TEST_DIR}/nested_switch_config.yaml
    ...    --unsafe
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    Should Not Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    ${RXPY_RUN_MODE_ERROR}

Test Mixed Route Types Detection
    [Documentation]    Verify detection works with mix of RxPY and LangGraph routes
    [Tags]    rxpy    mixed    validation

    Create Mixed Routes Config File

    # Should detect RxPY routes even if LangGraph routes also present
    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
    ...    -c    ${TEST_DIR}/mixed_config.yaml
    ...    --unsafe
    ...    test-actor    test
    ...    stderr=STDOUT    timeout=120s    on_timeout=kill

    Should Not Be Equal As Integers    ${result.rc}    0
    Should Contain    ${result.stdout}    ${RXPY_RUN_MODE_ERROR}

Test Context File Not Created On Multiple Runs
    [Documentation]    Verify repeated failed runs do not create context
    [Tags]    rxpy    context    validation

    ${context_name} =    Set Variable    rxpy_multi_test_${UNIQUE_ID}

    Create RxPY Config File
    Delete Context If Exists    ${context_name}

    # Run multiple times
    FOR    ${i}    IN RANGE    3
        ${result} =    Run Process    ${PYTHON}    -m    cleveragents    actor    run
        ...    -c    ${RXPY_CONFIG}
        ...    --unsafe
        ...    --context    ${context_name}
        ...    --context-dir    ${CONTEXT_DIR}
        ...    test-actor    test
        ...    stderr=STDOUT    timeout=120s    on_timeout=kill

        Should Not Be Equal As Integers    ${result.rc}    0
    END

    # Verify context still doesn't exist after multiple attempts
    ${context_exists} =    Context Exists    ${context_name}
    Should Not Be True    ${context_exists}

*** Keywords ***
Setup Test Environment
    [Documentation]    Setup test environment
    # Generate unique ID for this test run first, then scope all temp paths
    # to that ID so parallel suites cannot collide on shared temp files.
    ${timestamp} =    Get Time    epoch
    Set Suite Variable    ${UNIQUE_ID}    ${timestamp}
    ${test_dir} =    Set Variable    ${TEMPDIR}/rxpy_validation_test_${timestamp}
    ${rxpy_config} =    Set Variable    ${test_dir}/rxpy_config.yaml
    ${langgraph_config} =    Set Variable    ${test_dir}/langgraph_config.yaml
    ${context_dir} =    Set Variable    ${test_dir}/test_contexts
    Set Suite Variable    ${TEST_DIR}    ${test_dir}
    Set Suite Variable    ${RXPY_CONFIG}    ${rxpy_config}
    Set Suite Variable    ${LANGGRAPH_CONFIG}    ${langgraph_config}
    Set Suite Variable    ${CONTEXT_DIR}    ${context_dir}

    Create Directory    ${TEST_DIR}
    Create Directory    ${CONTEXT_DIR}

Cleanup Test Environment
    [Documentation]    Clean up test environment
    Remove Directory    ${TEST_DIR}    recursive=True

Create RxPY Config File
    [Documentation]    Create a test config with RxPY stream routes
    ${config} =    Catenate    SEPARATOR=\n
    ...    actors:
    ...    ${SPACE*2}echo_actor:
    ...    ${SPACE*4}type: tool
    ...    ${SPACE*4}config:
    ...    ${SPACE*6}tools:
    ...    ${SPACE*8}- echo
    ...    ${SPACE*6}safe_mode: true
    ...    ${EMPTY}
    ...    routes:
    ...    ${SPACE*2}echo_stream:
    ...    ${SPACE*4}type: stream
    ...    ${SPACE*4}stream_type: cold
    ...    ${SPACE*4}operators:
    ...    ${SPACE*6}- type: map
    ...    ${SPACE*8}params:
    ...    ${SPACE*10}actor: echo_actor
    ...    ${SPACE*4}publications:
    ...    ${SPACE*6}- __output__
    ...    ${EMPTY}
    ...    merges:
    ...    ${SPACE*2}- sources: [__input__]
    ...    ${SPACE*4}target: echo_stream

    Create File    ${RXPY_CONFIG}    ${config}

Create LangGraph Config File
    [Documentation]    Create a test config with LangGraph routes
    ${config} =    Catenate    SEPARATOR=\n
    ...    actors:
    ...    ${SPACE*2}echo_actor:
    ...    ${SPACE*4}type: tool
    ...    ${SPACE*4}config:
    ...    ${SPACE*6}tools:
    ...    ${SPACE*8}- echo
    ...    ${SPACE*6}safe_mode: true
    ...    ${EMPTY}
    ...    routes:
    ...    ${SPACE*2}main:
    ...    ${SPACE*4}type: graph
    ...    ${SPACE*4}nodes:
    ...    ${SPACE*6}- id: start
    ...    ${SPACE*8}actor: echo_actor
    ...    ${SPACE*4}edges:
    ...    ${SPACE*6}- from: __input__
    ...    ${SPACE*8}to: start
    ...    ${SPACE*6}- from: start
    ...    ${SPACE*8}to: __output__
    ...    ${EMPTY}
    ...    merges:
    ...    ${SPACE*2}- sources: [__input__]
    ...    ${SPACE*4}target: main

    Create File    ${LANGGRAPH_CONFIG}    ${config}

Create Nested Switch Config File
    [Documentation]    Create a config with nested switch operators
    ${config} =    Catenate    SEPARATOR=\n
    ...    actors:
    ...    ${SPACE*2}test_actor:
    ...    ${SPACE*4}type: tool
    ...    ${SPACE*4}config:
    ...    ${SPACE*6}tools:
    ...    ${SPACE*8}- echo
    ...    ${SPACE*6}safe_mode: true
    ...    ${EMPTY}
    ...    routes:
    ...    ${SPACE*2}main:
    ...    ${SPACE*4}type: stream
    ...    ${SPACE*4}stream_type: cold
    ...    ${SPACE*4}operators:
    ...    ${SPACE*6}- type: switch
    ...    ${SPACE*8}params:
    ...    ${SPACE*10}cases:
    ...    ${SPACE*12}- condition: "test"
    ...    ${SPACE*14}route: nested_route
    ...    ${SPACE*2}nested_route:
    ...    ${SPACE*4}type: stream
    ...    ${SPACE*4}stream_type: cold
    ...    ${SPACE*4}operators:
    ...    ${SPACE*6}- type: map
    ...    ${SPACE*8}params:
    ...    ${SPACE*10}actor: test_actor
    ...    ${SPACE*4}publications:
    ...    ${SPACE*6}- __output__
    ...    ${EMPTY}
    ...    merges:
    ...    ${SPACE*2}- sources: [__input__]
    ...    ${SPACE*4}target: main

    Create File    ${TEST_DIR}/nested_switch_config.yaml    ${config}

Create Mixed Routes Config File
    [Documentation]    Create a config with both RxPY and LangGraph routes
    ${config} =    Catenate    SEPARATOR=\n
    ...    actors:
    ...    ${SPACE*2}test_actor:
    ...    ${SPACE*4}type: tool
    ...    ${SPACE*4}config:
    ...    ${SPACE*6}tools:
    ...    ${SPACE*8}- echo
    ...    ${SPACE*6}safe_mode: true
    ...    ${EMPTY}
    ...    routes:
    ...    ${SPACE*2}stream_route:
    ...    ${SPACE*4}type: stream
    ...    ${SPACE*4}stream_type: cold
    ...    ${SPACE*4}operators:
    ...    ${SPACE*6}- type: map
    ...    ${SPACE*8}params:
    ...    ${SPACE*10}actor: test_actor
    ...    ${SPACE*4}publications:
    ...    ${SPACE*6}- __output__
    ...    ${SPACE*2}graph_route:
    ...    ${SPACE*4}type: graph
    ...    ${SPACE*4}entry_point: start
    ...    ${SPACE*4}nodes:
    ...    ${SPACE*6}start:
    ...    ${SPACE*8}type: actor
    ...    ${SPACE*8}actor: test_actor
    ...    ${SPACE*4}edges:
    ...    ${SPACE*6}- source: start
    ...    ${SPACE*8}target: end
    ...    ${EMPTY}
    ...    merges:
    ...    ${SPACE*2}- sources: [__input__]
    ...    ${SPACE*4}target: stream_route

    Create File    ${TEST_DIR}/mixed_config.yaml    ${config}

Delete Context If Exists
    [Documentation]    Delete a context if it exists
    [Arguments]    ${context_name}

    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    context    delete
    ...    ${context_name}
    ...    --context-dir    ${CONTEXT_DIR}
    ...    --yes
    ...    stderr=STDOUT

    # Don't fail if context doesn't exist
    Log    Deleted context ${context_name} (if it existed)

Context Exists
    [Documentation]    Check if a context exists
    [Arguments]    ${context_name}

    ${result} =    Run Process    ${PYTHON}    -m    cleveragents    context    show
    ...    ${context_name}
    ...    --context-dir    ${CONTEXT_DIR}
    ...    stderr=STDOUT

    # If return code is 0, context exists
    ${exists} =    Evaluate    ${result.rc} == 0
    RETURN    ${exists}