forked from cleveragents/cleveragents-core
brent.edwards
3eecb79003
TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839
85 lines
2.9 KiB
Plaintext
85 lines
2.9 KiB
Plaintext
*** Settings ***
|
|
Library Process
|
|
Library OperatingSystem
|
|
Library String
|
|
Resource ${CURDIR}/v2_paths.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
${CONFIG_FILE} ${CURDIR}/../examples/scientific_paper_writer.yaml
|
|
${CONTEXT_DIR} ${TEMPDIR}/paper_basic_contexts
|
|
${CONTEXT_NAME} ${EMPTY}
|
|
${TEST_ID} ${EMPTY}
|
|
|
|
*** Test Cases ***
|
|
Scientific Paper Writer Basic Integration Test
|
|
[Documentation] Minimal integration test to verify scientific paper writer works with real APIs
|
|
[Tags] integration code_blocks
|
|
[Timeout] 2 minutes
|
|
|
|
# Test 1: Basic intro response (uses OpenAI API)
|
|
${result}= Run Process ${PYTHON} -m cleveragents actor run
|
|
... -c ${CONFIG_FILE}
|
|
... --context ${CONTEXT_NAME}
|
|
... --context-dir ${CONTEXT_DIR}
|
|
... --unsafe
|
|
... --allow-rxpy-in-run-mode
|
|
... -p Hello
|
|
... stderr=DEVNULL
|
|
... timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} Paper Writer
|
|
Log Intro test passed
|
|
|
|
# Test 2: Command handler (no API needed)
|
|
${result}= Run Process ${PYTHON} -m cleveragents actor run
|
|
... -c ${CONFIG_FILE}
|
|
... --context ${CONTEXT_NAME}
|
|
... --context-dir ${CONTEXT_DIR}
|
|
... --unsafe
|
|
... --allow-rxpy-in-run-mode
|
|
... -p !help
|
|
... stderr=DEVNULL
|
|
... timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} Available Commands
|
|
Log Command handler test passed
|
|
|
|
# Test 3: Echo test to verify framework works
|
|
${result}= Run Process ${PYTHON} -m cleveragents actor run
|
|
... -c ${V2_CONFIG_DIR}/simple_echo_config.yaml
|
|
... --unsafe
|
|
... --allow-rxpy-in-run-mode
|
|
... -p Test message
|
|
... stderr=DEVNULL
|
|
... timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} Test message
|
|
Log Echo test passed
|
|
|
|
Log All integration tests passed successfully
|
|
|
|
*** Keywords ***
|
|
Setup Test Environment
|
|
${timestamp}= Get Time epoch
|
|
Set Suite Variable ${TEST_ID} ${timestamp}
|
|
Set Suite Variable ${CONTEXT_NAME} test_paper_simple_${TEST_ID}
|
|
Set Suite Variable ${CONTEXT_DIR} ${TEMPDIR}/paper_basic_contexts
|
|
Create Directory ${CONTEXT_DIR}
|
|
Log Test environment setup complete with ID: ${TEST_ID}
|
|
|
|
Cleanup Test Environment
|
|
Run Keyword If '${CONTEXT_DIR}' != '${EMPTY}' Remove Directory ${CONTEXT_DIR} recursive=True
|
|
Log Test environment cleaned up
|
|
|
|
Should Contain Any
|
|
[Arguments] ${text} @{patterns}
|
|
FOR ${pattern} IN @{patterns}
|
|
${status}= Run Keyword And Return Status Should Contain ${text} ${pattern}
|
|
IF ${status} RETURN
|
|
END
|
|
Fail Text does not contain any of: @{patterns}
|
|
|