fix(test): remove skip guard and soft assertions from scientific paper E2E test
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 16s
CI / security (pull_request) Successful in 31s
CI / e2e_tests (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 2m39s
CI / unit_tests (pull_request) Successful in 5m23s
CI / coverage (pull_request) Successful in 4m59s
CI / docker (pull_request) Successful in 38s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 15s
CI / e2e_tests (push) Successful in 23s
CI / typecheck (push) Successful in 31s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 35s
CI / unit_tests (push) Successful in 2m16s
CI / docker (push) Successful in 8s
CI / integration_tests (push) Successful in 2m36s
CI / coverage (push) Successful in 4m59s
CI / benchmark-publish (push) Successful in 18m41s
CI / benchmark-regression (pull_request) Successful in 36m52s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 16s
CI / security (pull_request) Successful in 31s
CI / e2e_tests (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 2m39s
CI / unit_tests (pull_request) Successful in 5m23s
CI / coverage (pull_request) Successful in 4m59s
CI / docker (pull_request) Successful in 38s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 15s
CI / e2e_tests (push) Successful in 23s
CI / typecheck (push) Successful in 31s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 35s
CI / unit_tests (push) Successful in 2m16s
CI / docker (push) Successful in 8s
CI / integration_tests (push) Successful in 2m36s
CI / coverage (push) Successful in 4m59s
CI / benchmark-publish (push) Successful in 18m41s
CI / benchmark-regression (pull_request) Successful in 36m52s
- Remove Require OpenAI Key skip guard that silently skipped the suite - Replace Run Keyword And Return Status soft RC check with direct assertion - Convert conditional LaTeX presence check to hard assertions - Convert conditional latex_source context field check to hard assertion - Remove dead stderr checks (stderr redirected to stdout via stderr=STDOUT)
This commit was merged in pull request #906.
This commit is contained in:
@@ -25,7 +25,6 @@ Scientific Paper Writer Full Workflow
|
||||
Log Testing intro stage...
|
||||
${result}= Run Paper Command Hello
|
||||
Should Contain ${result.stdout} Paper Writer
|
||||
Should Not Contain ${result.stderr} Error
|
||||
|
||||
# Test 2: Advance to discovery
|
||||
Log Advancing to discovery stage...
|
||||
@@ -59,7 +58,6 @@ Scientific Paper Writer Full Workflow
|
||||
# Test 6: Test brainstorming stage
|
||||
Log Testing brainstorming stage...
|
||||
${result}= Run Paper Command I want to write about the impact of machine learning on healthcare, focusing on diagnostic imaging and patient outcomes timeout=90s
|
||||
Should Not Contain ${result.stderr} Error
|
||||
Length Should Be Greater Than ${result.stdout} 50
|
||||
|
||||
# Test 7: Verify stage may have advanced (brainstorming stage might auto-advance after LLM response)
|
||||
@@ -68,10 +66,7 @@ Scientific Paper Writer Full Workflow
|
||||
# Test 8: Advance toward later stages (using structure or latex_generation as targets)
|
||||
Log Advancing toward later stages...
|
||||
${result}= Run Paper Command !next structure timeout=120s
|
||||
# Note: May fail if required context isn't set, which is acceptable for this E2E test
|
||||
# The important thing is the command doesn't crash
|
||||
${rc_ok}= Run Keyword And Return Status Should Be Equal As Integers ${result.rc} 0
|
||||
# Note: Stage may auto-advance; exact stage verification removed
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
|
||||
# Test 9: Verify stage list command works
|
||||
Log Verifying stage list command...
|
||||
@@ -127,24 +122,17 @@ Scientific Paper Writer LaTeX Generation
|
||||
... stderr=STDOUT
|
||||
... timeout=180s
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Not Contain ${result.stderr} Error
|
||||
Should Not Contain ${result.stderr} timed out
|
||||
# System should produce SOME output (not just echo the input)
|
||||
Should Not Be Equal ${result.stdout} Generate the complete LaTeX document
|
||||
|
||||
# Verify LaTeX structure (if LaTeX was generated)
|
||||
# Verify LaTeX structure
|
||||
Log Verifying LaTeX document structure...
|
||||
${has_latex}= Run Keyword And Return Status Should Contain ${result.stdout} \\documentclass
|
||||
IF ${has_latex}
|
||||
Should Contain ${result.stdout} \\begin{document}
|
||||
Should Contain ${result.stdout} \\end{document}
|
||||
Should Contain ${result.stdout} \\section
|
||||
Log LaTeX document was generated successfully
|
||||
ELSE
|
||||
Log LaTeX generation may require manual stage management or different prompting
|
||||
END
|
||||
Should Contain ${result.stdout} \\documentclass
|
||||
Should Contain ${result.stdout} \\begin{document}
|
||||
Should Contain ${result.stdout} \\end{document}
|
||||
Should Contain ${result.stdout} \\section
|
||||
|
||||
# Verify context was updated (may or may not have latex_source depending on stage behavior)
|
||||
# Verify context was updated
|
||||
Log Verifying context is accessible...
|
||||
${result}= Run Process ${PYTHON} -m cleveragents actor run
|
||||
... -c ${CONFIG_FILE}
|
||||
@@ -155,12 +143,7 @@ Scientific Paper Writer LaTeX Generation
|
||||
... -p !context
|
||||
... stderr=STDOUT
|
||||
... timeout=30s
|
||||
${has_latex_context}= Run Keyword And Return Status Should Contain ${result.stdout} latex_source
|
||||
IF ${has_latex_context}
|
||||
Log Context includes latex_source field
|
||||
ELSE
|
||||
Log Context does not include latex_source (may require different stage setup)
|
||||
END
|
||||
Should Contain ${result.stdout} latex_source
|
||||
|
||||
Log LaTeX generation test completed
|
||||
|
||||
@@ -235,7 +218,6 @@ Scientific Paper Writer Stage Navigation
|
||||
|
||||
*** Keywords ***
|
||||
Setup Test Environment
|
||||
Require OpenAI Key
|
||||
${timestamp}= Get Time epoch
|
||||
Set Suite Variable ${TEST_ID} ${timestamp}
|
||||
Set Suite Variable ${CONTEXT_NAME} paper_e2e_${TEST_ID}
|
||||
@@ -277,6 +259,3 @@ Length Should Be Greater Than
|
||||
${length}= Get Length ${text}
|
||||
Should Be True ${length} > ${min_length} Text length ${length} is not greater than ${min_length}
|
||||
|
||||
Require OpenAI Key
|
||||
${api_key}= Get Environment Variable OPENAI_API_KEY default=
|
||||
Run Keyword If '${api_key}' == '' Skip OPENAI_API_KEY not set; skipping LLM integration tests
|
||||
|
||||
Reference in New Issue
Block a user