fix(test): remove skip guard and soft assertions from scientific paper E2E test
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 16s
CI / security (pull_request) Successful in 31s
CI / e2e_tests (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 2m39s
CI / unit_tests (pull_request) Successful in 5m23s
CI / coverage (pull_request) Successful in 4m59s
CI / docker (pull_request) Successful in 38s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 15s
CI / e2e_tests (push) Successful in 23s
CI / typecheck (push) Successful in 31s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 35s
CI / unit_tests (push) Successful in 2m16s
CI / docker (push) Successful in 8s
CI / integration_tests (push) Successful in 2m36s
CI / coverage (push) Successful in 4m59s
CI / benchmark-publish (push) Successful in 18m41s
CI / benchmark-regression (pull_request) Successful in 36m52s

- Remove Require OpenAI Key skip guard that silently skipped the suite
- Replace Run Keyword And Return Status soft RC check with direct assertion
- Convert conditional LaTeX presence check to hard assertions
- Convert conditional latex_source context field check to hard assertion
- Remove dead stderr checks (stderr redirected to stdout via stderr=STDOUT)
This commit was merged in pull request #906.
This commit is contained in:
2026-03-14 00:00:16 +00:00
committed by Forgejo
parent 77a7813f0e
commit c169cb201d
+8 -29
View File
@@ -25,7 +25,6 @@ Scientific Paper Writer Full Workflow
Log Testing intro stage...
${result}= Run Paper Command Hello
Should Contain ${result.stdout} Paper Writer
Should Not Contain ${result.stderr} Error
# Test 2: Advance to discovery
Log Advancing to discovery stage...
@@ -59,7 +58,6 @@ Scientific Paper Writer Full Workflow
# Test 6: Test brainstorming stage
Log Testing brainstorming stage...
${result}= Run Paper Command I want to write about the impact of machine learning on healthcare, focusing on diagnostic imaging and patient outcomes timeout=90s
Should Not Contain ${result.stderr} Error
Length Should Be Greater Than ${result.stdout} 50
# Test 7: Verify stage may have advanced (brainstorming stage might auto-advance after LLM response)
@@ -68,10 +66,7 @@ Scientific Paper Writer Full Workflow
# Test 8: Advance toward later stages (using structure or latex_generation as targets)
Log Advancing toward later stages...
${result}= Run Paper Command !next structure timeout=120s
# Note: May fail if required context isn't set, which is acceptable for this E2E test
# The important thing is the command doesn't crash
${rc_ok}= Run Keyword And Return Status Should Be Equal As Integers ${result.rc} 0
# Note: Stage may auto-advance; exact stage verification removed
Should Be Equal As Integers ${result.rc} 0
# Test 9: Verify stage list command works
Log Verifying stage list command...
@@ -127,24 +122,17 @@ Scientific Paper Writer LaTeX Generation
... stderr=STDOUT
... timeout=180s
Should Be Equal As Integers ${result.rc} 0
Should Not Contain ${result.stderr} Error
Should Not Contain ${result.stderr} timed out
# System should produce SOME output (not just echo the input)
Should Not Be Equal ${result.stdout} Generate the complete LaTeX document
# Verify LaTeX structure (if LaTeX was generated)
# Verify LaTeX structure
Log Verifying LaTeX document structure...
${has_latex}= Run Keyword And Return Status Should Contain ${result.stdout} \\documentclass
IF ${has_latex}
Should Contain ${result.stdout} \\begin{document}
Should Contain ${result.stdout} \\end{document}
Should Contain ${result.stdout} \\section
Log LaTeX document was generated successfully
ELSE
Log LaTeX generation may require manual stage management or different prompting
END
Should Contain ${result.stdout} \\documentclass
Should Contain ${result.stdout} \\begin{document}
Should Contain ${result.stdout} \\end{document}
Should Contain ${result.stdout} \\section
# Verify context was updated (may or may not have latex_source depending on stage behavior)
# Verify context was updated
Log Verifying context is accessible...
${result}= Run Process ${PYTHON} -m cleveragents actor run
... -c ${CONFIG_FILE}
@@ -155,12 +143,7 @@ Scientific Paper Writer LaTeX Generation
... -p !context
... stderr=STDOUT
... timeout=30s
${has_latex_context}= Run Keyword And Return Status Should Contain ${result.stdout} latex_source
IF ${has_latex_context}
Log Context includes latex_source field
ELSE
Log Context does not include latex_source (may require different stage setup)
END
Should Contain ${result.stdout} latex_source
Log LaTeX generation test completed
@@ -235,7 +218,6 @@ Scientific Paper Writer Stage Navigation
*** Keywords ***
Setup Test Environment
Require OpenAI Key
${timestamp}= Get Time epoch
Set Suite Variable ${TEST_ID} ${timestamp}
Set Suite Variable ${CONTEXT_NAME} paper_e2e_${TEST_ID}
@@ -277,6 +259,3 @@ Length Should Be Greater Than
${length}= Get Length ${text}
Should Be True ${length} > ${min_length} Text length ${length} is not greater than ${min_length}
Require OpenAI Key
${api_key}= Get Environment Variable OPENAI_API_KEY default=
Run Keyword If '${api_key}' == '' Skip OPENAI_API_KEY not set; skipping LLM integration tests