cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

fix(test): add return code checks to intermediate commands in core CLI test
CI / benchmark-publish (pull_request) Has been skipped
Details
CI / lint (pull_request) Successful in 15s
Details
CI / build (pull_request) Successful in 15s
Details
CI / quality (pull_request) Successful in 17s
Details
CI / e2e_tests (pull_request) Successful in 27s
Details
CI / security (pull_request) Successful in 32s
Details
CI / typecheck (pull_request) Successful in 32s
Details
CI / unit_tests (pull_request) Successful in 2m2s
Details
CI / docker (pull_request) Successful in 9s
Details
CI / integration_tests (pull_request) Successful in 2m46s
Details
CI / coverage (pull_request) Successful in 5m2s
Details
CI / lint (push) Successful in 11s
Details
CI / build (push) Successful in 15s
Details
CI / quality (push) Successful in 15s
Details
CI / e2e_tests (push) Successful in 23s
Details
CI / security (push) Successful in 29s
Details
CI / typecheck (push) Successful in 36s
Details
CI / benchmark-regression (push) Has been skipped
Details
CI / unit_tests (push) Successful in 2m10s
Details
CI / integration_tests (push) Successful in 2m41s
Details
CI / docker (push) Successful in 37s
Details
CI / coverage (push) Successful in 6m58s
Details
CI / benchmark-publish (push) Successful in 19m25s
Details
CI / benchmark-regression (pull_request) Successful in 35m29s
Details

Browse Source 
Add rc == 0 assertions to ~11 intermediate Run Process calls in core_cli_commands.robot that were used as setup steps without any return code verification. If any setup command (init, tell, build, context add, plan new) fails silently, subsequent test assertions could pass for wrong reasons. All setup commands now capture their result and assert rc == 0.
This commit was merged in pull request #916.
This commit is contained in:
Jeffrey Phillips Freeman
2026-03-14 00:00:36 +00:00
committed by Forgejo
parent 14884ba15e
commit 586ea4557f
1 changed files with 32 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
robot/core_cli_commands.robot
+32 -16
View File
@@ -45,7 +45,8 @@ Test Project Initialization
Test Project Cannot Initialize Twice
[Documentation] Verify project cannot be initialized twice without force
Create Directory ${TEST_DIR}/project2
Run Process ${PYTHON} -m cleveragents init first-project cwd=${TEST_DIR}/project2 timeout=120s
${first_init}= Run Process ${PYTHON} -m cleveragents init first-project cwd=${TEST_DIR}/project2 timeout=120s
Should Be Equal As Numbers ${first_init.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents init second-project
... cwd=${TEST_DIR}/project2 timeout=120s
Should Not Be Equal As Numbers ${result.rc} 0
@@ -55,7 +56,8 @@ Test Project Cannot Initialize Twice
Test Force Reinitialize Project
[Documentation] Test force reinitialization of a project
Create Directory ${TEST_DIR}/project3
Run Process ${PYTHON} -m cleveragents init first-project cwd=${TEST_DIR}/project3 timeout=120s
${first_init}= Run Process ${PYTHON} -m cleveragents init first-project cwd=${TEST_DIR}/project3 timeout=120s
Should Be Equal As Numbers ${first_init.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents init second-project --force
... cwd=${TEST_DIR}/project3 timeout=120s
Should Be Equal As Numbers ${result.rc} 0
@@ -64,7 +66,8 @@ Test Force Reinitialize Project
Test Context Add Files
[Documentation] Test adding files to context
Create Directory ${TEST_DIR}/project4
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project4 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project4 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
Create File ${TEST_DIR}/project4/test.py print('hello world')
${result} = Run Process ${PYTHON} -m cleveragents context add test.py
... cwd=${TEST_DIR}/project4 timeout=120s
@@ -94,9 +97,11 @@ Test Context List Files
Test Context Clear
[Documentation] Test clearing context files
Create Directory ${TEST_DIR}/project6
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project6 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project6 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
Create File ${TEST_DIR}/project6/test.py # Test file
Run Process ${PYTHON} -m cleveragents context add test.py cwd=${TEST_DIR}/project6 timeout=120s
${add_result}= Run Process ${PYTHON} -m cleveragents context add test.py cwd=${TEST_DIR}/project6 timeout=120s
Should Be Equal As Numbers ${add_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents context clear --yes
... cwd=${TEST_DIR}/project6 timeout=120s
Should Be Equal As Numbers ${result.rc} 0
@@ -105,7 +110,8 @@ Test Context Clear
Test Plan Creation With Tell
[Documentation] Test creating a plan using tell command
Create Directory ${TEST_DIR}/project7
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project7 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project7 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents tell Add error handling to main function
... cwd=${TEST_DIR}/project7 timeout=120s
Should Be Equal As Numbers ${result.rc} 0
@@ -115,8 +121,10 @@ Test Plan Creation With Tell
Test Plan Build
[Documentation] Test building a plan
Create Directory ${TEST_DIR}/project8
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project8 timeout=120s
Run Process ${PYTHON} -m cleveragents tell Create example code cwd=${TEST_DIR}/project8 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project8 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
${tell_result}= Run Process ${PYTHON} -m cleveragents tell Create example code cwd=${TEST_DIR}/project8 timeout=120s
Should Be Equal As Numbers ${tell_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents build cwd=${TEST_DIR}/project8
... env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s
Should Be Equal As Numbers ${result.rc} 0
@@ -127,10 +135,13 @@ Test Plan Build
Test Plan Apply
[Documentation] Test applying plan changes
Create Directory ${TEST_DIR}/project9
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project9 timeout=120s
Run Process ${PYTHON} -m cleveragents tell Create example file cwd=${TEST_DIR}/project9 timeout=120s
Run Process ${PYTHON} -m cleveragents build cwd=${TEST_DIR}/project9
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project9 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
${tell_result}= Run Process ${PYTHON} -m cleveragents tell Create example file cwd=${TEST_DIR}/project9 timeout=120s
Should Be Equal As Numbers ${tell_result.rc} 0
${build_result}= Run Process ${PYTHON} -m cleveragents build cwd=${TEST_DIR}/project9
... env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s
Should Be Equal As Numbers ${build_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents apply --yes cwd=${TEST_DIR}/project9 timeout=120s
Should Be Equal As Numbers ${result.rc} 0
Should Contain ${result.stdout} Successfully applied
@@ -139,9 +150,12 @@ Test Plan Apply
Test Plan List
[Documentation] Test listing plans
Create Directory ${TEST_DIR}/project10
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project10 timeout=120s
Run Process ${PYTHON} -m cleveragents tell First plan cwd=${TEST_DIR}/project10 timeout=120s
Run Process ${PYTHON} -m cleveragents plan new second-plan cwd=${TEST_DIR}/project10 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project10 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
${tell_result}= Run Process ${PYTHON} -m cleveragents tell First plan cwd=${TEST_DIR}/project10 timeout=120s
Should Be Equal As Numbers ${tell_result.rc} 0
${plan_result}= Run Process ${PYTHON} -m cleveragents plan new second-plan cwd=${TEST_DIR}/project10 timeout=120s
Should Be Equal As Numbers ${plan_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents plan list cwd=${TEST_DIR}/project10 timeout=120s
Should Be Equal As Numbers ${result.rc} 0
Should Contain ${result.stdout} Plans (3 total)
@@ -149,7 +163,8 @@ Test Plan List
Test Shortcut Commands Work
[Documentation] Test that shortcut commands work properly
Create Directory ${TEST_DIR}/project11
Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project11 timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME} cwd=${TEST_DIR}/project11 timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
Create File ${TEST_DIR}/project11/test.txt Test content
# Test context-load shortcut
${result} = Run Process ${PYTHON} -m cleveragents context-load test.txt
@@ -200,7 +215,8 @@ Test Project Status Without Project
Test Project Status With Project
[Documentation] Test project status command with initialized project
Create Directory ${TEST_DIR}/with_project
Run Process ${PYTHON} -m cleveragents init status-test cwd=${TEST_DIR}/with_project timeout=120s
${init_result}= Run Process ${PYTHON} -m cleveragents init status-test cwd=${TEST_DIR}/with_project timeout=120s
Should Be Equal As Numbers ${init_result.rc} 0
${result} = Run Process ${PYTHON} -m cleveragents project status
... cwd=${TEST_DIR}/with_project timeout=120s
Should Be Equal As Numbers ${result.rc} 0