aditya
2764fcef5c
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 34s
CI / typecheck (pull_request) Successful in 39s
CI / security (pull_request) Successful in 52s
CI / unit_tests (pull_request) Successful in 2m51s
CI / integration_tests (pull_request) Successful in 3m41s
CI / docker (pull_request) Successful in 56s
CI / e2e_tests (pull_request) Successful in 3m56s
CI / coverage (pull_request) Successful in 6m28s
CI / benchmark-regression (pull_request) Successful in 38m39s
Address review-driven fixes across actor schema, preflight guardrails, docs/examples, and Behave/Robot coverage: unify preflight warning behavior with shared role-warning logic, resolve actor-name to config payloads in production preflight flow, harden response_format validation/coercion edge cases, extract duplicated helper logic, and expand negative-path test coverage. Also fix cross-scenario patcher leakage in step modules to eliminate full-run-only coverage failures.
303 lines
12 KiB
Plaintext
303 lines
12 KiB
Plaintext
*** Settings ***
|
|
Documentation CLI Plan and Context Commands - Robot Framework Tests
|
|
... Comprehensive testing of plan and context workflows
|
|
Resource ${CURDIR}/common.resource
|
|
Library OperatingSystem
|
|
Library Process
|
|
Library String
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
Test Timeout 300 seconds
|
|
|
|
*** Variables ***
|
|
# ${PYTHON} will be set in Setup Test Environment from common.resource
|
|
# Make sure to call Setup Test Environment before using ${PYTHON}
|
|
${TEST_DIR} ${TEMPDIR}${/}cleveragents_plan_ctx_test
|
|
${PROJECT_NAME} test-project
|
|
|
|
*** Test Cases ***
|
|
Initialize New Project
|
|
[Documentation] Test cleveragents init command
|
|
[Setup] Setup Test Directory
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME}
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true stderr=STDOUT timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0 Init failed: ${result.stdout}
|
|
Should Exist ${TEST_DIR}${/}.cleveragents
|
|
Should Exist ${TEST_DIR}${/}.cleveragents${/}db.sqlite
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Add Files To Context
|
|
[Documentation] Test context-load command
|
|
[Setup] Initialize Test Project
|
|
|
|
# Create test files
|
|
Create File ${TEST_DIR}${/}test.py print('hello')
|
|
Create File ${TEST_DIR}${/}main.py def main(): pass
|
|
|
|
# Add files to context
|
|
${result}= Run Process ${PYTHON} -m cleveragents context-load test.py main.py
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
# Verify files are in context
|
|
${result}= Run Process ${PYTHON} -m cleveragents context list
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} test.py
|
|
Should Contain ${result.stdout} main.py
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Create Plan With Tell
|
|
[Documentation] Test tell command
|
|
[Setup] Initialize Test Project
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents tell Add error handling
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
# Verify plan was created with correct name derived from prompt
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan current
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} add_error_handling
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Build Plan
|
|
[Documentation] Test build command
|
|
[Setup] Initialize Test Project With Plan
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents build
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Apply Plan Changes
|
|
[Documentation] Test apply command
|
|
[Setup] Initialize Test Project With Built Plan
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents apply --yes
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Create New Empty Plan
|
|
[Documentation] Test plan new command
|
|
[Setup] Initialize Test Project
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan new feature-plan
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
# Verify new plan is current
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan current
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} feature-plan
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Show Current Plan
|
|
[Documentation] Test plan current command
|
|
[Setup] Initialize Test Project
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan current
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} main
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
List All Plans
|
|
[Documentation] Test plan list command
|
|
[Setup] Initialize Test Project With Multiple Plans
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan list
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} main
|
|
Should Contain ${result.stdout} feature-1
|
|
Should Contain ${result.stdout} feature-2
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Switch Between Plans
|
|
[Documentation] Test plan cd command
|
|
[Setup] Initialize Test Project With Multiple Plans
|
|
|
|
# Switch to feature-1
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan cd feature-1
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
# Verify switch
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan current
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Contain ${result.stdout} feature-1
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Continue Working On Plan
|
|
[Documentation] Test plan continue command
|
|
[Setup] Initialize Test Project With Plan
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan continue Also add logging
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
List Context Files
|
|
[Documentation] Test context list command
|
|
[Setup] Initialize Test Project With Context
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents context list
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} test.py
|
|
Should Contain ${result.stdout} utils.py
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Show Context Content
|
|
[Documentation] Test context show command
|
|
[Setup] Initialize Test Project With Context
|
|
|
|
# Show specific file content
|
|
${result}= Run Process ${PYTHON} -m cleveragents context show test.py
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} def hello
|
|
|
|
# Also test showing summary (without file argument)
|
|
${result}= Run Process ${PYTHON} -m cleveragents context show
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} Context Summary
|
|
Should Contain ${result.stdout} Total files: 2
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Remove File From Context
|
|
[Documentation] Test context rm command
|
|
[Setup] Initialize Test Project With Context
|
|
|
|
# Remove test.py
|
|
${result}= Run Process ${PYTHON} -m cleveragents context rm test.py
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
# Verify removal
|
|
${result}= Run Process ${PYTHON} -m cleveragents context list
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
|
|
Should Not Contain ${result.stdout} test.py
|
|
Should Contain ${result.stdout} utils.py
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Clear All Context
|
|
[Documentation] Test context clear command
|
|
[Setup] Initialize Test Project With Context
|
|
|
|
${result}= Run Process ${PYTHON} -m cleveragents context clear --yes
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true stderr=STDOUT timeout=120s on_timeout=kill
|
|
|
|
Should Be Equal As Integers ${result.rc} 0 Context clear failed: ${result.stdout}
|
|
|
|
# Verify context is empty
|
|
${result}= Run Process ${PYTHON} -m cleveragents context list
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true stderr=STDOUT timeout=120s on_timeout=kill
|
|
|
|
Should Not Contain ${result.stdout} test.py
|
|
Should Not Contain ${result.stdout} utils.py
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
Command Error Handling
|
|
[Documentation] Test error cases for commands
|
|
[Setup] Setup Test Directory
|
|
|
|
# Try to use command without project
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan tell test
|
|
... cwd=${TEST_DIR} stderr=STDOUT timeout=120s on_timeout=kill
|
|
|
|
Should Not Be Equal As Integers ${result.rc} 0
|
|
|
|
[Teardown] Cleanup Test Directory
|
|
|
|
*** Keywords ***
|
|
Setup Test Directory
|
|
[Documentation] Create clean test directory
|
|
${run_id}= Evaluate __import__('uuid').uuid4().hex
|
|
Set Test Variable ${TEST_DIR} ${TEMPDIR}${/}cleveragents_plan_ctx_test_${run_id}
|
|
Set Test Variable ${PROJECT_NAME} test-project-${run_id}
|
|
Set Environment Variable CLEVERAGENTS_TESTING_USE_MOCK_AI true
|
|
Run Keyword And Ignore Error Remove Directory ${TEST_DIR} recursive=True
|
|
Create Directory ${TEST_DIR}
|
|
|
|
Cleanup Test Directory
|
|
[Documentation] Remove test directory and cleanup processes
|
|
# Clean up test directory
|
|
Run Keyword And Ignore Error Remove Directory ${TEST_DIR} recursive=True
|
|
|
|
Initialize Test Project
|
|
[Documentation] Initialize a test project
|
|
Setup Test Directory
|
|
${result}= Run Process ${PYTHON} -m cleveragents init ${PROJECT_NAME}
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Initialize Test Project With Plan
|
|
[Documentation] Initialize project and create a plan
|
|
Initialize Test Project
|
|
${result}= Run Process ${PYTHON} -m cleveragents tell Test instruction
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Initialize Test Project With Built Plan
|
|
[Documentation] Initialize project with a built plan
|
|
Initialize Test Project With Plan
|
|
${result}= Run Process ${PYTHON} -m cleveragents build
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_TESTING_USE_MOCK_AI=true timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Initialize Test Project With Multiple Plans
|
|
[Documentation] Initialize project with multiple plans
|
|
Initialize Test Project
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan new feature-1
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
${result}= Run Process ${PYTHON} -m cleveragents plan new feature-2
|
|
... cwd=${TEST_DIR} timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Initialize Test Project With Context
|
|
[Documentation] Initialize project and add files to context
|
|
Initialize Test Project
|
|
Create File ${TEST_DIR}${/}test.py def hello(): pass
|
|
Create File ${TEST_DIR}${/}utils.py import os
|
|
${result}= Run Process ${PYTHON} -m cleveragents context-load test.py utils.py
|
|
... cwd=${TEST_DIR} env:CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true stderr=STDOUT timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 context-load failed: ${result.stdout}
|