freemo
b122ec7ed5
CI / lint (pull_request) Successful in 23s
CI / quality (pull_request) Successful in 51s
CI / typecheck (pull_request) Successful in 1m0s
CI / security (pull_request) Successful in 55s
CI / build (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 32s
CI / push-validation (pull_request) Successful in 26s
CI / e2e_tests (pull_request) Successful in 3m38s
CI / integration_tests (pull_request) Successful in 6m42s
CI / unit_tests (pull_request) Successful in 8m19s
CI / docker (pull_request) Successful in 13s
CI / coverage (pull_request) Successful in 15m23s
CI / status-check (pull_request) Successful in 2s
Remove the local ${PYTHON} python (and python3) variable definitions from
the *** Variables *** sections of all affected robot files. These local
definitions were overriding the pabot-injected venv Python path passed via
--variable PYTHON:/path/to/venv/python, causing tests to use the system
Python (which lacks required packages like structlog, sqlalchemy, etc.)
instead of the nox venv Python.
The correct ${PYTHON} value is already set by Setup Test Environment in
common.resource via sys.executable, and pabot passes it via --variable.
The local fallback definitions are redundant and harmful in parallel runs.
Audit found 56 robot files with the pattern (more than the 9 originally
identified in the issue). All occurrences have been removed.
ISSUES CLOSED: #1309
133 lines
6.3 KiB
Plaintext
133 lines
6.3 KiB
Plaintext
*** Settings ***
|
|
Documentation CLI Consistency and UX Polish Integration Tests
|
|
...
|
|
... Verifies that shared exit-code constants, centralized error
|
|
... formatting, shell completion generation, and uniform
|
|
... --format json|text output switching work end-to-end.
|
|
Resource ${CURDIR}/common.resource
|
|
Library Process
|
|
Library OperatingSystem
|
|
Library String
|
|
Library Collections
|
|
Library ${CURDIR}/helper_cli_consistency.py
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Test Cases ***
|
|
Exit Code Constants Are Defined Correctly
|
|
[Documentation] Verify all standardized exit codes exist with correct values
|
|
${constants}= Get Exit Code Constants
|
|
Should Be Equal As Integers ${constants}[EXIT_SUCCESS] 0
|
|
Should Be Equal As Integers ${constants}[EXIT_ERROR] 1
|
|
Should Be Equal As Integers ${constants}[EXIT_USAGE] 2
|
|
Should Be Equal As Integers ${constants}[EXIT_NOT_FOUND] 3
|
|
Should Be Equal As Integers ${constants}[EXIT_CONFLICT] 4
|
|
|
|
Format Constants Are Defined Correctly
|
|
[Documentation] Verify format-related constants
|
|
${constants}= Get Format Constants
|
|
Should Contain ${constants}[FORMAT_HELP] json
|
|
Should Contain ${constants}[FORMAT_HELP] yaml
|
|
Should Contain ${constants}[FORMAT_HELP] plain
|
|
Should Be Equal As Strings ${constants}[DEFAULT_FORMAT] rich
|
|
${formats}= Set Variable ${constants}[VALID_FORMATS]
|
|
Should Contain ${formats} json
|
|
Should Contain ${formats} yaml
|
|
Should Contain ${formats} plain
|
|
Should Contain ${formats} table
|
|
Should Contain ${formats} rich
|
|
Should Be Equal As Strings ${constants}[FORMAT_TEXT] plain
|
|
Should Be Equal As Strings ${constants}[FORMAT_JSON] json
|
|
Should Be Equal As Strings ${constants}[FORMAT_TABLE] table
|
|
|
|
CLI Help Output Is Consistent
|
|
[Documentation] Verify --help output mentions key elements
|
|
${result}= Run Process ${PYTHON} -m cleveragents --help
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} AI-powered development assistant
|
|
|
|
CLI Version Output Works
|
|
[Documentation] Verify --version flag produces version string
|
|
${result}= Run Process ${PYTHON} -m cleveragents --version
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} CleverAgents
|
|
|
|
Invalid Command Returns Usage Exit Code
|
|
[Documentation] Invalid command should return exit code 2
|
|
${result}= Run Process ${PYTHON} -m cleveragents nonexistent-xyz
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 2
|
|
Should Contain ${result.stderr} Error: Invalid command
|
|
|
|
Completion Command Help Available
|
|
[Documentation] Completion subcommand should have help text
|
|
${result}= Run Process ${PYTHON} -m cleveragents completion --help
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} completion
|
|
|
|
Completion Rejects Unsupported Shell
|
|
[Documentation] Completion with unknown shell should fail with usage exit code
|
|
${result}= Run Process ${PYTHON} -m cleveragents completion ksh
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 2
|
|
|
|
Version JSON Format Output
|
|
[Documentation] --format json version should produce valid JSON (global flag)
|
|
${result}= Run Process ${PYTHON} -m cleveragents --format json version
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
${valid}= Is Valid Json ${result.stdout}
|
|
Should Be True ${valid} Output is not valid JSON: ${result.stdout}
|
|
|
|
Version YAML Format Output
|
|
[Documentation] --format yaml version should produce YAML with version key (global flag)
|
|
${result}= Run Process ${PYTHON} -m cleveragents --format yaml version
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} version:
|
|
|
|
Version Plain Format Output
|
|
[Documentation] --format plain version should produce plain key-value output (global flag)
|
|
${result}= Run Process ${PYTHON} -m cleveragents --format plain version
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} version:
|
|
|
|
CLI Error Format Is Standardized
|
|
[Documentation] cli_error should output Error: prefix and Hint: line
|
|
${result}= Verify Cli Error Format ${PYTHON}
|
|
Should Be Equal As Integers ${result}[rc] 1
|
|
Should Contain ${result}[stdout] Error:
|
|
Should Contain ${result}[stdout] Hint:
|
|
|
|
CLI Not Found Format Is Standardized
|
|
[Documentation] cli_not_found should output not found message and exit 3
|
|
${result}= Verify Cli Not Found Format ${PYTHON}
|
|
Should Be Equal As Integers ${result}[rc] 3
|
|
Should Contain ${result}[stdout] not found
|
|
Should Contain ${result}[stdout] Hint:
|
|
|
|
Info Command JSON Format
|
|
[Documentation] --format json info should produce valid JSON (global flag)
|
|
${result}= Run Process ${PYTHON} -m cleveragents --format json info
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
${valid}= Is Valid Json ${result.stdout}
|
|
Should Be True ${valid} Output is not valid JSON: ${result.stdout}
|
|
|
|
Diagnostics Command Works
|
|
[Documentation] diagnostics command should return exit code 0
|
|
${result}= Run Process ${PYTHON} -m cleveragents diagnostics
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Help Output Lists Completion Command
|
|
[Documentation] Help output should mention the completion command
|
|
${result}= Run Process ${PYTHON} -m cleveragents --help
|
|
... timeout=120s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} completion
|