forked from cleveragents/cleveragents-core
99 lines
No EOL
4.8 KiB
Text
99 lines
No EOL
4.8 KiB
Text
*** Settings ***
|
|
Documentation Robot Framework tests for CleverAgents Discovery Tools
|
|
Library OperatingSystem
|
|
Library Process
|
|
Library String
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
${PROJECT_DIR} /app
|
|
${DISCOVERY_DIR} ${PROJECT_DIR}/src/cleveragents/discovery
|
|
${DOCS_DIR} ${PROJECT_DIR}/docs/reference
|
|
|
|
*** Test Cases ***
|
|
Run Discovery Tool Suite
|
|
[Documentation] Test running the complete discovery tool suite
|
|
[Tags] discovery
|
|
${result}= Run Process ${PYTHON} ${DISCOVERY_DIR}/run_all.py cwd=${PROJECT_DIR}
|
|
Should Contain ${result.stdout} CleverAgents Discovery Tool Suite
|
|
Should Contain ${result.stdout} ✓ Extracted
|
|
Should Contain ${result.stdout} commands
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Extract CLI Inventory
|
|
[Documentation] Test CLI inventory extraction directly
|
|
[Tags] discovery
|
|
${result}= Run Process ${PYTHON} -m cleveragents.discovery.cli_inventory cwd=${PROJECT_DIR}
|
|
Should Contain ${result.stdout} Extracting CLI command metadata
|
|
Should Contain ${result.stdout} Extracted
|
|
Should Contain ${result.stdout} commands
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Verify Inventory Files Created
|
|
[Documentation] Verify that inventory files are created in the correct location
|
|
[Tags] discovery
|
|
File Should Exist ${DOCS_DIR}/cli_inventory.yaml
|
|
File Should Exist ${DOCS_DIR}/cli_inventory.json
|
|
|
|
${yaml_size}= Get File Size ${DOCS_DIR}/cli_inventory.yaml
|
|
${json_size}= Get File Size ${DOCS_DIR}/cli_inventory.json
|
|
|
|
Should Be True ${yaml_size} > 1000 YAML file too small
|
|
Should Be True ${json_size} > 1000 JSON file too small
|
|
|
|
Validate YAML Inventory Structure
|
|
[Documentation] Check YAML inventory contains expected sections
|
|
[Tags] discovery
|
|
${yaml_content}= Get File ${DOCS_DIR}/cli_inventory.yaml
|
|
Should Contain ${yaml_content} root:
|
|
Should Contain ${yaml_content} commands:
|
|
Should Contain ${yaml_content} hierarchy:
|
|
Should Contain ${yaml_content} metadata:
|
|
Should Contain ${yaml_content} statistics:
|
|
|
|
Validate JSON Inventory Structure
|
|
[Documentation] Check JSON inventory contains expected sections
|
|
[Tags] discovery
|
|
${json_content}= Get File ${DOCS_DIR}/cli_inventory.json
|
|
Should Contain ${json_content} "root"
|
|
Should Contain ${json_content} "commands"
|
|
Should Contain ${json_content} "hierarchy"
|
|
Should Contain ${json_content} "metadata"
|
|
Should Contain ${json_content} "statistics"
|
|
|
|
Check Command Count
|
|
[Documentation] Verify minimum number of commands extracted
|
|
[Tags] discovery
|
|
${json_content}= Get File ${DOCS_DIR}/cli_inventory.json
|
|
Should Match Regexp ${json_content} "total_commands":\\s*[6-9][0-9] At least 60 commands expected
|
|
|
|
Discovery Tool Performance
|
|
[Documentation] Benchmark discovery tool performance
|
|
[Tags] discovery
|
|
${start_time}= Get Time epoch
|
|
${result}= Run Process ${PYTHON} -m cleveragents.discovery.cli_inventory cwd=${PROJECT_DIR}
|
|
${end_time}= Get Time epoch
|
|
${elapsed}= Evaluate ${end_time} - ${start_time}
|
|
|
|
Log Discovery took ${elapsed} seconds
|
|
Should Be True ${elapsed} < 10 Discovery took too long (${elapsed}s)
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Test Import Discovery Module
|
|
[Documentation] Test that discovery module can be imported
|
|
[Tags] discovery
|
|
${result}= Run Process ${PYTHON} -c import cleveragents.discovery; print('OK') cwd=${PROJECT_DIR}
|
|
Should Contain ${result.stdout} OK
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
|
|
Test Discovery Module Has Expected Classes
|
|
[Documentation] Verify discovery module exports expected classes
|
|
[Tags] discovery
|
|
${result}= Run Process ${PYTHON} -c from cleveragents.discovery import CLIInventoryExtractor, DataContractExtractor, EnvironmentVariableExtractor, ImplicitBehaviorExtractor, ServerEndpointExtractor, ShellAssetExtractor; print(' '.join([CLIInventoryExtractor.__name__, DataContractExtractor.__name__, EnvironmentVariableExtractor.__name__, ImplicitBehaviorExtractor.__name__, ServerEndpointExtractor.__name__, ShellAssetExtractor.__name__])) cwd=${PROJECT_DIR}
|
|
Should Contain ${result.stdout} CLIInventoryExtractor
|
|
Should Contain ${result.stdout} DataContractExtractor
|
|
Should Contain ${result.stdout} EnvironmentVariableExtractor
|
|
Should Contain ${result.stdout} ImplicitBehaviorExtractor
|
|
Should Contain ${result.stdout} ServerEndpointExtractor
|
|
Should Contain ${result.stdout} ShellAssetExtractor
|
|
Should Be Equal As Integers ${result.rc} 0 |