151 lines
6.0 KiB
Plaintext
151 lines
6.0 KiB
Plaintext
*** Settings ***
|
|
Documentation Test environment variable extraction from Plandex
|
|
Library OperatingSystem
|
|
Library Collections
|
|
Library String
|
|
Resource discovery_common.resource
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
${ENV_JSON} ${OUTPUT_DIR}/env_variables.json
|
|
${ENV_YAML} ${OUTPUT_DIR}/env_variables.yaml
|
|
${ENV_MAPPING} ${OUTPUT_DIR}/env_mapping.txt
|
|
${ENV_GUIDE} ${OUTPUT_DIR}/env_migration_guide.md
|
|
|
|
*** Test Cases ***
|
|
Extract Environment Variables from Documentation
|
|
[Documentation] Extract environment variables from documentation files
|
|
[Tags] discovery environment
|
|
|
|
# Run extraction
|
|
${result}= Run Process ${PYTHON} -m cleveragents.discovery.env_variables
|
|
... ${PLANDEX_DIR} --output ${OUTPUT_DIR}
|
|
... cwd=${WORK_DIR}
|
|
|
|
Should Be Equal As Integers ${result.returncode} 0
|
|
Should Contain ${result.stdout} Environment variable extraction complete
|
|
|
|
# Verify output files exist
|
|
File Should Exist ${ENV_JSON}
|
|
File Should Exist ${ENV_YAML}
|
|
File Should Exist ${ENV_MAPPING}
|
|
File Should Exist ${ENV_GUIDE}
|
|
|
|
Validate Environment Variable JSON Structure
|
|
[Documentation] Validate the structure of extracted environment variables
|
|
[Tags] discovery environment
|
|
|
|
${json_content}= Get File ${ENV_JSON}
|
|
${data}= Evaluate json.loads('''${json_content}''') json
|
|
|
|
# Check required keys
|
|
Dictionary Should Contain Key ${data} variables
|
|
Dictionary Should Contain Key ${data} conflicts
|
|
Dictionary Should Contain Key ${data} mapping
|
|
Dictionary Should Contain Key ${data} statistics
|
|
|
|
# Check statistics
|
|
${stats}= Get From Dictionary ${data} statistics
|
|
Dictionary Should Contain Key ${stats} total_variables
|
|
Dictionary Should Contain Key ${stats} documented_variables
|
|
Dictionary Should Contain Key ${stats} code_only_variables
|
|
Dictionary Should Contain Key ${stats} migration_required
|
|
|
|
# Verify we found variables
|
|
${total}= Get From Dictionary ${stats} total_variables
|
|
Should Be True ${total} > 0 Should find environment variables
|
|
|
|
Check Variable Naming Conventions
|
|
[Documentation] Verify variable naming conventions and mappings
|
|
[Tags] discovery environment
|
|
|
|
${json_content}= Get File ${ENV_JSON}
|
|
${data}= Evaluate json.loads('''${json_content}''') json
|
|
${mapping}= Get From Dictionary ${data} mapping
|
|
|
|
# Check PLANDEX to CLEVERAGENTS mapping
|
|
FOR ${old_name} IN @{mapping.keys()}
|
|
${new_name}= Get From Dictionary ${mapping} ${old_name}
|
|
Run Keyword If '${old_name}'.startswith('PLANDEX_')
|
|
... Should Start With ${new_name} CLEVERAGENTS_
|
|
END
|
|
|
|
Validate Migration Guide Content
|
|
[Documentation] Verify migration guide has expected content
|
|
[Tags] discovery environment
|
|
|
|
${content}= Get File ${ENV_GUIDE}
|
|
|
|
# Check headers
|
|
Should Contain ${content} \# Environment Variable Migration Guide
|
|
Should Contain ${content} \#\# Overview
|
|
Should Contain ${content} \#\# Variable Mappings
|
|
Should Contain ${content} \#\# Conflicts and Warnings
|
|
|
|
# Check table format
|
|
Should Contain ${content} | Old Name | New Name |
|
|
Should Contain ${content} |----------|----------|
|
|
|
|
# Check statistics
|
|
Should Contain ${content} Total variables:
|
|
Should Contain ${content} Variables requiring migration:
|
|
|
|
Environment Variable Categories
|
|
[Documentation] Check that variables are properly categorized
|
|
[Tags] discovery environment
|
|
|
|
${json_content}= Get File ${ENV_JSON}
|
|
${data}= Evaluate json.loads('''${json_content}''') json
|
|
${variables}= Get From Dictionary ${data} variables
|
|
|
|
${categories}= Create List
|
|
FOR ${var_name} ${var_info} IN &{variables}
|
|
${category}= Get From Dictionary ${var_info} category
|
|
Append To List ${categories} ${category}
|
|
END
|
|
|
|
# Remove duplicates
|
|
${unique_categories}= Remove Duplicates ${categories}
|
|
|
|
# Should have multiple categories
|
|
${length}= Get Length ${unique_categories}
|
|
Should Be True ${length} > 1 Should have multiple categories
|
|
|
|
# Check for expected categories
|
|
${expected}= Create List CLI Server LLM Providers General
|
|
FOR ${cat} IN @{expected}
|
|
${found}= Run Keyword And Return Status List Should Contain Value ${unique_categories} ${cat}
|
|
Log Category ${cat}: ${found}
|
|
END
|
|
|
|
Provider Variables Preservation
|
|
[Documentation] Verify provider-specific variables are preserved
|
|
[Tags] discovery environment
|
|
|
|
${json_content}= Get File ${ENV_JSON}
|
|
${data}= Evaluate json.loads('''${json_content}''') json
|
|
${variables}= Get From Dictionary ${data} variables
|
|
|
|
# Check provider variables
|
|
${providers}= Create List OPENAI_ ANTHROPIC_ GEMINI_ AZURE_ AWS_
|
|
|
|
FOR ${prefix} IN @{providers}
|
|
${provider_vars}= Create List
|
|
FOR ${var_name} ${var_info} IN &{variables}
|
|
Run Keyword If '${var_name}'.startswith('${prefix}')
|
|
... Append To List ${provider_vars} ${var_name}
|
|
END
|
|
|
|
# Check that provider variables keep their names
|
|
FOR ${var} IN @{provider_vars}
|
|
${var_info}= Get From Dictionary ${variables} ${var}
|
|
${proposed}= Get From Dictionary ${var_info} proposed_cleveragents_name
|
|
Should Be Equal ${var} ${proposed} Provider variable should keep its name
|
|
END
|
|
END
|
|
|
|
*** Keywords ***
|
|
Run Process
|
|
[Arguments] @{args} &{config}
|
|
${result}= Evaluate subprocess.run(${args}, capture_output=True, text=True, **${config}) subprocess
|
|
RETURN ${result} |