brent.edwards
8d108cb5d1
CI / lint (push) Successful in 21s
CI / build (push) Successful in 27s
CI / quality (push) Successful in 30s
CI / typecheck (push) Successful in 40s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 54s
CI / integration_tests (push) Successful in 3m57s
CI / e2e_tests (push) Successful in 4m51s
CI / unit_tests (push) Successful in 5m23s
CI / docker (push) Successful in 1m2s
CI / coverage (push) Successful in 6m57s
CI / benchmark-publish (push) Successful in 19m29s
## Summary Add tool-level execution environment preferences with four modes: `required`, `preferred`, `specific`, and `none`. ### Changes **New model** (`domain/models/core/execution_environment_preference.py`): - `EnvironmentPreferenceMode` enum: REQUIRED, PREFERRED, SPECIFIC, NONE - `ExecutionEnvironmentPreference` frozen Pydantic model with `mode` and `target_resource` fields - Model validator ensures `target_resource` required iff mode is SPECIFIC **ToolSpec & Tool model updates:** - Added `execution_environment: ExecutionEnvironmentPreference` field to both `ToolSpec` (runtime) and `Tool` (domain) - `Tool.from_config()` parses `execution_environment` from YAML config dicts **ToolRunner integration** (`tool/runner.py`): - Before calling `_env_resolver.resolve()`, checks `spec.execution_environment.mode`: - REQUIRED: raises `ContainerUnavailableError` if resolved env is not CONTAINER - PREFERRED: tries container, gracefully falls back to host - SPECIFIC: overrides `tool_env` with `target_resource` - NONE: current behavior (caller-supplied tool_env) ### Tests - **27 Behave scenarios** covering model validation, serialization, preference routing, YAML parsing, error cases - **12 Robot integration tests** covering CLI tool preference display and end-to-end routing ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,833 scenarios) | | `nox -s integration_tests` | PASS (12 new tests) | Closes #879 Reviewed-on: #970 Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
84 lines
4.7 KiB
Plaintext
84 lines
4.7 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for tool-level execution environment preferences
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_tool_env_preferences.py
|
|
|
|
*** Test Cases ***
|
|
Default Preference Is NONE
|
|
[Documentation] Default ExecutionEnvironmentPreference has mode=none
|
|
${result}= Run Process ${PYTHON} ${HELPER} default-preference cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} default-preference-ok
|
|
|
|
Required Mode Validates
|
|
[Documentation] Required mode creates correctly with no target_resource
|
|
${result}= Run Process ${PYTHON} ${HELPER} required-mode cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} required-mode-ok
|
|
|
|
Preferred Mode Validates
|
|
[Documentation] Preferred mode creates correctly with no target_resource
|
|
${result}= Run Process ${PYTHON} ${HELPER} preferred-mode cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} preferred-mode-ok
|
|
|
|
Specific Mode With Target
|
|
[Documentation] Specific mode requires and accepts target_resource
|
|
${result}= Run Process ${PYTHON} ${HELPER} specific-mode cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} specific-mode-ok
|
|
|
|
Specific Mode Validates Target Required
|
|
[Documentation] Specific mode validates target_resource is present
|
|
${result}= Run Process ${PYTHON} ${HELPER} specific-validates-target cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} specific-validates-target-ok
|
|
|
|
ToolSpec Has Execution Environment Field
|
|
[Documentation] ToolSpec model includes execution_environment field
|
|
${result}= Run Process ${PYTHON} ${HELPER} toolspec-field cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} toolspec-field-ok
|
|
|
|
Domain Tool Has Execution Environment Field
|
|
[Documentation] Domain Tool model includes execution_environment field
|
|
${result}= Run Process ${PYTHON} ${HELPER} domain-tool-field cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} domain-tool-field-ok
|
|
|
|
Tool From Config Parses Execution Environment
|
|
[Documentation] Tool.from_config parses execution_environment from YAML dict
|
|
${result}= Run Process ${PYTHON} ${HELPER} tool-from-config cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} tool-from-config-ok
|
|
|
|
Runner NONE Preference Uses Caller Env
|
|
[Documentation] ToolRunner with NONE preference uses caller-supplied env
|
|
${result}= Run Process ${PYTHON} ${HELPER} runner-none-preference cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} runner-none-preference-ok
|
|
|
|
Runner Required Fails Without Container
|
|
[Documentation] ToolRunner with REQUIRED preference fails without container
|
|
${result}= Run Process ${PYTHON} ${HELPER} runner-required-no-container cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} runner-required-no-container-ok
|
|
|
|
Runner Preferred Falls Back To Host
|
|
[Documentation] ToolRunner with PREFERRED preference falls back to host
|
|
${result}= Run Process ${PYTHON} ${HELPER} runner-preferred-fallback cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} runner-preferred-fallback-ok
|
|
|
|
Runner Specific Targets Container
|
|
[Documentation] ToolRunner with SPECIFIC preference targets named container
|
|
${result}= Run Process ${PYTHON} ${HELPER} runner-specific-container cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} runner-specific-container-ok
|