brent.edwards
8d108cb5d1
CI / lint (push) Successful in 21s
CI / build (push) Successful in 27s
CI / quality (push) Successful in 30s
CI / typecheck (push) Successful in 40s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 54s
CI / integration_tests (push) Successful in 3m57s
CI / e2e_tests (push) Successful in 4m51s
CI / unit_tests (push) Successful in 5m23s
CI / docker (push) Successful in 1m2s
CI / coverage (push) Successful in 6m57s
CI / benchmark-publish (push) Successful in 19m29s
## Summary Add tool-level execution environment preferences with four modes: `required`, `preferred`, `specific`, and `none`. ### Changes **New model** (`domain/models/core/execution_environment_preference.py`): - `EnvironmentPreferenceMode` enum: REQUIRED, PREFERRED, SPECIFIC, NONE - `ExecutionEnvironmentPreference` frozen Pydantic model with `mode` and `target_resource` fields - Model validator ensures `target_resource` required iff mode is SPECIFIC **ToolSpec & Tool model updates:** - Added `execution_environment: ExecutionEnvironmentPreference` field to both `ToolSpec` (runtime) and `Tool` (domain) - `Tool.from_config()` parses `execution_environment` from YAML config dicts **ToolRunner integration** (`tool/runner.py`): - Before calling `_env_resolver.resolve()`, checks `spec.execution_environment.mode`: - REQUIRED: raises `ContainerUnavailableError` if resolved env is not CONTAINER - PREFERRED: tries container, gracefully falls back to host - SPECIFIC: overrides `tool_env` with `target_resource` - NONE: current behavior (caller-supplied tool_env) ### Tests - **27 Behave scenarios** covering model validation, serialization, preference routing, YAML parsing, error cases - **12 Robot integration tests** covering CLI tool preference display and end-to-end routing ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,833 scenarios) | | `nox -s integration_tests` | PASS (12 new tests) | Closes #879 Reviewed-on: #970 Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
152 lines
8.1 KiB
Gherkin
152 lines
8.1 KiB
Gherkin
Feature: Tool-level execution environment preferences
|
|
As the tool execution engine
|
|
I need tools to declare intrinsic execution environment preferences
|
|
So that execution routing respects tool requirements
|
|
|
|
# ── EnvironmentPreferenceMode enum ─────────────────────────────────
|
|
|
|
Scenario: EnvironmentPreferenceMode has NONE value
|
|
Given I import EnvironmentPreferenceMode
|
|
Then the NONE preference value should be "none"
|
|
|
|
Scenario: EnvironmentPreferenceMode has REQUIRED value
|
|
Given I import EnvironmentPreferenceMode
|
|
Then the REQUIRED preference value should be "required"
|
|
|
|
Scenario: EnvironmentPreferenceMode has PREFERRED value
|
|
Given I import EnvironmentPreferenceMode
|
|
Then the PREFERRED preference value should be "preferred"
|
|
|
|
Scenario: EnvironmentPreferenceMode has SPECIFIC value
|
|
Given I import EnvironmentPreferenceMode
|
|
Then the SPECIFIC preference value should be "specific"
|
|
|
|
Scenario: EnvironmentPreferenceMode is a StrEnum
|
|
Given I import EnvironmentPreferenceMode
|
|
Then EnvironmentPreferenceMode should be a StrEnum subclass
|
|
|
|
# ── Default preference ─────────────────────────────────────────────
|
|
|
|
Scenario: Default preference is NONE
|
|
Given I create a default ExecutionEnvironmentPreference
|
|
Then the preference mode should be "none"
|
|
And the preference target_resource should be None
|
|
|
|
# ── Required mode ──────────────────────────────────────────────────
|
|
|
|
Scenario: Required mode with no target_resource
|
|
Given I create an ExecutionEnvironmentPreference with mode "required"
|
|
Then the preference mode should be "required"
|
|
And the preference target_resource should be None
|
|
|
|
Scenario: Required mode rejects target_resource
|
|
When I try to create an ExecutionEnvironmentPreference with mode "required" and target "local/api-dev"
|
|
Then the preference creation should fail with "target_resource only valid"
|
|
|
|
# ── Preferred mode ─────────────────────────────────────────────────
|
|
|
|
Scenario: Preferred mode with no target_resource
|
|
Given I create an ExecutionEnvironmentPreference with mode "preferred"
|
|
Then the preference mode should be "preferred"
|
|
And the preference target_resource should be None
|
|
|
|
Scenario: Preferred mode rejects target_resource
|
|
When I try to create an ExecutionEnvironmentPreference with mode "preferred" and target "local/api-dev"
|
|
Then the preference creation should fail with "target_resource only valid"
|
|
|
|
# ── Specific mode ──────────────────────────────────────────────────
|
|
|
|
Scenario: Specific mode requires target_resource
|
|
When I try to create an ExecutionEnvironmentPreference with mode "specific" and no target
|
|
Then the preference creation should fail with "'specific' mode requires target_resource"
|
|
|
|
Scenario: Specific mode with target_resource
|
|
Given I create a specific ExecutionEnvironmentPreference targeting "local/api-dev"
|
|
Then the preference mode should be "specific"
|
|
And the preference target_resource should be "local/api-dev"
|
|
|
|
# ── None mode ──────────────────────────────────────────────────────
|
|
|
|
Scenario: None mode rejects target_resource
|
|
When I try to create an ExecutionEnvironmentPreference with mode "none" and target "local/api-dev"
|
|
Then the preference creation should fail with "target_resource only valid"
|
|
|
|
# ── Frozen model ───────────────────────────────────────────────────
|
|
|
|
Scenario: ExecutionEnvironmentPreference is frozen
|
|
Given I create a default ExecutionEnvironmentPreference
|
|
When I try to mutate the preference mode
|
|
Then the mutation should fail
|
|
|
|
# ── ToolSpec field ─────────────────────────────────────────────────
|
|
|
|
Scenario: ToolSpec has default execution_environment
|
|
Given I create a ToolSpec with default execution_environment
|
|
Then the ToolSpec execution_environment mode should be "none"
|
|
|
|
Scenario: ToolSpec accepts execution_environment preference
|
|
Given I create a ToolSpec with required execution_environment
|
|
Then the ToolSpec execution_environment mode should be "required"
|
|
|
|
# ── Domain Tool field ──────────────────────────────────────────────
|
|
|
|
Scenario: Domain Tool has default execution_environment
|
|
Given I create a domain Tool with default execution_environment
|
|
Then the domain Tool execution_environment mode should be "none"
|
|
|
|
Scenario: Domain Tool accepts execution_environment preference
|
|
Given I create a domain Tool with required execution_environment
|
|
Then the domain Tool execution_environment mode should be "required"
|
|
|
|
# ── Tool YAML config parsing ───────────────────────────────────────
|
|
|
|
Scenario: Tool.from_config parses execution_environment with required mode
|
|
Given I have a tool config dict with execution_environment mode "required"
|
|
When I create a Tool from the config
|
|
Then the tool execution_environment mode should be "required"
|
|
|
|
Scenario: Tool.from_config parses execution_environment with specific mode
|
|
Given I have a tool config dict with execution_environment targeting "local/api-dev"
|
|
When I create a Tool from the config
|
|
Then the tool execution_environment mode should be "specific"
|
|
And the tool execution_environment target_resource should be "local/api-dev"
|
|
|
|
Scenario: Tool.from_config defaults execution_environment to none
|
|
Given I have a tool config dict without execution_environment
|
|
When I create a Tool from the config
|
|
Then the tool execution_environment mode should be "none"
|
|
|
|
# ── as_cli_dict ────────────────────────────────────────────────────
|
|
|
|
Scenario: as_cli_dict omits execution_environment when none
|
|
Given I create a domain Tool with default execution_environment
|
|
When I call as_cli_dict on the tool
|
|
Then the cli dict should not contain "execution_environment"
|
|
|
|
Scenario: as_cli_dict includes execution_environment when non-none
|
|
Given I create a domain Tool with required execution_environment
|
|
When I call as_cli_dict on the tool
|
|
Then the cli dict should contain execution_environment with mode "required"
|
|
|
|
# ── ToolRunner preference routing ──────────────────────────────────
|
|
|
|
Scenario: ToolRunner with NONE preference uses caller-supplied env
|
|
Given I have a ToolRunner with a tool with NONE preference
|
|
When I execute the tool with no caller env
|
|
Then the tool preference routing should succeed on host
|
|
|
|
Scenario: ToolRunner with REQUIRED preference fails without container
|
|
Given I have a ToolRunner with a tool with REQUIRED preference and no container
|
|
When I execute the required-preference tool
|
|
Then the execution should fail with container required error
|
|
|
|
Scenario: ToolRunner with PREFERRED preference falls back to host
|
|
Given I have a ToolRunner with a tool with PREFERRED preference and no container
|
|
When I execute the preferred-preference tool
|
|
Then the tool preference routing should succeed on host
|
|
|
|
Scenario: ToolRunner with SPECIFIC preference overrides tool_env
|
|
Given I have a ToolRunner with a tool with SPECIFIC preference targeting "local/api-dev"
|
|
When I execute the specific-preference tool with container available
|
|
Then the tool preference routing should succeed in container
|