7ac3f1352c
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 20s
CI / lint (pull_request) Successful in 26s
CI / security (pull_request) Successful in 37s
CI / typecheck (pull_request) Successful in 57s
CI / unit_tests (pull_request) Successful in 3m14s
CI / integration_tests (pull_request) Successful in 3m33s
CI / docker (pull_request) Successful in 39s
CI / coverage (pull_request) Successful in 5m29s
CI / lint (push) Successful in 16s
CI / quality (push) Successful in 16s
CI / build (push) Successful in 16s
CI / security (push) Successful in 35s
CI / typecheck (push) Successful in 47s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m1s
CI / integration_tests (push) Successful in 3m31s
CI / docker (push) Successful in 52s
CI / coverage (push) Successful in 5m47s
CI / benchmark-publish (push) Successful in 19m3s
CI / benchmark-regression (pull_request) Successful in 35m19s
Implement ContainerToolExecutor for delegating tool invocations to devcontainer environments with full I/O forwarding. Add PathMapper for bidirectional host/container path translation. Wire container routing into ToolRunner with graceful fallback when no executor is configured. Add container_metadata field to ToolInvocation for tracking execution context. New modules: - tool/container_executor.py: ContainerToolExecutor, ContainerConfig, ContainerMetadata, ContainerExecutionError, ContainerTimeoutError - tool/path_mapper.py: PathMapper with host_to_container/container_to_host Modified: - tool/runner.py: container execution routing via ExecutionEnvironment - domain/models/core/change.py: container_metadata on ToolInvocation - tool/__init__.py: new public exports Review fixes applied: - Add Alembic migration m6_004 for container_metadata_json column - Enforce _MAX_OUTPUT_BYTES (50 MiB) truncation in _run_command() - Fix path traversal in sync_results_to_host (Path.is_relative_to) - Allow spaces in _looks_like_path() for valid filesystem paths - Preserve negative exit codes from signal kills in metadata - Add default=str to json.dumps(invocation.arguments) safety net - Log warnings when path mapping recursion depth exceeded - Warn when devcontainer binary not found on PATH - Use default allow_nan for host-path JSON validation in runner.py (only container path requires RFC 7159 strict mode) - Reject URL-like patterns in _looks_like_path() to avoid false positives on API routes, protocol-relative URIs, and query strings - Add extract_container_metadata() static helper on ContainerToolExecutor as bridge for ToolInvocation wiring - Use raw_stdout bytes in sync_results_to_host to prevent binary file corruption from text-mode decode/re-encode - Apply posixpath.normpath() in workspace_folder validator and reject path components containing '..' - Check result.timed_out in sync_results_to_host and raise ContainerTimeoutError instead of always raising ContainerExecutionError - Detect overlapping host_root/container_root in PathMapper and raise ValueError to prevent corrupt bidirectional mappings - Wrap host-side I/O in sync_results_to_host with try/except OSError to produce ContainerExecutionError on write failure - Enforce int(timeout) in _build_exec_command to prevent shell injection via malicious objects with __str__ methods - Change ToolResult validator from 'not self.error' to 'self.error is None' so empty-string errors are accepted - Iterate required list directly in ToolRunner schema validation to detect fields listed in required but absent from properties Closes #515
226 lines
7.1 KiB
Markdown
226 lines
7.1 KiB
Markdown
# Execution Environment Reference
|
|
|
|
This document describes how CleverAgents routes tool execution between
|
|
host and container environments, including the container-aware tool
|
|
executor, path mapping, and I/O forwarding.
|
|
|
|
## Overview
|
|
|
|
When a plan or tool specifies `execution_environment: container`,
|
|
the tool runner delegates execution to a **ContainerToolExecutor** that:
|
|
|
|
1. Maps host-side file paths to container-side equivalents.
|
|
2. Executes the tool via `devcontainer exec`.
|
|
3. Retrieves results and maps output paths back to the host.
|
|
4. Records container metadata on the `ToolInvocation` audit trail.
|
|
|
|
## Execution Environment Resolution
|
|
|
|
The execution environment is resolved using a priority chain:
|
|
|
|
| Priority | Source | Example |
|
|
|----------|--------|---------|
|
|
| 1 (highest) | Tool-level override | `tool_env="container"` |
|
|
| 2 | Plan-level override | `--execution-environment container` |
|
|
| 3 | Project-level default | `ContextConfig.execution_environment` |
|
|
| 4 (lowest) | System default | `HOST` |
|
|
|
|
If `CONTAINER` is resolved, the system validates that a container
|
|
resource (devcontainer-instance, container-instance, or
|
|
devcontainer-file) is linked to the project.
|
|
|
|
## Path Mapping
|
|
|
|
The `PathMapper` utility translates file paths between host and
|
|
container filesystems:
|
|
|
|
```python
|
|
from cleveragents.tool.path_mapper import PathMapper
|
|
|
|
mapper = PathMapper(
|
|
host_root="/tmp/sandbox/project-1",
|
|
container_root="/workspace",
|
|
)
|
|
|
|
# Host to container
|
|
mapper.host_to_container("/tmp/sandbox/project-1/src/main.py")
|
|
# => "/workspace/src/main.py"
|
|
|
|
# Container to host
|
|
mapper.container_to_host("/workspace/src/main.py")
|
|
# => "/tmp/sandbox/project-1/src/main.py"
|
|
|
|
# Paths outside root are returned unchanged
|
|
mapper.host_to_container("/usr/lib/libc.so")
|
|
# => "/usr/lib/libc.so"
|
|
```
|
|
|
|
### Path Mapping in Tool Inputs
|
|
|
|
File path arguments in tool inputs are automatically mapped before
|
|
container execution. The executor recursively traverses the input
|
|
dict, mapping any string values that fall under the host root:
|
|
|
|
```python
|
|
# Before mapping
|
|
{"path": "/tmp/sandbox/src/main.py", "count": 5}
|
|
|
|
# After mapping
|
|
{"path": "/workspace/src/main.py", "count": 5}
|
|
```
|
|
|
|
Nested dicts and lists are also traversed.
|
|
|
|
## Container Tool Executor
|
|
|
|
The `ContainerToolExecutor` wraps tool execution via `devcontainer exec`:
|
|
|
|
```python
|
|
from cleveragents.tool.container_executor import (
|
|
ContainerConfig,
|
|
ContainerToolExecutor,
|
|
)
|
|
|
|
config = ContainerConfig(
|
|
workspace_folder="/workspace",
|
|
container_id="my-container",
|
|
image="ghcr.io/org/devimage:latest",
|
|
host_sandbox_path="/tmp/sandbox/project-1",
|
|
timeout_seconds=120,
|
|
)
|
|
|
|
executor = ContainerToolExecutor(config)
|
|
result = executor.execute_tool("file_read", {"path": "/tmp/sandbox/project-1/src/main.py"})
|
|
```
|
|
|
|
### Configuration
|
|
|
|
| Parameter | Type | Default | Description |
|
|
|-----------|------|---------|-------------|
|
|
| `workspace_folder` | `str` | `/workspace` | Workspace root inside the container |
|
|
| `container_id` | `str` | `""` | Docker/Podman container ID |
|
|
| `image` | `str` | `""` | Container image reference |
|
|
| `timeout_seconds` | `int` | `120` | Max execution time (seconds) |
|
|
| `host_sandbox_path` | `str` | `""` | Host-side sandbox root |
|
|
|
|
### Timeout Handling
|
|
|
|
If a container tool execution exceeds `timeout_seconds`, the
|
|
executor returns a `ToolResult` with `success=False` and an error
|
|
message indicating the timeout. The `metadata.container` dict on
|
|
the result will have `timed_out=True`.
|
|
|
|
### Error Reporting
|
|
|
|
Container execution failures produce structured `ToolResult` objects:
|
|
|
|
- **Timeout**: `error="Container execution timed out after Ns"`
|
|
- **Non-zero exit**: `error="Container execution failed (exit code N): stderr"`
|
|
- **OS error**: `error="Container execution failed (exit code -1): error message"`
|
|
|
|
All errors include `container_metadata` with execution details.
|
|
|
|
## Result Retrieval
|
|
|
|
After container execution, file-based outputs can be synced back to
|
|
the host using `sync_results_to_host`:
|
|
|
|
```python
|
|
host_path = executor.sync_results_to_host("/workspace/build/output.txt")
|
|
# => "/tmp/sandbox/project-1/build/output.txt"
|
|
```
|
|
|
|
## Audit Trail
|
|
|
|
When a tool executes inside a container, the `ToolResult.metadata`
|
|
dict includes a `container` key with execution details, and the
|
|
`ToolInvocation` record stores this as the `container_metadata` field:
|
|
|
|
```json
|
|
{
|
|
"metadata": {
|
|
"container": {
|
|
"container_id": "abc123",
|
|
"image": "ghcr.io/org/devimage:latest",
|
|
"workspace_folder": "/workspace",
|
|
"exec_time_ms": 1500.0,
|
|
"exit_code": 0,
|
|
"timed_out": false
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
This provides full traceability for container-routed tool executions.
|
|
|
|
## Security Model
|
|
|
|
Container tool execution enforces several layers of protection:
|
|
|
|
### Environment Variable Filtering
|
|
|
|
Only a minimal allowlist of environment variables (`PATH`, `LANG`,
|
|
`TERM`) is forwarded to the subprocess that invokes `devcontainer
|
|
exec`. All other host environment variables—including secrets,
|
|
credentials, and `HOME`—are excluded.
|
|
|
|
### Symlink and Path Traversal Protection
|
|
|
|
- **Sandbox root validation**: `sync_results_to_host` resolves the
|
|
mapped host path and verifies it falls within the sandbox root
|
|
*before* writing any data. Path traversal via `..` is rejected.
|
|
- **Symlink guard**: The default sandbox fallback (`/tmp/sandbox`)
|
|
is rejected if it is a symlink. Explicit `host_sandbox_path` is
|
|
required in that case.
|
|
- **`O_NOFOLLOW` writes**: Files written by `sync_results_to_host`
|
|
use `O_NOFOLLOW` to prevent symlink-following at write time.
|
|
- **Root mapping rejection**: `PathMapper` rejects `/` and `//` as
|
|
host or container roots to prevent mapping every absolute path.
|
|
|
|
### Output Caps
|
|
|
|
Captured stdout and stderr are bounded to 50 MiB per stream.
|
|
Output exceeding this limit is truncated with a warning log entry.
|
|
This prevents a misbehaving container from exhausting host memory.
|
|
|
|
### Permission Model
|
|
|
|
- Files written by `sync_results_to_host` are created with mode
|
|
`0o600` (owner read/write only).
|
|
- Parent directories are created with mode `0o700` (owner only).
|
|
- The `devcontainer exec` binary is resolved at init time via
|
|
`shutil.which` to prevent PATH hijacking. If the binary is not
|
|
found, all execution calls fail fast with a descriptive error.
|
|
|
|
### Input Validation
|
|
|
|
- Null bytes in paths are rejected at all entry points.
|
|
- Tool inputs are validated as JSON-serialisable before container
|
|
execution (with `allow_nan=False` per RFC 7159).
|
|
- JSON inputs are piped via stdin rather than shell arguments to
|
|
avoid `ARG_MAX` limits and shell injection.
|
|
|
|
## ToolRunner Integration
|
|
|
|
The `ToolRunner` accepts an optional `container_executor` parameter:
|
|
|
|
```python
|
|
from cleveragents.tool.runner import ToolRunner
|
|
from cleveragents.tool.container_executor import (
|
|
ContainerConfig,
|
|
ContainerToolExecutor,
|
|
)
|
|
|
|
config = ContainerConfig(...)
|
|
executor = ContainerToolExecutor(config)
|
|
runner = ToolRunner(
|
|
registry=registry,
|
|
container_executor=executor,
|
|
)
|
|
```
|
|
|
|
When the execution environment resolves to `CONTAINER` and a
|
|
`ContainerToolExecutor` is configured, the runner delegates to it
|
|
automatically. If no executor is configured, the runner returns an
|
|
error result indicating that container execution is not available.
|