feat(validation): implement tool wrapping runtime (wraps + transform delegation) #543

Closed
opened 2026-03-04 01:00:41 +00:00 by freemo · 1 comment
Owner

Metadata

Field Value
Type Feature
Priority High
MoSCoW Should Have
Points 8
Milestone v3.4.0
Assignee freemo
Parent Epic #395 (Epic: Validation & Quality Gating)
Depends On None (CLI parsing already exists)

Background

The specification (§ Core Concepts > Validation > Tool Wrapping, lines 22474-22690) describes a mechanism where Validations can wrap existing Tools via the wraps field. A wrapping validation:

  1. Intercepts calls to the wrapped tool
  2. Delegates execution to the original tool
  3. Passes the output through a transform function
  4. Applies argument_mapping to adapt arguments between the wrapper and the wrapped tool

The YAML schema includes wraps, transform, and argument_mapping fields, and the CLI already parses these fields in validation commands. However, no runtime execution engine exists that actually performs the delegation — the wrapping is parsed but never executed.

Acceptance Criteria

  1. When a validation with wraps: <tool_name> is invoked, the runtime delegates execution to the wrapped tool.
  2. argument_mapping is applied to translate arguments from the wrapper's schema to the wrapped tool's schema.
  3. The wrapped tool's output is passed through the transform function before being returned.
  4. If the wrapped tool is not found, a clear error is raised (not silent failure).
  5. Wrapping is composable: a wrapper can wrap another wrapper (chain of delegation).
  6. The execution context (sandbox, resources) is inherited from the wrapping validation's context.

Subtasks

1. Design

  • Design the delegation chain resolution algorithm
  • Design argument mapping execution model
  • Design transform function execution sandbox

2. Implementation

  • Implement WrappedToolExecutor that resolves wraps references
  • Implement ArgumentMapper that applies argument_mapping config
  • Implement TransformExecutor that runs the transform function on tool output
  • Wire into the validation execution pipeline

3. Testing

  • Unit test: simple wrapping (single level)
  • Unit test: argument mapping translation
  • Unit test: transform function application
  • Unit test: chained wrapping (wrapper wraps wrapper)
  • Unit test: missing wrapped tool error handling
  • Integration test: end-to-end validation wrapping a real tool

4. Documentation

  • Docstrings for all new classes
  • Example YAML showing wrapping configuration

5. Integration

  • Wire into existing tool execution pipeline
  • Verify CLI wraps parsing feeds into runtime

6. Observability

  • Log delegation chain for debugging
  • Log transform input/output for diagnostics

7. Security

  • Transform functions execute in sandboxed context
  • Validate that wrapping cannot bypass tool access controls

Definition of Done

  • All acceptance criteria met
  • All subtask checkboxes checked
  • Tests pass in CI
  • Code reviewed and approved
## Metadata | Field | Value | |-------|-------| | **Type** | Feature | | **Priority** | High | | **MoSCoW** | Should Have | | **Points** | 8 | | **Milestone** | v3.4.0 | | **Assignee** | freemo | | **Parent Epic** | #395 (Epic: Validation & Quality Gating) | | **Depends On** | None (CLI parsing already exists) | ## Background The specification (§ Core Concepts > Validation > Tool Wrapping, lines 22474-22690) describes a mechanism where Validations can **wrap existing Tools** via the `wraps` field. A wrapping validation: 1. Intercepts calls to the wrapped tool 2. Delegates execution to the original tool 3. Passes the output through a `transform` function 4. Applies `argument_mapping` to adapt arguments between the wrapper and the wrapped tool The YAML schema includes `wraps`, `transform`, and `argument_mapping` fields, and the CLI already parses these fields in validation commands. However, **no runtime execution engine** exists that actually performs the delegation — the wrapping is parsed but never executed. ## Acceptance Criteria 1. When a validation with `wraps: <tool_name>` is invoked, the runtime delegates execution to the wrapped tool. 2. `argument_mapping` is applied to translate arguments from the wrapper's schema to the wrapped tool's schema. 3. The wrapped tool's output is passed through the `transform` function before being returned. 4. If the wrapped tool is not found, a clear error is raised (not silent failure). 5. Wrapping is composable: a wrapper can wrap another wrapper (chain of delegation). 6. The execution context (sandbox, resources) is inherited from the wrapping validation's context. ## Subtasks ### 1. Design - [x] Design the delegation chain resolution algorithm - [x] Design argument mapping execution model - [x] Design transform function execution sandbox ### 2. Implementation - [x] Implement `WrappedToolExecutor` that resolves `wraps` references - [x] Implement `ArgumentMapper` that applies `argument_mapping` config - [x] Implement `TransformExecutor` that runs the `transform` function on tool output - [x] Wire into the validation execution pipeline ### 3. Testing - [x] Unit test: simple wrapping (single level) - [x] Unit test: argument mapping translation - [x] Unit test: transform function application - [x] Unit test: chained wrapping (wrapper wraps wrapper) - [x] Unit test: missing wrapped tool error handling - [x] Integration test: end-to-end validation wrapping a real tool ### 4. Documentation - [x] Docstrings for all new classes - [x] Example YAML showing wrapping configuration ### 5. Integration - [x] Wire into existing tool execution pipeline - [x] Verify CLI `wraps` parsing feeds into runtime ### 6. Observability - [x] Log delegation chain for debugging - [x] Log transform input/output for diagnostics ### 7. Security - [x] Transform functions execute in sandboxed context - [x] Validate that wrapping cannot bypass tool access controls ## Definition of Done - [x] All acceptance criteria met - [x] All subtask checkboxes checked - [ ] Tests pass in CI - [ ] Code reviewed and approved
freemo added this to the v3.2.0 milestone 2026-03-04 01:01:15 +00:00
freemo modified the milestone from v3.2.0 to v3.4.0 2026-03-04 01:09:35 +00:00
freemo self-assigned this 2026-03-04 01:41:12 +00:00
Author
Owner

Implementation Complete — PR #555

Implementation of tool wrapping runtime has been submitted in PR #555.

What was implemented

src/cleveragents/tool/wrapping.py (~470 lines) — the core runtime engine with three main components:

  1. ArgumentMapper — Translates arguments from wrapper schema to wrapped tool schema using configurable argument_mapping. Supports identity mapping (passthrough) and explicit key remapping.

  2. TransformExecutor — Executes transform functions on wrapped tool output in a sandboxed RestrictedPython context. The sandbox provides result (tool output), json/re/math/datetime modules, and blocks all I/O, imports, and attribute access outside the allowlist.

  3. WrappedToolExecutor — Orchestrates the full delegation pipeline: resolves wraps references via the tool registry, detects cycles, enforces max chain depth (10), maps arguments, delegates execution, and applies transforms. Supports composable chaining (wrapper wrapping wrapper).

Error types: WrappedToolNotFoundError, WrappingCycleError, WrappingDepthExceededError, TransformExecutionError — all with clear diagnostic messages.

Testing

  • 20 Behave BDD scenarios (features/tool_wrapping_runtime.feature) covering argument mapping, transform execution, delegation, chaining, cycle detection, depth limits, sandbox restrictions, and error handling.
  • 8 Robot Framework integration tests (robot/tool_wrapping_runtime.robot) for end-to-end smoke testing.
  • ASV benchmarks (benchmarks/tool_wrapping_bench.py) for mapping, transform, and delegation overhead.

Quality gates passed

  • nox -s lint
  • nox -s typecheck (0 Pyright errors)
  • nox -s unit_tests (8154 scenarios, 0 failures)
  • Robot tests (8/8 passed)
## Implementation Complete — PR #555 Implementation of tool wrapping runtime has been submitted in PR #555. ### What was implemented **`src/cleveragents/tool/wrapping.py`** (~470 lines) — the core runtime engine with three main components: 1. **`ArgumentMapper`** — Translates arguments from wrapper schema to wrapped tool schema using configurable `argument_mapping`. Supports identity mapping (passthrough) and explicit key remapping. 2. **`TransformExecutor`** — Executes `transform` functions on wrapped tool output in a sandboxed `RestrictedPython` context. The sandbox provides `result` (tool output), `json`/`re`/`math`/`datetime` modules, and blocks all I/O, imports, and attribute access outside the allowlist. 3. **`WrappedToolExecutor`** — Orchestrates the full delegation pipeline: resolves `wraps` references via the tool registry, detects cycles, enforces max chain depth (10), maps arguments, delegates execution, and applies transforms. Supports composable chaining (wrapper wrapping wrapper). **Error types**: `WrappedToolNotFoundError`, `WrappingCycleError`, `WrappingDepthExceededError`, `TransformExecutionError` — all with clear diagnostic messages. ### Testing - **20 Behave BDD scenarios** (`features/tool_wrapping_runtime.feature`) covering argument mapping, transform execution, delegation, chaining, cycle detection, depth limits, sandbox restrictions, and error handling. - **8 Robot Framework integration tests** (`robot/tool_wrapping_runtime.robot`) for end-to-end smoke testing. - **ASV benchmarks** (`benchmarks/tool_wrapping_bench.py`) for mapping, transform, and delegation overhead. ### Quality gates passed - `nox -s lint` ✅ - `nox -s typecheck` ✅ (0 Pyright errors) - `nox -s unit_tests` ✅ (8154 scenarios, 0 failures) - Robot tests ✅ (8/8 passed)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#543
No description provided.