brent.edwards
3d2cd05f56
CI / lint (pull_request) Successful in 15s
CI / typecheck (pull_request) Successful in 28s
CI / quality (pull_request) Successful in 16s
CI / security (pull_request) Successful in 51s
CI / build (pull_request) Successful in 21s
CI / integration_tests (pull_request) Successful in 9m6s
CI / unit_tests (pull_request) Failing after 26m29s
CI / coverage (pull_request) Successful in 8m38s
CI / docker (pull_request) Has been skipped
92 lines
3.2 KiB
Markdown
92 lines
3.2 KiB
Markdown
# Tool & Validation Domain Models
|
|
|
|
The `Tool` and `Validation` domain models are defined in
|
|
`src/cleveragents/domain/models/core/tool.py`.
|
|
|
|
## Overview
|
|
|
|
| Model | Purpose |
|
|
|--------------|--------------------------------------------|
|
|
| `Tool` | Atomic unit of execution — namespaced, independently registered callable operation |
|
|
| `Validation` | Extends `Tool` with pass/fail semantics — always read-only, never writes, never checkpointable |
|
|
|
|
### Key classes
|
|
|
|
- **`ToolSource`** — enum: `mcp`, `agent_skill`, `builtin`, `custom`, `wrapped`
|
|
- **`ToolType`** — enum: `tool`, `validation`
|
|
- **`ValidationMode`** — enum: `required`, `informational`
|
|
- **`ToolCapability`** — capability metadata (read_only, writes, checkpointable, side_effects, etc.)
|
|
- **`ResourceSlot`** — typed resource binding declaration
|
|
- **`ToolLifecycle`** — optional lifecycle hooks (discover, activate, deactivate)
|
|
|
|
### Factory methods
|
|
|
|
- `Tool.from_config(config)` — create a Tool from a YAML configuration dict
|
|
- `Validation.from_config(config)` — create a Validation from a YAML configuration dict
|
|
|
|
### Serialization
|
|
|
|
- `tool.as_cli_dict()` — stable-ordered dictionary for CLI rendering
|
|
|
|
## Testing
|
|
|
|
### Behave BDD tests (unit tests)
|
|
|
|
The primary test suite lives in `features/tool_model.feature` with step
|
|
definitions in `features/steps/tool_model_steps.py`.
|
|
|
|
Run with nox:
|
|
|
|
```bash
|
|
nox -s unit_tests -- features/tool_model.feature
|
|
```
|
|
|
|
The Behave suite covers:
|
|
|
|
- Tool/Validation model creation and field validation
|
|
- Source-conditional field requirements
|
|
- Capability constraint enforcement
|
|
- ResourceSlot binding validation
|
|
- `from_config()` loading (including YAML example files)
|
|
- `as_cli_dict()` output stability
|
|
- Enum value completeness
|
|
|
|
### Robot Framework smoke tests
|
|
|
|
The Robot smoke suite in `robot/tool_model.robot` provides integration-level
|
|
smoke tests that verify the same model creation and YAML loader outputs
|
|
through a subprocess helper (`robot/helper_tool_model.py`).
|
|
|
|
Run with nox:
|
|
|
|
```bash
|
|
nox -s integration_tests -- robot/tool_model.robot
|
|
```
|
|
|
|
Or directly with Robot Framework:
|
|
|
|
```bash
|
|
robot --outputdir build/reports/robot robot/tool_model.robot
|
|
```
|
|
|
|
The Robot suite covers:
|
|
|
|
| Test Case | What it verifies |
|
|
|-----------|-----------------|
|
|
| Create Minimal Tool Model | Builtin tool creation, identity fields |
|
|
| Create Validation Model With Forced Constraints | Validation read_only/writes/checkpointable forcing |
|
|
| Load Custom Tool From YAML Config | `Tool.from_config()` with `examples/tools/custom-tool.yaml` |
|
|
| Load MCP Tool From YAML Config | `Tool.from_config()` with `examples/tools/mcp-tool.yaml` |
|
|
| Load Wrapped Validation From YAML Config | `Validation.from_config()` with `examples/validations/wrapped-validation.yaml` |
|
|
| Load Required Validation From YAML Config | `Validation.from_config()` with `examples/validations/required-validation.yaml` |
|
|
| Validate Forced Constraints On Writable Capability | Validation forces read_only even with writable capability |
|
|
| Reject Invalid Tool Configurations | Invalid name, missing code, conflicting capabilities rejected |
|
|
|
|
### ASV benchmarks
|
|
|
|
Performance benchmarks live in `benchmarks/tool_model_bench.py`:
|
|
|
|
```bash
|
|
nox -s benchmark
|
|
```
|