UAT: `agents validation attach` rejects plain Tool names without type validation error — spec requires rejection #1416

New issue

Open

opened 2026-04-02 17:45:32 +00:00 by freemo · 0 comments

freemo commented

2026-04-02 17:45:32 +00:00

Owner

Bug Report

Feature Area: Validation Pipeline (v3.4.0)

What Was Tested

The agents validation attach command's enforcement that only Validation-type tools (not plain Tools) can be attached.

Expected Behavior (from spec)

Per docs/specification.md line 22310:

agents validation attach rejects a name that resolves to a plain Tool rather than a Validation. This distinction is enforced by the type discriminator stored in the Tool Registry: each entry is tagged as either tool or validation.

Actual Behavior

The ToolRegistryService.attach_validation() method does NOT check the tool_type field of the retrieved tool. It only checks that the tool exists (not None), then proceeds to attach regardless of whether it's a tool or validation type:

# In tool_registry_service.py attach_validation():
existing = self._tool_repo.get_by_name(validation_name)
if existing is None:
    raise NotFoundError(...)
# NO check: if existing.tool_type != "validation": raise error

This means a plain Tool can be attached as a validation, violating the type safety guarantee described in the spec.

Steps to Reproduce

Register a plain tool: agents tool add --config ./tools/my-tool.yaml
Attempt to attach it as a validation: agents validation attach local/my-resource local/my-tool
Expected: Error "local/my-tool is a plain Tool, not a Validation"
Actual: Attachment succeeds silently

Code Location

src/cleveragents/application/services/tool_registry_service.py — attach_validation() method (lines ~169-185). Missing type check after get_by_name().

Severity

High — This violates a core type safety invariant described in the spec. It allows plain tools to be used where validations are specifically required, undermining the validation pipeline's integrity.

Metadata

Branch: bugfix/m5-validation-attach-type-check
Commit Message: fix(validation): enforce tool_type check in attach_validation to reject plain Tools
Milestone: v3.5.0
Parent Epic: #357

Subtasks

Add tool_type check in ToolRegistryService.attach_validation() after get_by_name() call
Define and raise a new InvalidToolTypeError (or reuse an existing domain error) when tool_type != "validation"
Propagate the error through the CLI with a user-friendly message: "<name> is a plain Tool, not a Validation"
Tests (Behave): Add scenario "Attaching a plain Tool as a validation is rejected"
Tests (Robot): Add integration test for type rejection
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

All subtasks are completed and checked off
agents validation attach rejects plain Tools with a clear error message
The type check is enforced at the service layer, not just the CLI layer
BDD scenarios cover the rejection case
A PR is opened from the branch to master, reviewed, and merged
All nox stages pass
Coverage >= 97%

## Bug Report **Feature Area:** Validation Pipeline (v3.4.0) ### What Was Tested The `agents validation attach` command's enforcement that only Validation-type tools (not plain Tools) can be attached. ### Expected Behavior (from spec) Per `docs/specification.md` line 22310: > `agents validation attach` rejects a name that resolves to a plain Tool rather than a Validation. This distinction is enforced by the `type` discriminator stored in the Tool Registry: each entry is tagged as either `tool` or `validation`. ### Actual Behavior The `ToolRegistryService.attach_validation()` method does NOT check the `tool_type` field of the retrieved tool. It only checks that the tool exists (not None), then proceeds to attach regardless of whether it's a `tool` or `validation` type: ```python # In tool_registry_service.py attach_validation(): existing = self._tool_repo.get_by_name(validation_name) if existing is None: raise NotFoundError(...) # NO check: if existing.tool_type != "validation": raise error ``` This means a plain Tool can be attached as a validation, violating the type safety guarantee described in the spec. ### Steps to Reproduce 1. Register a plain tool: `agents tool add --config ./tools/my-tool.yaml` 2. Attempt to attach it as a validation: `agents validation attach local/my-resource local/my-tool` 3. Expected: Error "local/my-tool is a plain Tool, not a Validation" 4. Actual: Attachment succeeds silently ### Code Location `src/cleveragents/application/services/tool_registry_service.py` — `attach_validation()` method (lines ~169-185). Missing type check after `get_by_name()`. ### Severity High — This violates a core type safety invariant described in the spec. It allows plain tools to be used where validations are specifically required, undermining the validation pipeline's integrity. --- ## Metadata - **Branch**: `bugfix/m5-validation-attach-type-check` - **Commit Message**: `fix(validation): enforce tool_type check in attach_validation to reject plain Tools` - **Milestone**: v3.5.0 - **Parent Epic**: #357 ## Subtasks - [ ] Add `tool_type` check in `ToolRegistryService.attach_validation()` after `get_by_name()` call - [ ] Define and raise a new `InvalidToolTypeError` (or reuse an existing domain error) when `tool_type != "validation"` - [ ] Propagate the error through the CLI with a user-friendly message: `"<name> is a plain Tool, not a Validation"` - [ ] Tests (Behave): Add scenario "Attaching a plain Tool as a validation is rejected" - [ ] Tests (Robot): Add integration test for type rejection - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] All subtasks are completed and checked off - [ ] `agents validation attach` rejects plain Tools with a clear error message - [ ] The type check is enforced at the service layer, not just the CLI layer - [ ] BDD scenarios cover the rejection case - [ ] A PR is opened from the branch to `master`, reviewed, and merged - All nox stages pass - Coverage >= 97%

freemo added the

labels

2026-04-02 17:47:22 +00:00

freemo added this to the v3.5.0 milestone