cleveragents/cleveragents-core

Fork 3

UAT: `ActorLoader._resolve_tools()` only logs a warning for unresolved tool references instead of raising a validation error #5480

New issue

Open

opened 2026-04-09 06:58:36 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 06:58:36 +00:00

Owner

Bug Report

Feature Area

Actor System Compilation — Actor YAML schema validation and error handling

What Was Tested

Code-level analysis of src/cleveragents/actor/loader.py against the specification's actor validation requirements.

Expected Behavior (from spec)

The spec states that actor YAML files are validated at load time. Tool references in actor configs should be validated against the Tool Registry. An actor with an unresolvable tool reference should fail validation with a clear error message.

Actual Behavior (from code)

src/cleveragents/actor/loader.py lines 290-301 show that _resolve_tools() only logs a warning for unresolved tool references:

def _resolve_tools(self, config: ActorConfigSchema) -> None:
    if self._tool_registry is None:
        return
    for tool_ref in config.tools:
        if isinstance(tool_ref, str):
            spec = self._tool_registry.get(tool_ref)
            if spec is None:
                msg = (
                    f"Unresolved tool reference '{tool_ref}' "
                    f"in actor '{config.name}'"
                )
                logger.warning(msg)
                self._warnings.append(msg)

The method:

Logs a warning but does NOT raise a ValidationError
Adds the message to self._warnings (accessible via loader.warnings)
Allows the actor to be loaded and cached despite having broken tool references

This means an actor with a tool reference like files/nonexistent_tool will be loaded successfully, and the broken reference will only be discovered at runtime when the tool is actually invoked.

Impact

Actors with broken tool references are silently loaded and cached.
The broken reference is only discovered at runtime, not at load time.
This violates the fail-fast principle stated in CONTRIBUTING.md: "Public and protected methods must validate all arguments as the first step (fail-fast)."
Users get no immediate feedback that their actor configuration is broken.
The features/actor_loading.feature scenario "Actor with unresolvable tool reference logs warning" explicitly tests and accepts this behavior, suggesting it was intentional — but it conflicts with the spec's validation requirements.

Code Location

src/cleveragents/actor/loader.py lines 290-301 (_resolve_tools())
features/actor_loading.feature lines 221-229 (test that accepts warning behavior)

Note

This is a design decision that may have been intentional (to allow actors to be loaded even when the tool registry is incomplete). However, it creates a silent failure mode that conflicts with the spec's validation requirements. The behavior should be documented or changed to raise a validation error.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report ### Feature Area Actor System Compilation — Actor YAML schema validation and error handling ### What Was Tested Code-level analysis of `src/cleveragents/actor/loader.py` against the specification's actor validation requirements. ### Expected Behavior (from spec) The spec states that actor YAML files are validated at load time. Tool references in actor configs should be validated against the Tool Registry. An actor with an unresolvable tool reference should fail validation with a clear error message. ### Actual Behavior (from code) `src/cleveragents/actor/loader.py` lines 290-301 show that `_resolve_tools()` only logs a warning for unresolved tool references: ```python def _resolve_tools(self, config: ActorConfigSchema) -> None: if self._tool_registry is None: return for tool_ref in config.tools: if isinstance(tool_ref, str): spec = self._tool_registry.get(tool_ref) if spec is None: msg = ( f"Unresolved tool reference '{tool_ref}' " f"in actor '{config.name}'" ) logger.warning(msg) self._warnings.append(msg) ``` The method: - Logs a warning but does NOT raise a `ValidationError` - Adds the message to `self._warnings` (accessible via `loader.warnings`) - Allows the actor to be loaded and cached despite having broken tool references This means an actor with a tool reference like `files/nonexistent_tool` will be loaded successfully, and the broken reference will only be discovered at runtime when the tool is actually invoked. ### Impact - Actors with broken tool references are silently loaded and cached. - The broken reference is only discovered at runtime, not at load time. - This violates the fail-fast principle stated in CONTRIBUTING.md: "Public and protected methods must validate all arguments as the first step (fail-fast)." - Users get no immediate feedback that their actor configuration is broken. - The `features/actor_loading.feature` scenario "Actor with unresolvable tool reference logs warning" explicitly tests and accepts this behavior, suggesting it was intentional — but it conflicts with the spec's validation requirements. ### Code Location - `src/cleveragents/actor/loader.py` lines 290-301 (`_resolve_tools()`) - `features/actor_loading.feature` lines 221-229 (test that accepts warning behavior) ### Note This is a design decision that may have been intentional (to allow actors to be loaded even when the tool registry is incomplete). However, it creates a silent failure mode that conflicts with the spec's validation requirements. The behavior should be documented or changed to raise a validation error. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 06:58:57 +00:00

HAL9000 referenced this issue

2026-04-09 07:06:47 +00:00

EPIC: Actor Execution & Configuration — GRAPH Actor, Config Schema & CLI Compliance (v3.5.0) #5502

HAL9000 referenced this issue

2026-04-09 07:13:39 +00:00

[AUTO-EPIC] Epic Planning Health Report (Cycle 11) #5298

HAL9000 referenced this issue

2026-04-09 08:20:03 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 1) #5369

HAL9000 added the

State

Verified

label

2026-04-09 10:16:32 +00:00

HAL9000 commented

2026-04-09 10:20:23 +00:00

Author

Owner

🏷️ Label Fix Applied by Backlog Groomer

The State/Verified label has been added to this issue.

Reason: During a routine backlog grooming pass, this issue was found to have Type/Bug and a Priority/ label but was missing a State/* label entirely — a violation of the CONTRIBUTING.md requirement that every issue must have exactly one State/ label.

Since this issue has been triaged with a priority and type, State/Verified is the appropriate state: the issue has been confirmed as legitimate and is now part of the active backlog.

No other changes were made.

Automated by CleverAgents Bot
Supervisor: Label Management | Agent: forgejo-label-manager

## 🏷️ Label Fix Applied by Backlog Groomer The `State/Verified` label has been added to this issue. **Reason**: During a routine backlog grooming pass, this issue was found to have `Type/Bug` and a `Priority/` label but was missing a `State/*` label entirely — a violation of the CONTRIBUTING.md requirement that every issue must have exactly one `State/` label. Since this issue has been triaged with a priority and type, `State/Verified` is the appropriate state: the issue has been confirmed as legitimate and is now part of the active backlog. No other changes were made. --- **Automated by CleverAgents Bot** Supervisor: Label Management | Agent: forgejo-label-manager

HAL9000 referenced this issue

2026-04-09 12:09:11 +00:00

[AUTO-GROOMER] Backlog Grooming Report (Cycle 59) #5949

HAL9000 referenced this issue

2026-04-14 04:36:00 +00:00

fix(actor): Fix Actor CLI compliance — actor add positional arg, actor remove --format flag, ActorLoader tool validation (fixes #5425, #5457, #5480) #8995