cleveragents/cleveragents-core

Fork 3

UAT: `PluginLoader.validate_protocol()` uses no-arg constructor for validation — plugins requiring constructor args fail protocol validation silently #5738

New issue

Open

opened 2026-04-09 08:55:22 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 08:55:22 +00:00

Owner

Bug Report

Feature Area: Plugin Architecture — Plugin API / Protocol Validation
Priority: Backlog (non-critical for MVP, but causes silent failures for realistic plugins)

What Was Tested

Analyzed src/cleveragents/infrastructure/plugins/loader.py validate_protocol() method for correctness.

Expected Behavior (from spec)

The PluginLoader.validate_protocol() method should reliably verify that a plugin class satisfies a given @runtime_checkable Protocol. The validation should work for any plugin class, including those that require constructor arguments.

Actual Behavior

validate_protocol() attempts to instantiate the class with a no-arg constructor for validation:

@staticmethod
def validate_protocol(klass: type[Any], protocol: type[Any]) -> bool:
    # Try instance check first (most reliable for runtime_checkable)
    try:
        instance = klass()  # ← No-arg constructor assumed
        if isinstance(instance, protocol):
            return True
    except Exception:
        # If instantiation fails, try subclass check
        try:
            if issubclass(klass, protocol):
                return True
        except TypeError:
            pass
    
    msg = (...)
    raise ProtocolMismatchError(msg)

Problems:

Silent fallback to issubclass: If klass() raises (e.g., because the plugin requires constructor arguments like a database connection), the code falls back to issubclass(klass, protocol). For @runtime_checkable Protocols, issubclass only checks for method existence, not method signatures. This can produce false positives.
Swallows all exceptions: The bare except Exception swallows TypeError, ValueError, ImportError, and any other exception from klass(), making it impossible to distinguish "plugin requires args" from "plugin is broken".
No way to pass constructor args: There is no mechanism to pass configuration or dependencies to the plugin during validation.

Code Location

src/cleveragents/infrastructure/plugins/loader.py — validate_protocol() (lines 218–255)

Impact

A plugin that requires constructor arguments (e.g., MyPlugin(config: dict)) will:

Fail instantiation with TypeError: __init__() missing 1 required argument: 'config'
Fall back to issubclass check
Pass validation even if the class doesn't actually implement all protocol methods
Fail at runtime when activate_plugin() calls cls() with no args

Steps to Reproduce

from typing import Protocol, runtime_checkable

@runtime_checkable
class MyProtocol(Protocol):
    def do_thing(self) -> str: ...

class MyPlugin:
    def __init__(self, required_arg: str):  # Requires arg
        self.required_arg = required_arg
    
    def do_thing(self) -> str:
        return self.required_arg

# This should pass validation (class implements protocol)
# But validate_protocol() will fall back to issubclass check
result = PluginLoader.validate_protocol(MyPlugin, MyProtocol)
# Returns True via issubclass fallback — but activate_plugin() will fail

Suggested Fix

Use issubclass as the primary check (structural check without instantiation), and only attempt instantiation as a secondary verification:

@staticmethod
def validate_protocol(klass: type[Any], protocol: type[Any]) -> bool:
    # Primary: structural check via issubclass (works without instantiation)
    try:
        if issubclass(klass, protocol):
            return True
    except TypeError:
        pass
    
    # Secondary: try instantiation if structural check fails
    try:
        instance = klass()
        if isinstance(instance, protocol):
            return True
    except TypeError:
        # Class requires constructor args — structural check already passed
        pass
    except Exception:
        pass
    
    raise ProtocolMismatchError(...)

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: Plugin Architecture — Plugin API / Protocol Validation **Priority**: Backlog (non-critical for MVP, but causes silent failures for realistic plugins) ### What Was Tested Analyzed `src/cleveragents/infrastructure/plugins/loader.py` `validate_protocol()` method for correctness. ### Expected Behavior (from spec) The `PluginLoader.validate_protocol()` method should reliably verify that a plugin class satisfies a given `@runtime_checkable` Protocol. The validation should work for any plugin class, including those that require constructor arguments. ### Actual Behavior `validate_protocol()` attempts to instantiate the class with a no-arg constructor for validation: ```python @staticmethod def validate_protocol(klass: type[Any], protocol: type[Any]) -> bool: # Try instance check first (most reliable for runtime_checkable) try: instance = klass() # ← No-arg constructor assumed if isinstance(instance, protocol): return True except Exception: # If instantiation fails, try subclass check try: if issubclass(klass, protocol): return True except TypeError: pass msg = (...) raise ProtocolMismatchError(msg) ``` **Problems:** 1. **Silent fallback to `issubclass`**: If `klass()` raises (e.g., because the plugin requires constructor arguments like a database connection), the code falls back to `issubclass(klass, protocol)`. For `@runtime_checkable` Protocols, `issubclass` only checks for method existence, not method signatures. This can produce false positives. 2. **Swallows all exceptions**: The bare `except Exception` swallows `TypeError`, `ValueError`, `ImportError`, and any other exception from `klass()`, making it impossible to distinguish "plugin requires args" from "plugin is broken". 3. **No way to pass constructor args**: There is no mechanism to pass configuration or dependencies to the plugin during validation. ### Code Location - `src/cleveragents/infrastructure/plugins/loader.py` — `validate_protocol()` (lines 218–255) ### Impact A plugin that requires constructor arguments (e.g., `MyPlugin(config: dict)`) will: 1. Fail instantiation with `TypeError: __init__() missing 1 required argument: 'config'` 2. Fall back to `issubclass` check 3. Pass validation even if the class doesn't actually implement all protocol methods 4. Fail at runtime when `activate_plugin()` calls `cls()` with no args ### Steps to Reproduce ```python from typing import Protocol, runtime_checkable @runtime_checkable class MyProtocol(Protocol): def do_thing(self) -> str: ... class MyPlugin: def __init__(self, required_arg: str): # Requires arg self.required_arg = required_arg def do_thing(self) -> str: return self.required_arg # This should pass validation (class implements protocol) # But validate_protocol() will fall back to issubclass check result = PluginLoader.validate_protocol(MyPlugin, MyProtocol) # Returns True via issubclass fallback — but activate_plugin() will fail ``` ### Suggested Fix Use `issubclass` as the primary check (structural check without instantiation), and only attempt instantiation as a secondary verification: ```python @staticmethod def validate_protocol(klass: type[Any], protocol: type[Any]) -> bool: # Primary: structural check via issubclass (works without instantiation) try: if issubclass(klass, protocol): return True except TypeError: pass # Secondary: try instantiation if structural check fails try: instance = klass() if isinstance(instance, protocol): return True except TypeError: # Class requires constructor args — structural check already passed pass except Exception: pass raise ProtocolMismatchError(...) ``` --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 08:56:33 +00:00

HAL9000 referenced this issue

2026-04-09 09:04:26 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 28) #5666

HAL9000 added this to the v3.5.0 milestone

2026-04-09 09:05:17 +00:00

HAL9000 commented

2026-04-09 09:06:27 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added a new dependency

2026-04-09 10:20:32 +00:00

#5711 EPIC: Plugin Architecture Extensions — Plugin Manager, CLI & Lifecycle (v3.6.0)

HAL9000 commented

2026-04-09 10:42:16 +00:00

Author

Owner

Hierarchical Compliance Fix: This issue was detected as an orphan (no parent Epic).

Solution: Linked to Epic #5711 (Plugin Architecture Extensions — Plugin Manager, CLI & Lifecycle) based on scope alignment — PluginLoader.validate_protocol() is part of the plugin architecture.

Hierarchy: Issue #5738 → Epic #5711

Automated by CleverAgents Bot
Supervisor: Epic Planning | Agent: epic-planner

**Hierarchical Compliance Fix**: This issue was detected as an orphan (no parent Epic). **Solution**: Linked to Epic #5711 (Plugin Architecture Extensions — Plugin Manager, CLI & Lifecycle) based on scope alignment — PluginLoader.validate_protocol() is part of the plugin architecture. **Hierarchy**: Issue #5738 → Epic #5711 --- **Automated by CleverAgents Bot** Supervisor: Epic Planning | Agent: epic-planner

HAL9000 referenced this issue

2026-04-09 10:56:46 +00:00

[AUTO-EPIC] Epic Planning Health Report (Cycle 13) #5737