HAL9000
cef70ff98a
Add Robot Framework integration test verifying that load_from_entry_points does not call ep.load() for entry points with disallowed module prefixes (security regression test for issue #7476). Also add HAL 9000 to CONTRIBUTORS.md per CONTRIBUTING.md process rules. ISSUES CLOSED: #7476
393 lines
18 KiB
Gherkin
393 lines
18 KiB
Gherkin
@extensibility @plugin_architecture
|
|
Feature: Plugin Architecture Framework with module:ClassName resolution
|
|
As a CleverAgents developer
|
|
I want a plugin architecture that loads custom implementations via
|
|
module:ClassName resolution and entry-point discovery
|
|
So that tools, strategies, backends, and pipeline components can be
|
|
extended without modifying core code
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# PluginState enum
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_state
|
|
Scenario: PluginState enum has all required values
|
|
Given the PluginState enum is available
|
|
Then it should have values "discovered", "activated", "executing", "deactivated", "errored"
|
|
|
|
@plugin_state
|
|
Scenario: PluginState values are strings
|
|
Given the PluginState enum is available
|
|
Then each PluginState value should be a string
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ExtensionPoint model
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@extension_point
|
|
Scenario: ExtensionPoint creation with required fields
|
|
Given I create an ExtensionPoint with name "test-ep" and a protocol type
|
|
Then the ExtensionPoint name should be "test-ep"
|
|
And the ExtensionPoint should have a protocol_type
|
|
|
|
@extension_point
|
|
Scenario: ExtensionPoint is frozen
|
|
Given I create an ExtensionPoint with name "frozen-ep" and a protocol type
|
|
Then attempting to mutate the ExtensionPoint name should raise an error
|
|
|
|
@extension_point
|
|
Scenario: ExtensionPoint rejects empty name
|
|
When I attempt to create an ExtensionPoint with an empty name
|
|
Then a plugin validation error should be raised
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# PluginDescriptor model
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_descriptor
|
|
Scenario: PluginDescriptor creation with defaults
|
|
Given I create a PluginDescriptor with name "my-plugin"
|
|
Then the descriptor state should be "discovered"
|
|
And the descriptor version should be "0.0.0"
|
|
And the descriptor dependencies should be empty
|
|
|
|
@plugin_descriptor
|
|
Scenario: PluginDescriptor with full metadata
|
|
Given I create a PluginDescriptor with full metadata
|
|
Then the descriptor should have name "full-plugin"
|
|
And the descriptor should have version "1.2.3"
|
|
And the descriptor should have author "TestAuthor"
|
|
And the descriptor should have module_path "cleveragents.test"
|
|
And the descriptor should have class_name "TestClass"
|
|
|
|
@plugin_descriptor
|
|
Scenario: PluginDescriptor state is mutable
|
|
Given I create a PluginDescriptor with name "mutable-plugin"
|
|
When I set the descriptor state to "activated"
|
|
Then the descriptor state should be "activated"
|
|
|
|
@plugin_descriptor
|
|
Scenario: PluginDescriptor rejects empty name
|
|
When I attempt to create a PluginDescriptor with an empty name
|
|
Then a plugin validation error should be raised
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Plugin exceptions
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_exceptions
|
|
Scenario: PluginError is base exception
|
|
Then PluginLoadError should be a subclass of PluginError
|
|
And PluginNotFoundError should be a subclass of PluginError
|
|
And ProtocolMismatchError should be a subclass of PluginError
|
|
|
|
@plugin_exceptions
|
|
Scenario: PluginLoadError carries message
|
|
When I raise a PluginLoadError with message "test error"
|
|
Then the exception message should contain "test error"
|
|
|
|
@plugin_exceptions
|
|
Scenario: PluginNotFoundError carries message
|
|
When I raise a PluginNotFoundError with message "not found"
|
|
Then the exception message should contain "not found"
|
|
|
|
@plugin_exceptions
|
|
Scenario: ProtocolMismatchError carries message
|
|
When I raise a ProtocolMismatchError with message "mismatch"
|
|
Then the exception message should contain "mismatch"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# PluginLoader — load_class success and failure
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_loader @success
|
|
Scenario: PluginLoader loads a valid class
|
|
Given a PluginLoader with default prefixes
|
|
When I load class "InMemoryTextBackend" from module "cleveragents.domain.models.acms.stubs"
|
|
Then the loaded class should not be None
|
|
And the loaded class name should be "InMemoryTextBackend"
|
|
|
|
@plugin_loader @success
|
|
Scenario: PluginLoader loads another valid class
|
|
Given a PluginLoader with default prefixes
|
|
When I load class "InMemoryVectorBackend" from module "cleveragents.domain.models.acms.stubs"
|
|
Then the loaded class should not be None
|
|
|
|
@plugin_loader @failure
|
|
Scenario: PluginLoader rejects module outside allowed prefix
|
|
Given a PluginLoader with default prefixes
|
|
When I attempt to load class "Path" from module "pathlib"
|
|
Then a PluginLoadError should be raised
|
|
And the plugin error message should contain "not in the allowed prefix list"
|
|
|
|
@plugin_loader @failure
|
|
Scenario: PluginLoader fails on nonexistent module
|
|
Given a PluginLoader with default prefixes
|
|
When I attempt to load class "Foo" from module "cleveragents.nonexistent_module_xyz"
|
|
Then a PluginLoadError should be raised
|
|
And the plugin error message should contain "Cannot import module"
|
|
|
|
@plugin_loader @failure
|
|
Scenario: PluginLoader fails on nonexistent class
|
|
Given a PluginLoader with default prefixes
|
|
When I attempt to load class "NonExistentClassXyz" from module "cleveragents.domain.models.acms.stubs"
|
|
Then a PluginLoadError should be raised
|
|
And the plugin error message should contain "not found in module"
|
|
|
|
@plugin_loader @security
|
|
Scenario: PluginLoader with custom prefixes allows matching module
|
|
Given a PluginLoader with allowed prefixes "cleveragents.,mypackage."
|
|
When I load class "InMemoryTextBackend" from module "cleveragents.domain.models.acms.stubs"
|
|
Then the loaded class should not be None
|
|
|
|
@plugin_loader @security
|
|
Scenario: PluginLoader with empty prefix allowlist allows anything
|
|
Given a PluginLoader with empty prefix allowlist
|
|
When I load class "Path" from module "pathlib"
|
|
Then the loaded class should not be None
|
|
|
|
@plugin_loader
|
|
Scenario: PluginLoader allowed_prefixes property
|
|
Given a PluginLoader with default prefixes
|
|
Then the loader allowed_prefixes should contain "cleveragents."
|
|
|
|
@plugin_loader @failure
|
|
Scenario: PluginLoader rejects non-class attribute
|
|
Given a PluginLoader with default prefixes
|
|
When I attempt to load class "logger" from module "cleveragents.infrastructure.plugins.loader"
|
|
Then a PluginLoadError should be raised
|
|
And the plugin error message should contain "is not a class"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Protocol validation
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@protocol_validation @success
|
|
Scenario: validate_protocol passes for conforming class
|
|
Given a class that implements TextBackend protocol
|
|
When I validate it against the TextBackend protocol
|
|
Then the validation should return True
|
|
|
|
@protocol_validation @failure
|
|
Scenario: validate_protocol fails for non-conforming class
|
|
Given a class that does not implement any protocol
|
|
When I attempt to validate it against the TextBackend protocol
|
|
Then a ProtocolMismatchError should be raised
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Entry-point discovery
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@entry_points
|
|
Scenario: load_from_entry_points returns empty for unknown group
|
|
Given a PluginLoader with default prefixes
|
|
When I discover plugins from entry point group "cleveragents.nonexistent_group_xyz"
|
|
Then the discovered plugin list should be empty
|
|
|
|
@entry_points @mock
|
|
Scenario: load_from_entry_points discovers mocked entry points
|
|
Given a PluginLoader with default prefixes
|
|
And a mocked entry point group "cleveragents.plugins" with entry "test-ep=cleveragents.domain.models.acms.stubs:InMemoryTextBackend"
|
|
When I discover plugins from the mocked entry point group
|
|
Then the discovered plugin list should have 1 entry
|
|
And the first descriptor name should be "test-ep"
|
|
|
|
@entry_points @security @tdd_issue @tdd_issue_7476
|
|
Scenario: Entry point with disallowed prefix is skipped without loading module
|
|
Given a PluginLoader with default prefixes
|
|
And a mocked entry point group "cleveragents.plugins" with raw entry "malicious=maliciouspkg.attack:Exploit"
|
|
When I discover plugins from the mocked entry point group
|
|
Then the discovered plugin list should be empty
|
|
And the mocked entry point load should not be called
|
|
And a security warning should be emitted for disallowed entry point "malicious"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# PluginManager — lifecycle
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_manager @registration
|
|
Scenario: PluginManager registers and retrieves a plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "test-plugin" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
Then get_plugin should return the descriptor for "test-plugin"
|
|
And the plugin state should be "discovered"
|
|
|
|
@plugin_manager @registration
|
|
Scenario: PluginManager rejects duplicate registration
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "dup-plugin" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
And I attempt to register the same descriptor again
|
|
Then a PluginError should be raised
|
|
|
|
@plugin_manager @listing
|
|
Scenario: PluginManager lists all registered plugins
|
|
Given a fresh PluginManager instance
|
|
And I register plugins named "alpha", "beta", "gamma"
|
|
Then list_plugins should return 3 descriptors
|
|
|
|
@plugin_manager @not_found
|
|
Scenario: PluginManager raises on unknown plugin
|
|
Given a fresh PluginManager instance
|
|
When I attempt to get plugin "nonexistent"
|
|
Then a PluginNotFoundError should be raised
|
|
|
|
@plugin_manager @activate
|
|
Scenario: PluginManager activates a registered plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "act-plugin" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
And I activate the plugin "act-plugin"
|
|
Then the plugin state should be "activated"
|
|
And get_plugin_class for "act-plugin" should not be None
|
|
And get_plugin_instance for "act-plugin" should not be None
|
|
|
|
@plugin_manager @activate
|
|
Scenario: PluginManager rejects activating already-activated plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "double-act" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
And I activate the plugin "double-act"
|
|
And I attempt to activate the plugin "double-act" again
|
|
Then a PluginError should be raised
|
|
|
|
@plugin_manager @activate @failure
|
|
Scenario: PluginManager transitions to ERRORED on activation failure
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "bad-plugin" with module "cleveragents.nonexistent_module_xyz" and class "BadClass"
|
|
When I register the plugin descriptor
|
|
And I attempt to activate the plugin "bad-plugin"
|
|
Then a PluginLoadError should be raised
|
|
And the plugin "bad-plugin" state should be "errored"
|
|
|
|
@plugin_manager @activate @failure
|
|
Scenario: PluginManager fails activation with empty module_path
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "no-path" with empty module_path
|
|
When I register the plugin descriptor
|
|
And I attempt to activate the plugin "no-path"
|
|
Then a PluginLoadError should be raised
|
|
|
|
@plugin_manager @deactivate
|
|
Scenario: PluginManager deactivates an activated plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "deact-plugin" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
And I activate the plugin "deact-plugin"
|
|
And I deactivate the plugin "deact-plugin"
|
|
Then the plugin state should be "deactivated"
|
|
And get_plugin_class for "deact-plugin" should be None
|
|
And get_plugin_instance for "deact-plugin" should be None
|
|
|
|
@plugin_manager @deactivate
|
|
Scenario: PluginManager rejects deactivating a discovered plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "disc-plugin" with module "cleveragents.domain.models.acms.stubs" and class "InMemoryTextBackend"
|
|
When I register the plugin descriptor
|
|
And I attempt to deactivate the plugin "disc-plugin"
|
|
Then a PluginError should be raised
|
|
|
|
@plugin_manager @deactivate
|
|
Scenario: PluginManager can deactivate an errored plugin
|
|
Given a fresh PluginManager instance
|
|
And a PluginDescriptor for "err-plugin" with module "cleveragents.nonexistent_module_xyz" and class "BadClass"
|
|
When I register the plugin descriptor
|
|
And I attempt to activate the plugin "err-plugin" ignoring error
|
|
And I deactivate the plugin "err-plugin"
|
|
Then the plugin state should be "deactivated"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Config-driven registration
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_manager @config
|
|
Scenario: PluginManager registers from config dict
|
|
Given a fresh PluginManager instance
|
|
When I register a plugin from config with custom_module "cleveragents.domain.models.acms.stubs" and custom_class "InMemoryTextBackend"
|
|
Then the registered plugin descriptor should not be None
|
|
And the plugin should be in the manager registry
|
|
|
|
@plugin_manager @config
|
|
Scenario: PluginManager ignores config without plugin keys
|
|
Given a fresh PluginManager instance
|
|
When I register a plugin from config with no custom_module or custom_class
|
|
Then the registered plugin descriptor should be None
|
|
|
|
@plugin_manager @config
|
|
Scenario: PluginManager returns existing for duplicate config
|
|
Given a fresh PluginManager instance
|
|
When I register a plugin from config with custom_module "cleveragents.domain.models.acms.stubs" and custom_class "InMemoryTextBackend"
|
|
And I register the same plugin from config again
|
|
Then list_plugins should return 1 descriptor
|
|
|
|
@plugin_manager @config
|
|
Scenario: PluginManager batch registers from config list
|
|
Given a fresh PluginManager instance
|
|
When I register plugins from a config list of 3 entries
|
|
Then list_plugins should return 3 descriptors
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Extension point management
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_manager @extension_points
|
|
Scenario: PluginManager registers and lists extension points
|
|
Given a fresh PluginManager instance
|
|
When I register an extension point named "text-backend"
|
|
Then list_extension_points should return 1 entry
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Clear / cleanup
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@plugin_manager @clear
|
|
Scenario: PluginManager clear removes all plugins
|
|
Given a fresh PluginManager instance
|
|
And I register plugins named "a", "b", "c"
|
|
When I clear the plugin manager
|
|
Then list_plugins should return 0 descriptors
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Thread safety
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@thread_safety
|
|
Scenario: PluginManager supports concurrent registration
|
|
Given a fresh PluginManager instance
|
|
When 10 threads register plugins concurrently
|
|
Then list_plugins should return 10 descriptors
|
|
|
|
@thread_safety
|
|
Scenario: PluginManager supports concurrent activate and deactivate
|
|
Given a fresh PluginManager with 5 registered plugins
|
|
When 5 threads activate plugins concurrently
|
|
Then all 5 plugins should be in "activated" state
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DI container integration
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@di_container
|
|
Scenario: DI container provides PluginManager singleton
|
|
When I resolve PluginManager from the DI container
|
|
Then the resolved PluginManager should not be None
|
|
And resolving PluginManager again should return the same instance
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Discovery integration
|
|
# ---------------------------------------------------------------------------
|
|
|
|
@discover
|
|
Scenario: PluginManager discover with no entry points
|
|
Given a fresh PluginManager instance
|
|
When I call discover with group "cleveragents.nonexistent_ep_group_xyz"
|
|
Then the discovered list should be empty
|
|
|
|
@discover
|
|
Scenario: PluginManager discover registers newly found plugins
|
|
Given a fresh PluginManager instance
|
|
And a mocked entry point group for discover
|
|
When I call discover through the manager
|
|
Then newly discovered plugins should be in the registry
|