feat(registry): extend TemplateType and integrate PackageReference into template system #35

Merged
CoreRasurae merged 1 commits from feature/m1-registry-template-alignment into master 2026-06-10 12:10:12 +00:00
25 changed files with 3289 additions and 71 deletions
+7
View File
@@ -7,12 +7,19 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm
## [Unreleased]
### Fixed
- **Registry Resolution Bugfixes (PR Review rui.hu)**: Fixed `TemplateRegistry._instantiate_from_registry_ref` hardcoding `package_type="template"` instead of threading the template type, which broke all non-template registry resolutions. `_try_parse_registry_ref` now filters to `ReferenceType.REGISTRY` only, preserving the "local templates unaffected" acceptance criterion. `_original_reference` in resolved results now stores the verbatim original reference string, not the cache-key with type suffix. `application.py` template-type mapping extended to cover all 8 types (previously only AGENT/GRAPH/STREAM — the other 5 silently misclassified as STREAM). Duplicated `_instantiate_from_registry_ref`/`_try_parse_registry_ref` logic extracted into shared `_resolve_registry_ref` helper in `base.py`. `RegistryClient` now URL-encodes namespace/name path components to prevent injection. HTTPS enforcement with `allow_insecure` flag per spec §12.2. `ReferenceResolver.close_all()` closes clients under the async lock to prevent concurrent client leak. Async lock lazily initialised to avoid deprecation warning. Added `plural_name` property to `TemplateType`.
- **Registry Reference Resolution**: Fixed `ReferenceResolver._resolve_registry_async` passing the package name instead of the type code to `RegistryClient.resolve_package`, which would construct invalid API paths and break all registry integrations. Added `package_type` parameter to `resolve()` and `aresolve()`, with automatic mapping from `TemplateType` values to `PackageType` codes. Replaced `ValueError` with `TemplateError` for all registry fetch failure paths per acceptance criteria. Cache and client-pool operations now use `asyncio.Lock` in async coroutines (`aresolve`, `_resolve_registry_async`, `_get_client`, `close_all`) to prevent event-loop blocking, while retaining `threading.Lock` for the synchronous `resolve()` path. Use bounded `OrderedDict` with LRU eviction and consistently return deep copies to prevent cache poisoning. Fixed `_get_client()` race condition by protecting the full check-and-create under a single lock acquisition. Ad-hoc `ReferenceResolver` instances created during `_instantiate_from_registry_ref` are now properly closed in both success and error paths. Registry reference detection now uses proper `PackageReference.from_string()` parsing instead of a fragile `":" in template_name` check, and returns PackageReference for all reference types (LOCAL, ID, REGISTRY) enabling resolution of local and ID references through `instantiate_from_config` and `EnhancedTemplateRegistry.instantiate()`. Template `params` from registry-based configs are now applied to resolved content instead of being silently discarded. Added async `aresolve()` entry point to prevent `RuntimeError` when called from running event loops; `resolve()` now detects running loops and directs callers to `aresolve()`. Bare `except Exception: pass` blocks replaced with debug-level logging.
### Added
- **`create_executor()` router-facing API** (`cleveractors.create_executor`): New module-level factory function that constructs an `Executor` wrapping `PureLangGraph` and `AgentFactory`. Accepts `config_dict` (validated actor configuration), `credentials` (per-provider credential dict for per-request injection), `limits` (execution budget), and `pricing` (per-model cost table). `executor.execute(message)` runs the actor graph and returns an `ActorResult(response, prompt_tokens, completion_tokens, nodes)`. All four execution paths are supported — `llm`, `graph`, `tool`, and `multi_actor`. Credentials are passed to `AgentFactory` and never injected into the stored `config_dict` (ADR-2026 AC8). Token counts use tiktoken with heuristic fallback until #14 implements real `usage_metadata` extraction. Exported from `cleveractors.__init__` and `__all__` (ADR-2024, ADR-2026, ADR-2029).
- **Per-request credential injection** (`AgentFactory` + `LLMAgent`): `AgentFactory` now accepts an optional `credentials: dict[str, dict[str, str]] | None` parameter. When supplied, each credential entry (keyed by provider name) is forwarded to the corresponding `LLMAgent`. The LangChain client is constructed lazily on first access of the `chat_model` property using the injected credentials, so API keys are never baked into the stored actor config dict (ADR-2026).
- **Extended provider routing** (`LLMAgent`): Any provider not in `{openai, anthropic, google}` (e.g., `groq`, `fireworks`, `together`, `mistral`, `openrouter`, or the generic `openai_compatible` extension) is now routed to `ChatOpenAI(base_url=..., api_key=...)` using the `base_url` supplied in the credentials entry for that provider. Named providers and `openai_compatible` are treated identically under this routing path (ADR-2028).
- **`ReactiveAgentFactory` backward-compatibility alias**: A module-level type alias `ReactiveAgentFactory: type[AgentFactory] = AgentFactory` is provided in `cleveractors.agents.factory` for backward compatibility with existing code that references the old name from v2.0.0.
- **Registry Template Integration**: Extended `TemplateType` enum with five new members (`TEMPLATE`, `SKILL`, `ACTOR`, `MCP`, `LSP`) to align with the Package Registry Standard v1.0.0 package types (§3.2). Added optional `package_ref: PackageReference | None` field to `ComponentReference` for external registry references. Created `ReferenceResolver` class with multi-server `RegistryClient` pool and resolution caching for local, ID, and registry reference formats (§5.3). Updated `InstantiationContext` with registry-aware resolution and `_original_reference` propagation through component metadata. Extended `TemplateRegistry`, `EnhancedTemplateRegistry`, and `TemplateStore` to support all eight template types. Registry-style template references (e.g. `"git.cleverthis.com:acme/helper@v1.0.0"`) are now detected and resolved via `ReferenceResolver` during `instantiate_from_config`. 25 Robot Framework integration tests, 24 Behave BDD scenarios, and full coverage of all new modules.
- **`merge_configs()` public API** (`cleveractors.merge_configs`): New module-level function implementing the Actor Configuration Standard §3.1 deep-merge algorithm. Accepts an arbitrary number of `dict[str, Any]` arguments and returns a fresh merged dict without mutating any input. Merge semantics: absent key → add; both mappings → deep-merge recursively; both sequences → append; otherwise → replace. Zero-argument call returns `{}`. Exported from `cleveractors.__init__` and `__all__`.
- **`validate_dict()` public API** (`cleveractors.validate_dict`): New router-facing validation function that validates a spec-conformant Actor Configuration Standard v1.0.0 dict against the full schema and enforces platform-level structural constraints. Validates top-level key presence (`agents`, `routes`), agent types, LLM provider allowlists, route/edge field names (`source`/`target` only — legacy `from`/`to` rejected), node types, stream operator types, and structural limits (`max_graph_depth`, `max_subgraph_depth`, `max_total_nodes`) from the supplied `platform_limits` dict. Returns the dict unchanged when valid; raises `ConfigurationError` on any violation. Pure static validator with no file I/O, env-var reads, or app construction. Exported from `cleveractors.__init__` and listed in `__all__`. (ADR-2024, ADR-2025, ADR-2029)
- **Registry HTTP Client** (`cleveractors.registry`): Async registry client implementing the Package Registry Standard v1.0.0 HTTP API endpoints (§8) using `httpx.AsyncClient`. Supports all four endpoints: `GET /packages/{id}` (retrieve by PackageId), `GET /{type}/{ns}/{name}?version=` (reference resolution with version alias resolution per §4.2), `GET /browse` (discovery with type/namespace filters), and `GET /.well-known/cleverthis-packages` (registry metadata). Maps all 8 standard error types (§13.2) to typed exceptions via error-body type parsing with status-code fallback. Supports anonymous reads (§9.1) and optional API key authentication (§9.2). Includes async context manager support, 32 Behave BDD scenarios, 7 Robot Framework integration tests against a fake server, and ASV benchmarks.
+5 -5
View File
@@ -20,14 +20,14 @@ class RegistryClientBenchmark:
params: list[str] = ["actor", "template", "skill"]
def setup(self) -> None:
def setup(self, pkg_type: str = "") -> None:
self.client = RegistryClient(
base_url="http://127.0.0.1:9199",
timeout=5.0,
)
self.fake_package_id = "pkg_act_a1b2c3d4e5f67890abcdef1234567890abcdef"
def teardown(self) -> None:
def teardown(self, pkg_type: str = "") -> None:
async def _close() -> None:
await self.client.close()
@@ -36,7 +36,7 @@ class RegistryClientBenchmark:
except RuntimeError:
asyncio.run(_close())
def time_get_package(self) -> None:
def time_get_package(self, pkg_type: str = "") -> None:
async def _bench() -> None:
await self.client.get_package(self.fake_package_id)
@@ -64,7 +64,7 @@ class RegistryClientBenchmark:
except Exception:
pass
def time_browse(self) -> None:
def time_browse(self, pkg_type: str = "") -> None:
async def _bench() -> None:
await self.client.browse()
@@ -78,7 +78,7 @@ class RegistryClientBenchmark:
except Exception:
pass
def time_discover(self) -> None:
def time_discover(self, pkg_type: str = "") -> None:
async def _bench() -> None:
await self.client.discover()
+15 -1
View File
@@ -197,4 +197,18 @@ Feature: Registry HTTP Client
Scenario: Client can be used as async context manager
When I create a RegistryClient pointed at "https://registry.example.com"
When I use the client as an async context manager
Then the client should be closed after the context block
Then the client should be closed after the context block
# ── HTTPS enforcement (spec §12.2) ────────────────────────────────────
Scenario: Client warns when HTTP is used with an API key
When I create a RegistryClient with base URL "http://registry.example.com" and API key "token123"
Then the client should log a warning about cleartext credentials
Scenario: Client logs info when HTTP is used without an API key
When I create a RegistryClient pointed at "http://registry.example.com"
Then the client should log info about HTTPS requirement
Scenario: Client with allow_insecure suppresses HTTP warnings
When I create an insecure RegistryClient pointed at "http://registry.example.com"
Then the client should not log HTTPS warnings
@@ -0,0 +1,133 @@
Feature: Registry Reference Resolver and Generic Template Coverage
As a developer
I want the PackageContentResolver to resolve local, ID, and registry references with caching
So that package references are properly resolved and cached
Background:
Given RRC: I have a clean test environment for reference resolver coverage
Scenario: PackageContentResolver resolves LOCAL reference
Given RRC: I have a PackageContentResolver instance
When RRC: I resolve a LOCAL package reference
Then RRC: the result should include _original_reference
And RRC: the result should have name and type fields
Scenario: PackageContentResolver resolves ID reference
Given RRC: I have a PackageContentResolver instance
When RRC: I resolve an ID package reference
Then RRC: the result should include id and type fields
And RRC: _original_reference should match the ID string
Scenario: PackageContentResolver caches resolved references
Given RRC: I have a PackageContentResolver instance
When RRC: I resolve a LOCAL reference twice
Then RRC: the second resolve should return cached result
And RRC: the cache should contain the reference key
Scenario: PackageContentResolver get_client creates and reuses clients
Given RRC: I have a PackageContentResolver instance
When RRC: I get a client for a server
Then RRC: the client should be stored in the clients dict
And RRC: getting the same server should return the same client
Scenario: PackageContentResolver clear_cache removes all cached entries
Given RRC: I have a PackageContentResolver with cached entries
When RRC: I clear the cache
Then RRC: the cache should be empty
Scenario: GenericTemplate instantiate returns definition with defaults
Given RRC: I have a GenericTemplate instance with definition
When RRC: I instantiate the generic template
Then RRC: the result should be the definition dict
And RRC: _original_reference should be None by default
Scenario: GenericTemplate instantiate with params applies template vars
Given RRC: I have a GenericTemplate instance with template variables
And RRC: I have instantiation params
When RRC: I instantiate the generic template with params
Then RRC: the result should contain rendered variables
Scenario: PackageContentResolver resolves REGISTRY reference with mock
Given RRC: I have a PackageContentResolver instance
When RRC: I resolve a REGISTRY reference with a mocked client
Then RRC: the result should include _original_reference
And RRC: the result should contain server and namespace fields
Scenario: PackageContentResolver aresolve handles REGISTRY references
Given RRC: I have a PackageContentResolver instance
When RRC: I call aresolve on a REGISTRY reference
Then RRC: the result should contain server and namespace fields
Scenario: PackageContentResolver close_all closes all clients
Given RRC: I have a PackageContentResolver with multiple clients
When RRC: I call close_all synchronously
Then RRC: all client connections should be closed
Scenario: PackageContentResolver cache evicts oldest entry at max size
Given RRC: I have a PackageContentResolver instance
When RRC: I resolve more than max cache size LOCAL references
Then RRC: the cache should not exceed max size
And RRC: the oldest entry should be evicted
# ── PackageContentResolver aresolve and error paths ──
Scenario: PackageContentResolver aresolve handles LOCAL references via async lock
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on a LOCAL reference
Then FBF: the result should have name and type fields
And FBF: the result should include _original_reference
Scenario: PackageContentResolver aresolve handles ID references via async lock
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on an ID reference
Then FBF: the result should include id and type fields
Scenario: PackageContentResolver aresolve caches and returns cached results
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on a LOCAL reference twice
Then FBF: the second aresolve should return cached result
Scenario: PackageContentResolver resolve rejects incomplete registry reference
Given RRC: I have a PackageContentResolver instance
When FBF: I resolve a REGISTRY reference missing server/namespace/name
Then FBF: the result should be None
Scenario: PackageContentResolver resolve rejects missing package_type for REGISTRY
Given RRC: I have a PackageContentResolver instance
When FBF: I resolve a REGISTRY reference without package_type
Then FBF: the result should be None
Scenario: PackageContentResolver aresolve rejects incomplete registry reference async
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on a REGISTRY reference missing server/namespace/name
Then FBF: the aresolve result should be None
Scenario: PackageContentResolver aresolve rejects missing package_type async
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on a REGISTRY reference without package_type
Then FBF: the aresolve result should be None
Scenario: PackageContentResolver resolve detects running event loop
Given RRC: I have a PackageContentResolver instance
When FBF: I try to resolve a REGISTRY reference from inside a running event loop
Then FBF: a RuntimeError should be raised about event loop conflict
Scenario: PackageContentResolver close_all handles individual client close exceptions
Given RRC: I have a PackageContentResolver with clients where one raises on close
When FBF: I call close_all
Then FBF: all clients should be closed without raising
Scenario: PackageContentResolver resolve returns None for unknown reference type
Given RRC: I have a PackageContentResolver instance
When FBF: I resolve a reference with an unknown reference type
Then FBF: the result should be None
Scenario: PackageContentResolver _put_cache moves existing key to end
Given RRC: I have a PackageContentResolver instance
When FBF: I call _put_cache with an already existing key
Then FBF: the key should be moved to the end of the cache
Scenario: PackageContentResolver aresolve returns None for unknown reference type
Given RRC: I have a PackageContentResolver instance
When FBF: I call aresolve on a reference with an unknown reference type
Then FBF: the result should be None
@@ -0,0 +1,225 @@
Feature: Registry Template Integration
As a developer
I want templates to resolve package references from external registries
So that templates can reference remote packages via the Package Registry Standard
Background:
Given RTI: I have a clean test environment for registry template integration
Scenario: TemplateType includes new registry types
When RTI: I access all TemplateType enum values
Then RTI: all template types including new ones should be available
And RTI: new enum values should have correct string representations
Scenario: ComponentReference supports optional package_ref
Given RTI: I create a PackageReference from a registry string
When RTI: I create a ComponentReference with package_ref
Then RTI: the ComponentReference should have the package_ref set
And RTI: ref_type and ref_name should be accessible
Scenario: InstantiationContext resolves local components with package_ref
Given RTI: I have an InstantiationContext with a PackageContentResolver
And RTI: I add a component to the context
When RTI: I resolve a ComponentReference without package_ref
Then RTI: the local component should be returned
Scenario: InstantiationContext handles missing package_ref with resolver
Given RTI: I have an InstantiationContext without a PackageContentResolver
And RTI: I have a ComponentReference with package_ref
When RTI: I try to resolve the package reference
Then RTI: a TemplateError should be raised about missing resolver
Scenario: TemplateRegistry registers new template types
Given RTI: I have a clean TemplateRegistry
When RTI: I register a template with type TEMPLATE
Then RTI: the template should be registered successfully
And RTI: has_template should return true for TEMPLATE type
Scenario: TemplateRegistry registers SKILL type template
Given RTI: I have a clean TemplateRegistry
When RTI: I register a template with type SKILL
Then RTI: the template should be registered successfully
And RTI: has_template should return true for SKILL type
Scenario: TemplateRegistry registers ACTOR type template
Given RTI: I have a clean TemplateRegistry
When RTI: I register a template with type ACTOR
Then RTI: the template should be registered successfully
And RTI: has_template should return true for ACTOR type
Scenario: TemplateRegistry registers MCP type template
Given RTI: I have a clean TemplateRegistry
When RTI: I register a template with type MCP
Then RTI: the template should be registered successfully
And RTI: has_template should return true for MCP type
Scenario: TemplateRegistry registers LSP type template
Given RTI: I have a clean TemplateRegistry
When RTI: I register a template with type LSP
Then RTI: the template should be registered successfully
And RTI: has_template should return true for LSP type
Scenario: TemplateRegistry instantiate_from_config detects registry reference
Given RTI: I have a TemplateRegistry with PackageContentResolver
And RTI: I have a config with registry-style template reference
When RTI: I instantiate from config with registry reference
Then RTI: the result should include _original_reference
And RTI: the result should contain registry metadata
Scenario: TemplateRegistry instantiate_from_config with local template fallback
Given RTI: I have a clean TemplateRegistry
And RTI: I register a local template named "helper"
And RTI: I have a config referencing the local template
When RTI: I instantiate from config with local template
Then RTI: the local template should be used
And RTI: local template behavior should be unchanged
Scenario: TemplateRegistry list_templates includes new types
Given RTI: I have a clean TemplateRegistry
And RTI: I register templates of all types
When RTI: I list all templates
Then RTI: all eight template types should be included in list
Scenario: Registry reference with _original_reference in component metadata
Given RTI: I have an InstantiationContext with a PackageContentResolver
And RTI: I have a ComponentReference with a local package_ref
When RTI: I resolve the package reference
Then RTI: the resolved metadata should include _original_reference
Scenario: Registry fetch failure produces descriptive error
Given RTI: I have an InstantiationContext with a broken PackageContentResolver
And RTI: I have a ComponentReference with an invalid package_ref
When RTI: I try to resolve the invalid package reference
Then RTI: a TemplateError should be raised with reference info
Scenario: EnhancedTemplateRegistry accepts new template types
Given RTI: I have a clean EnhancedTemplateRegistry
When RTI: I register a template dict with type TEMPLATE in enhanced registry
Then RTI: the enhanced registry should accept the template
Scenario: EnhancedTemplateRegistry register_all_templates with new types
Given RTI: I have a clean EnhancedTemplateRegistry
And RTI: I have a multi-type templates configuration
When RTI: I register all templates in enhanced registry
Then RTI: all eight template types should be registered in enhanced registry
# ── Registry/Enhanced error and edge paths ──
Scenario: TemplateRegistry instantiate_from_config with failed resolve raises TemplateError
Given RTI: I have a clean TemplateRegistry with a mock raising context
When FBF: I instantiate from config with a registry reference that fails to resolve
Then FBF: a TemplateError should be raised about failed resolve
Scenario: TemplateRegistry instantiate_from_config resolved as None raises TemplateError
Given RTI: I have a cleanup TemplateRegistry with a mock returning None context
When FBF: I instantiate from config with a registry reference resolving to None
Then FBF: a TemplateError should be raised about failed resolve
Scenario: TemplateRegistry register_all_templates handles new template types
Given RTI: I have a clean TemplateRegistry
When FBF: I register all templates config with skills actors mcps lsps
Then FBF: TEMPLATE SKILL ACTOR MCP and LSP types should be registered
Scenario: TemplateRegistry _instantiate_from_registry_ref creates resolver when context has none
Given RTI: I have a TemplateRegistry with a context lacking reference_resolver
And FBF: I have a PackageContentResolver mock that returns content
When FBF: I instantiate from config with a registry reference through a resolver-free context
Then FBF: the resolved result should be returned
Scenario: TemplateRegistry handle close_all exception in ad-hoc resolver
Given RTI: I have a TemplateRegistry with a context lacking reference_resolver
And FBF: I have a PackageContentResolver mock that raises on close_all
When FBF: I instantiate from config with resolver-free context and failing close
Then FBF: the resolved result should still be returned despite registry close error
Scenario: EnhancedTemplateRegistry instantiate with registry reference resolves via resolver
Given RTI: I have a clean EnhancedTemplateRegistry with a PackageContentResolver context
When FBF: I instantiate a template with a registry-style name
Then FBF: the result should be the resolved package content
Scenario: EnhancedTemplateRegistry instantiate with None context creates ad-hoc resolver
Given RTI: I have a clean EnhancedTemplateRegistry
And FBF: I have a PackageContentResolver mock that returns content
When FBF: I instantiate with a registry ref and no context
Then FBF: the resolved content should include _original_reference
Scenario: EnhancedTemplateRegistry handle close_all exception in ad-hoc resolver
Given RTI: I have a clean EnhancedTemplateRegistry
And FBF: I have a PackageContentResolver mock that raises on close_all
When FBF: I instantiate with a registry ref and no context with failing close
Then FBF: the resolved content should still be returned despite close error
Scenario: EnhancedTemplateRegistry handle resolve returning None
Given RTI: I have a clean EnhancedTemplateRegistry
And FBF: I have a PackageContentResolver mock that returns None
When FBF: I instantiate with a registry ref and resolver that returns None
Then FBF: a TemplateError should be raised about failed registry resolve
Scenario: EnhancedTemplateRegistry handle resolve raising exception
Given RTI: I have a clean EnhancedTemplateRegistry
And FBF: I have a PackageContentResolver mock that raises on resolve
When FBF: I instantiate with a registry ref and resolver that raises
Then FBF: a TemplateError should be raised about failed registry resolve
# ── InstantiationContext / base / template_store edge paths ──
Scenario: InstantiationContext resolve_reference handles package_ref resolve exception
Given RTI: I have an InstantiationContext with a resolver that raises on resolve
And RTI: I have a ComponentReference with package_ref
When FBF: I try to resolve the ComponentReference
Then FBF: a TemplateError should be raised about failed package resolve
Scenario: InstantiationContext resolve_reference handles resolve returning None
Given RTI: I have an InstantiationContext with a resolver returning None
And RTI: I have a ComponentReference with package_ref
When FBF: I try to resolve the ComponentReference
Then FBF: a TemplateError should be raised about failed package resolve
Scenario: InstantiationContext add_component to existing category assigns correctly
Given RTI: I have an InstantiationContext
When FBF: I add a component to an existing category key
Then FBF: the component should be stored under that key
Scenario: InstantiationContext add_component with new category creates key
Given RTI: I have an InstantiationContext
When FBF: I add a component with a type that does not exist in components
Then FBF: the component should be stored under the new key
Scenario: InstantiationContext _resolve_package_reference sets _original_reference when missing
Given RTI: I have an InstantiationContext with a resolver returning dict without _original_reference
And RTI: I have a ComponentReference with package_ref
When FBF: I resolve the ComponentReference
Then FBF: the result should have _original_reference set from package_ref
Scenario: InstantiationContext _resolve_package_reference guards against None package_ref
Given RTI: I have an InstantiationContext
And FBF: I have a ComponentReference with NO package_ref
When FBF: I directly call _resolve_package_reference on the ComponentReference
Then FBF: the result should be None
Scenario: BaseTemplate abstract instantiate raises NotImplementedError
Given RTI: I have a concrete BaseTemplate subclass without instantiate override
When FBF: I call instantiate on the subclass
Then FBF: a NotImplementedError should be raised
Scenario: TemplateDefinition get_parameters raises on non-dict params
Given RTI: I have a TemplateDefinition with non-dict parameters
When FBF: I get parameters from the definition
Then FBF: a ValueError should be raised about params type
Scenario: TemplateDefinition get_type returns unknown for non-string type
Given RTI: I have a TemplateDefinition with a non-string type field
When FBF: I get the type from the definition
Then FBF: the type should be "unknown"
Scenario: TemplateDefinition instantiate raises on non-dict parsed without templates
Given RTI: I have a TemplateDefinition with non-dict parsed content and no templates
When FBF: I instantiate the definition
Then FBF: a ValueError should be raised about parsed type
# ── _resolve_registry_ref ValueError path ──
Scenario: _resolve_registry_ref raises TemplateError for invalid reference string
Given RTI: I have a clean test environment for registry template integration
When FBF: I call _resolve_registry_ref with an invalid reference string
Then FBF: a TemplateError should be raised about invalid ref
@@ -427,3 +427,45 @@ def step_call_get_package_with_key(context: Any, package_id: str) -> None:
context.result = await context.client.get_package(package_id)
asyncio.get_event_loop().run_until_complete(_call())
@when('I create an insecure RegistryClient pointed at "{base_url}"')
def step_create_insecure_client(context: Any, base_url: str) -> None:
from cleveractors.registry.client import RegistryClient
context.client = RegistryClient(base_url=base_url, allow_insecure=True)
@then("the client should log a warning about cleartext credentials")
def step_client_logs_cleartext_warning(context: Any) -> None:
from cleveractors.registry.client import RegistryClient
with patch("cleveractors.registry.client.logger.warning") as mock_warn:
RegistryClient(base_url="http://example.com", api_key="test-key")
mock_warn.assert_called_once()
assert "cleartext" in mock_warn.call_args[0][0].lower()
@then("the client should log info about HTTPS requirement")
def step_client_logs_https_info(context: Any) -> None:
from cleveractors.registry.client import RegistryClient
with patch("cleveractors.registry.client.logger.info") as mock_info:
RegistryClient(base_url="http://example.com")
mock_info.assert_called_once()
assert "https" in mock_info.call_args[0][0].lower()
@then("the client should not log HTTPS warnings")
def step_client_no_https_warnings(context: Any) -> None:
from cleveractors.registry.client import RegistryClient
with patch("cleveractors.registry.client.logger.warning") as mock_warn:
with patch("cleveractors.registry.client.logger.info") as mock_info:
RegistryClient(base_url="http://example.com", allow_insecure=True)
for call_args in mock_warn.call_args_list:
msg = call_args[0][0].lower()
assert "cleartext" not in msg
for call_args in mock_info.call_args_list:
msg = call_args[0][0].lower()
assert "https" not in msg or "production" not in msg
@@ -0,0 +1,580 @@
"""
Step definitions for Registry Reference Resolver and Generic Template coverage tests.
"""
import asyncio
from unittest.mock import AsyncMock, Mock, patch
from behave import given, then, when
from cleveractors.registry.reference_resolver import (
_MAX_CACHE_SIZE,
PackageContentResolver,
)
from cleveractors.registry.types import PackageReference, ReferenceType
from cleveractors.templates.base import InstantiationContext, TemplateType
from cleveractors.templates.generic_template import GenericTemplate
@given("RRC: I have a clean test environment for reference resolver coverage")
def step_rrc_clean_environment(context):
"""Initialize clean test environment."""
context.resolver = None
context.error = None
context.result = None
context.package_ref = None
context.client = None
@given("RRC: I have a PackageContentResolver instance")
def step_rrc_create_resolver(context):
"""Create a PackageContentResolver instance."""
context.resolver = PackageContentResolver()
@when("RRC: I resolve a LOCAL package reference")
def step_rrc_resolve_local(context):
"""Resolve a LOCAL package reference."""
context.package_ref = PackageReference.from_string("local:my_template.yaml")
try:
context.result = context.resolver.resolve(context.package_ref)
except Exception as e:
context.error = e
@then("RRC: the result should include _original_reference")
def step_rrc_result_has_original_ref(context):
"""Verify _original_reference in result."""
assert context.error is None
assert context.result is not None
assert "_original_reference" in context.result
@then("RRC: the result should have name and type fields")
def step_rrc_result_has_name_type(context):
"""Verify name and type in result."""
assert context.result.get("type") == "local"
assert context.result.get("name") == "my_template.yaml"
@when("RRC: I resolve an ID package reference")
def step_rrc_resolve_id(context):
"""Resolve an ID package reference."""
context.package_ref = PackageReference.from_string(
"ID:pkg_tpl_a1b2c3d4e5f67890abcdef12345678900abcdef0"
)
try:
context.result = context.resolver.resolve(context.package_ref)
except Exception as e:
context.error = e
@then("RRC: the result should include id and type fields")
def step_rrc_result_has_id_type(context):
"""Verify id and type in result."""
assert context.result.get("type") == "id"
assert context.result.get("id") is not None
@then("RRC: _original_reference should match the ID string")
def step_rrc_original_ref_matches(context):
"""Verify _original_reference matches the ID reference."""
assert context.result["_original_reference"] == (
"ID:pkg_tpl_a1b2c3d4e5f67890abcdef12345678900abcdef0"
)
@when("RRC: I resolve a LOCAL reference twice")
def step_rrc_resolve_local_twice(context):
"""Resolve a LOCAL reference twice to test caching."""
context.package_ref = PackageReference.from_string("local:test_cache.yaml")
context.resolver.resolve(context.package_ref)
context.result = context.resolver.resolve(context.package_ref)
@then("RRC: the second resolve should return cached result")
def step_rrc_cached_result(context):
"""Verify second resolve returned cached result."""
assert context.result is not None
assert context.result.get("type") == "local"
@then("RRC: the cache should contain the reference key")
def step_rrc_cache_contains_key(context):
"""Verify cache contains the reference key."""
assert "local:test_cache.yaml" in context.resolver.cache
@when("RRC: I get a client for a server")
def step_rrc_get_client(context):
"""Get a client for a server URL."""
context.first_client = asyncio.run(
context.resolver._get_client("https://registry.example.com")
)
@then("RRC: the client should be stored in the clients dict")
def step_rrc_client_stored(context):
"""Verify client is stored."""
assert "https://registry.example.com" in context.resolver.clients
@then("RRC: getting the same server should return the same client")
def step_rrc_same_client(context):
"""Verify getting same server returns same client."""
second_client = asyncio.run(
context.resolver._get_client("https://registry.example.com")
)
assert second_client is context.first_client
@given("RRC: I have a PackageContentResolver with cached entries")
def step_rrc_resolver_with_cache(context):
"""Create resolver with pre-filled cache."""
context.resolver = PackageContentResolver()
pr = PackageReference.from_string("local:entry1.yaml")
context.resolver.resolve(pr)
assert len(context.resolver.cache) > 0
@when("RRC: I clear the cache")
def step_rrc_clear_cache(context):
"""Clear the resolver cache."""
context.resolver.clear_cache()
@then("RRC: the cache should be empty")
def step_rrc_cache_empty(context):
"""Verify cache is empty."""
assert len(context.resolver.cache) == 0
@given("RRC: I have a GenericTemplate instance with definition")
def step_rrc_generic_template(context):
"""Create a GenericTemplate instance."""
context.generic_template = GenericTemplate(
name="test_tpl",
template_type=TemplateType.TEMPLATE,
definition={"type": "test", "version": "1.0.0", "name": "hello"},
)
@when("RRC: I instantiate the generic template")
def step_rrc_instantiate_generic(context):
"""Instantiate the generic template."""
try:
context.result = context.generic_template.instantiate(
{}, None, InstantiationContext()
)
except Exception as e:
context.error = e
@then("RRC: the result should be the definition dict")
def step_rrc_result_is_definition(context):
"""Verify result is the definition dict."""
assert context.error is None
assert context.result is not None
assert context.result.get("type") == "test"
assert context.result.get("name") == "hello"
@then("RRC: _original_reference should be None by default")
def step_rrc_original_ref_none(context):
"""Verify _original_reference is None for plain template."""
assert context.result.get("_original_reference") is None
@given("RRC: I have a GenericTemplate instance with template variables")
def step_rrc_generic_with_vars(context):
"""Create a GenericTemplate with Jinja2 template variables."""
context.generic_template = GenericTemplate(
name="tpl_vars",
template_type=TemplateType.TEMPLATE,
definition={
"type": "rendered",
"model": "{{ model_name }}",
"temp": "{{ temperature }}",
},
)
@given("RRC: I have instantiation params")
def step_rrc_instantiation_params(context):
"""Set instantiation params."""
context.params = {"model_name": "gpt-4", "temperature": 0.7}
@when("RRC: I instantiate the generic template with params")
def step_rrc_instantiate_generic_with_params(context):
"""Instantiate the generic template with params."""
try:
context.result = context.generic_template.instantiate(
context.params, None, InstantiationContext()
)
except Exception as e:
context.error = e
@then("RRC: the result should contain rendered variables")
def step_rrc_rendered_variables(context):
"""Verify rendered variables in result."""
assert context.error is None
assert context.result.get("model") == "gpt-4"
assert context.result.get("temp") == 0.7
# ── new scenarios: REGISTRY branch, aresolve, close_all, LRU eviction ──
_MOCK_RESPONSE = {"package_id": "pkg_tpl_deadbeef1234", "type": "template"}
@when("RRC: I resolve a REGISTRY reference with a mocked client")
def step_rrc_resolve_registry_mocked(context):
"""Resolve a REGISTRY reference using a mocked client."""
from unittest.mock import MagicMock
mock_client = MagicMock()
mock_client.base_url = "https://reg.example.com"
mock_client.resolve_package = AsyncMock(return_value=_MOCK_RESPONSE)
async def _fake_get_client(server):
return mock_client
with patch.object(context.resolver, "_get_client", side_effect=_fake_get_client):
context.package_ref = PackageReference.from_string(
"reg.example.com:ns/pkg@v1.0.0"
)
context.result = context.resolver.resolve(
context.package_ref, package_type="template"
)
@then("RRC: the result should contain server and namespace fields")
def step_rrc_result_has_registry_fields(context):
"""Verify result has registry metadata fields."""
if context.result is None:
raise AssertionError("context.result is None — aresolve returned None")
if not isinstance(context.result, dict):
raise AssertionError(
f"context.result is {type(context.result).__name__}, not dict"
)
if "_server" not in context.result:
raise AssertionError(
f"Missing _server in keys: {sorted(context.result.keys())}"
)
if "_namespace" not in context.result:
raise AssertionError(
f"Missing _namespace in keys: {sorted(context.result.keys())}"
)
if "_name" not in context.result:
raise AssertionError(f"Missing _name in keys: {sorted(context.result.keys())}")
if "_original_reference" not in context.result:
raise AssertionError(
f"Missing _original_reference in keys: {sorted(context.result.keys())}"
)
assert (
context.result["_original_reference"] == context.package_ref.original_reference
)
@when("RRC: I call aresolve on a REGISTRY reference")
def step_rrc_call_aresolve(context):
"""Call aresolve() on a REGISTRY reference."""
import types
async def _fake_resolve_package(self, package_type, namespace, name, version=None):
return {"package_id": "pkg_skl_abc123", "type": "skill"}
class FakeClient:
pass
FakeClient.resolve_package = _fake_resolve_package
original_get_client = context.resolver._get_client
async def _async_get_client(server):
return FakeClient()
context.resolver._get_client = _async_get_client
try:
async def _do():
context.package_ref = PackageReference.from_string(
"reg2.example.com:ns2/pkg2@latest"
)
return await context.resolver.aresolve(
context.package_ref, package_type="skill"
)
context.result = asyncio.run(_do())
finally:
context.resolver._get_client = original_get_client
@given("RRC: I have a PackageContentResolver with multiple clients")
def step_rrc_resolver_with_clients(context):
"""Create resolver with pre-registered clients."""
from unittest.mock import MagicMock
context.resolver = PackageContentResolver()
client_a = MagicMock()
client_a.base_url = "https://a.example.com"
client_a.close = AsyncMock()
client_b = MagicMock()
client_b.base_url = "https://b.example.com"
client_b.close = AsyncMock()
context.resolver.clients["https://a.example.com"] = client_a
context.resolver.clients["https://b.example.com"] = client_b
context._mock_clients = [client_a, client_b]
@when("RRC: I call close_all synchronously")
def step_rrc_close_all(context):
"""Call close_all synchronously via asyncio.run."""
async def _do():
await context.resolver.close_all()
asyncio.run(_do())
@then("RRC: all client connections should be closed")
def step_rrc_clients_closed(context):
"""Verify all clients were closed."""
assert len(context.resolver.clients) == 0
for client in context._mock_clients:
client.close.assert_called_once()
@when("RRC: I resolve more than max cache size LOCAL references")
def step_rrc_resolve_overflow(context):
"""Resolve more than _MAX_CACHE_SIZE LOCAL references."""
for i in range(_MAX_CACHE_SIZE + 5):
pr = PackageReference.from_string(f"local:cache_test_{i:04d}.yaml")
context.resolver.resolve(pr)
@then("RRC: the cache should not exceed max size")
def step_rrc_cache_not_exceed(context):
"""Verify cache does not exceed max size."""
assert len(context.resolver.cache) <= _MAX_CACHE_SIZE
@then("RRC: the oldest entry should be evicted")
def step_rrc_oldest_evicted(context):
"""Verify oldest entries were evicted."""
cache_keys = list(context.resolver.cache.keys())
assert "local:cache_test_0000.yaml" not in cache_keys
assert "local:cache_test_0001.yaml" not in cache_keys
assert "local:cache_test_0127.yaml" in cache_keys
# ── FBF-prefix aresolve and error-path steps for PackageContentResolver ──
def _fbf_aresolve_sync(resolver, package_ref, package_type=None):
async def _do():
return await resolver.aresolve(package_ref, package_type=package_type)
return asyncio.run(_do())
@when("FBF: I call aresolve on a LOCAL reference")
def step_fbf_aresolve_local(context):
context.package_ref = PackageReference.from_string("local:async_local.yaml")
try:
context.result = _fbf_aresolve_sync(context.resolver, context.package_ref)
except Exception as e:
context.error = e
@then("FBF: the result should include _original_reference")
def step_fbf_result_has_original_ref(context):
assert context.error is None
assert context.result is not None
assert "_original_reference" in context.result
@then("FBF: the result should have name and type fields")
def step_fbf_result_has_name_type(context):
assert context.result.get("type") == "local"
assert context.result.get("name") == "async_local.yaml"
@when("FBF: I call aresolve on an ID reference")
def step_fbf_aresolve_id(context):
context.package_ref = PackageReference.from_string(
"ID:pkg_tpl_ffff0000aaaa11112222bbbbccccdddd33334444"
)
try:
context.result = _fbf_aresolve_sync(context.resolver, context.package_ref)
except Exception as e:
context.error = e
@then("FBF: the result should include id and type fields")
def step_fbf_result_has_id_type(context):
assert context.result.get("type") == "id"
assert context.result.get("id") is not None
@when("FBF: I call aresolve on a LOCAL reference twice")
def step_fbf_aresolve_twice(context):
context.package_ref = PackageReference.from_string("local:async_cache_double.yaml")
_fbf_aresolve_sync(context.resolver, context.package_ref)
context.result = _fbf_aresolve_sync(context.resolver, context.package_ref)
@then("FBF: the second aresolve should return cached result")
def step_fbf_aresolve_cached(context):
assert context.result is not None
assert context.result.get("type") == "local"
assert "local:async_cache_double.yaml" in context.resolver.cache
@when("FBF: I resolve a REGISTRY reference missing server/namespace/name")
def step_fbf_resolve_incomplete_registry(context):
ref = PackageReference(
original_reference="incomplete",
reference_type=ReferenceType.REGISTRY,
server=None,
namespace=None,
name="",
)
context.result = context.resolver.resolve(ref, package_type="template")
@then("FBF: the result should be None")
def step_fbf_result_is_none(context):
assert context.result is None
@when("FBF: I resolve a REGISTRY reference without package_type")
def step_fbf_resolve_no_package_type(context):
ref = PackageReference(
original_reference="reg.example.com:ns/mypkg@v1.0.0",
reference_type=ReferenceType.REGISTRY,
server="reg.example.com",
namespace="ns",
name="mypkg",
version="v1.0.0",
package_type=None,
)
context.result = context.resolver.resolve(ref)
@when("FBF: I call aresolve on a REGISTRY reference missing server/namespace/name")
def step_fbf_aresolve_incomplete(context):
ref = PackageReference(
original_reference="bad:ref",
reference_type=ReferenceType.REGISTRY,
server=None,
namespace=None,
name="",
)
context.result = _fbf_aresolve_sync(context.resolver, ref, package_type="template")
@then("FBF: the aresolve result should be None")
def step_fbf_aresolve_none(context):
assert context.result is None
@when("FBF: I call aresolve on a REGISTRY reference without package_type")
def step_fbf_aresolve_no_package_type(context):
ref = PackageReference(
original_reference="reg.example.com:ns/p2@v1.0.0",
reference_type=ReferenceType.REGISTRY,
server="reg.example.com",
namespace="ns",
name="p2",
version="v1.0.0",
package_type=None,
)
context.result = _fbf_aresolve_sync(context.resolver, ref)
@when("FBF: I try to resolve a REGISTRY reference from inside a running event loop")
def step_fbf_resolve_inside_loop(context):
async def _inner():
ref = PackageReference.from_string("reg.example.com:ns/pkg@v1.0.0")
r = PackageContentResolver()
return r.resolve(ref, package_type="template")
try:
asyncio.run(_inner())
except Exception as e:
context.error = e
@then("FBF: a RuntimeError should be raised about event loop conflict")
def step_fbf_loop_runtime_error(context):
assert context.error is not None
assert isinstance(context.error, RuntimeError)
assert "running event loop" in str(context.error)
@given("RRC: I have a PackageContentResolver with clients where one raises on close")
def step_rrc_resolver_with_faulty_clients(context):
context.resolver = PackageContentResolver()
client_good = Mock()
client_good.base_url = "https://good.example.com"
client_good.close = AsyncMock()
client_bad = Mock()
client_bad.base_url = "https://bad.example.com"
client_bad.close = AsyncMock(side_effect=RuntimeError("connection lost"))
context.resolver.clients["https://good.example.com"] = client_good
context.resolver.clients["https://bad.example.com"] = client_bad
context._mock_clients = [client_good, client_bad]
@when("FBF: I call close_all")
def step_fbf_close_all(context):
async def _do():
await context.resolver.close_all()
asyncio.run(_do())
@then("FBF: all clients should be closed without raising")
def step_fbf_clients_closed_no_raise(context):
assert len(context.resolver.clients) == 0
for client in context._mock_clients:
client.close.assert_called_once()
@when("FBF: I resolve a reference with an unknown reference type")
def step_fbf_resolve_unknown_type(context):
ref = PackageReference(
original_reference="bogus:ref",
reference_type="__unknown__",
)
context.result = context.resolver.resolve(ref)
@when("FBF: I call _put_cache with an already existing key")
def step_fbf_put_cache_existing(context):
context.resolver._put_cache("existing_key", {"val": 1})
context.resolver._put_cache("existing_key", {"val": 2})
@then("FBF: the key should be moved to the end of the cache")
def step_fbf_key_moved_to_end(context):
assert "existing_key" in context.resolver.cache
keys = list(context.resolver.cache.keys())
assert keys[-1] == "existing_key"
assert context.resolver.cache["existing_key"] == {"val": 2}
@when("FBF: I call aresolve on a reference with an unknown reference type")
def step_fbf_aresolve_unknown_type(context):
ref = PackageReference(
original_reference="bogus_async:ref",
reference_type="__unknown__",
)
context.result = _fbf_aresolve_sync(context.resolver, ref)
+12 -3
View File
@@ -709,7 +709,7 @@ def step_verify_registry_initialized_correctly(context):
"""Verify registry initialized correctly."""
assert context.registry is not None
assert hasattr(context.registry, "templates")
assert len(context.registry.templates) == 3 # AGENT, GRAPH, STREAM
assert len(context.registry.templates) == 8 # all 8 template types
@then("all template type collections should be empty")
@@ -997,8 +997,17 @@ def step_verify_all_template_types_present_but_empty(context):
"""Verify all template types present but empty."""
assert context.error is None
assert context.result is not None
assert len(context.result) == 3
for template_type in ["agent", "graph", "stream"]:
assert len(context.result) == 8
for template_type in [
"agent",
"graph",
"stream",
"template",
"skill",
"actor",
"mcp",
"lsp",
]:
assert template_type in context.result
assert len(context.result[template_type]) == 0
File diff suppressed because it is too large Load Diff
@@ -36,22 +36,30 @@ def step_template_store_initialized_empty(context):
assert isinstance(context.template_store.raw_templates, dict)
assert isinstance(context.template_store.metadata, dict)
# Check all template types are empty
for template_type in ["agents", "graphs", "streams"]:
for template_type in [
"agents",
"graphs",
"streams",
"templates",
"skills",
"actors",
"mcps",
"lsps",
]:
assert template_type in context.template_store.raw_templates
assert template_type in context.template_store.metadata
assert len(context.template_store.raw_templates[template_type]) == 0
assert len(context.template_store.metadata[template_type]) == 0
@then("the template store should have three template types")
def step_template_store_has_three_types(context):
"""Verify template store has three template types."""
assert len(context.template_store.raw_templates) == 3
assert len(context.template_store.metadata) == 3
@then("the template store should have eight template types")
def step_template_store_has_eight_types(context):
"""Verify template store has eight template types."""
assert len(context.template_store.raw_templates) == 8
assert len(context.template_store.metadata) == 8
assert "agents" in context.template_store.raw_templates
assert "graphs" in context.template_store.raw_templates
assert "streams" in context.template_store.raw_templates
assert "templates" in context.template_store.raw_templates
assert "skills" in context.template_store.raw_templates
@given("I have a TemplateStore")
+1 -1
View File
@@ -9,7 +9,7 @@ Feature: Template Store Persistence and Instantiation
Scenario: Test TemplateStore initialization
When I create a new TemplateStore
Then the template store should be initialized with empty collections
And the template store should have three template types
And the template store should have eight template types
Scenario: Test add_template with string definition
Given I have a TemplateStore
+233
View File
@@ -33,6 +33,13 @@ from cleveractors.core.progress import ProgressBarManager
from cleveractors.templates.enhanced_registry import EnhancedTemplateRegistry
from cleveractors.templates.registry import TemplateRegistry
from cleveractors.templates.renderer import TemplateEngine, TemplateRenderer
from cleveractors.templates.base import (
ComponentReference,
InstantiationContext,
TemplateType,
)
from cleveractors.registry.reference_resolver import PackageContentResolver
from cleveractors.registry.types import PackageReference
class CleverActorsLib: # pragma: no cover - integration test library
@@ -500,3 +507,229 @@ class CleverActorsLib: # pragma: no cover - integration test library
raise AssertionError(
f"Jinja2 render: expected containing {expected!r}, got {result!r}"
)
# ── template type enumeration (issue #27) ──────────────────────────
def template_type_count_equals(self, expected: str) -> None:
count = len(TemplateType)
if count != int(expected):
raise AssertionError(
f"Expected {expected} TemplateType values, got {count}"
)
def template_type_value_equals(self, name: str, expected: str) -> None:
value = TemplateType[name].value
if value != expected:
raise AssertionError(
f"TemplateType.{name}: expected {expected!r}, got {value!r}"
)
def template_type_has_member(self, name: str) -> None:
if name not in TemplateType.__members__:
raise AssertionError(
f"TemplateType missing member {name!r}: {list(TemplateType.__members__)}"
)
# ── component reference with package_ref (issue #27) ───────────────
def create_component_reference(
self, ref_type: str, ref_name: str, ref_params_json: str = "{}"
) -> None:
params = json.loads(ref_params_json)
self._comp_ref = ComponentReference(
ref_type=ref_type, ref_name=ref_name, ref_params=params
)
def component_ref_field_equals(self, field: str, expected: str) -> None:
value = getattr(self._comp_ref, field)
if str(value) != expected:
raise AssertionError(
f"ComponentReference.{field}: expected {expected!r}, got {value!r}"
)
def component_ref_package_ref_is_none(self) -> None:
if self._comp_ref.package_ref is not None:
raise AssertionError(
f"Expected package_ref to be None, got {self._comp_ref.package_ref}"
)
def set_component_ref_package_ref(self, ref_string: str) -> None:
self._comp_ref.package_ref = PackageReference.from_string(ref_string)
def component_ref_package_ref_has_attr(self, attr: str, expected: str) -> None:
pr = self._comp_ref.package_ref
if pr is None:
raise AssertionError("package_ref is None")
value = getattr(pr, attr)
if str(value) != expected:
raise AssertionError(
f"package_ref.{attr}: expected {expected!r}, got {value!r}"
)
# ── template registry with 8 types (issue #27) ─────────────────────
def register_template_by_type_name(
self, type_name: str, template_name: str, definition_json: str
) -> None:
ttype = TemplateType[type_name]
definition = json.loads(definition_json)
self._registry.register_template(ttype, template_name, definition)
def template_registry_has_template(
self, type_name: str, template_name: str
) -> None:
ttype = TemplateType[type_name]
if not self._registry.has_template(ttype, template_name):
raise AssertionError(f"Registry does not have {type_name}/{template_name}")
def template_registry_count_for_type(self, type_name: str, expected: str) -> None:
ttype = TemplateType[type_name]
count = len(self._registry.templates[ttype])
if count != int(expected):
raise AssertionError(
f"Expected {expected} templates for {type_name}, got {count}"
)
def template_registry_list_all_count(self, expected: str) -> None:
result = self._registry.list_templates()
if len(result) != int(expected):
raise AssertionError(
f"Expected {expected} template types in list, got {len(result)}: {sorted(result.keys())}"
)
# ── enhanced registry with 8 types (issue #27) ─────────────────────
def register_enhanced_template_by_type_name(
self, type_name: str, template_name: str, definition_json: str
) -> None:
ttype = TemplateType[type_name]
definition = json.loads(definition_json)
self._reg_enhanced.register_template_dict(ttype, template_name, definition)
def enhanced_registry_has_template(
self, type_name: str, template_name: str
) -> None:
ttype = TemplateType[type_name]
if not self._reg_enhanced.has_template(ttype, template_name):
raise AssertionError(
f"Enhanced registry does not have {type_name}/{template_name}"
)
# ── reference resolver (issue #27) ─────────────────────────────────
def create_reference_resolver(self) -> None:
self._ref_resolver = PackageContentResolver()
def resolve_reference(self, ref_string: str) -> None:
pr = PackageReference.from_string(ref_string)
self._resolved = self._ref_resolver.resolve(pr)
def resolved_has_key(self, key: str) -> None:
if key not in self._resolved:
raise AssertionError(
f"Resolved result missing key {key!r}: {sorted(self._resolved.keys())}"
)
def resolved_key_value_equals(self, key: str, expected: str) -> None:
value = str(self._resolved.get(key))
if value != expected:
raise AssertionError(
f"Resolved[{key}]: expected {expected!r}, got {value!r}"
)
def resolve_should_return_none(self, ref_string: str) -> None:
try:
pr = PackageReference.from_string(ref_string)
result = self._ref_resolver.resolve(pr)
if result is not None:
raise AssertionError(
f"Expected None for {ref_string!r}, got {result!r}"
)
except (ValueError, Exception):
return
def resolver_cache_contains(self, ref_string: str) -> None:
if ref_string not in self._ref_resolver.cache:
raise AssertionError(
f"Cache missing {ref_string!r}: {sorted(self._ref_resolver.cache.keys())}"
)
def resolver_client_pool_has_server(self, server: str) -> None:
if server not in self._ref_resolver.clients:
raise AssertionError(
f"Client pool missing {server!r}: {sorted(self._ref_resolver.clients.keys())}"
)
def create_ref_resolver_client_for_server(self, server: str) -> None:
import asyncio
asyncio.run(self._ref_resolver._get_client(server))
def resolver_clear_cache(self) -> None:
self._ref_resolver.clear_cache()
def resolver_cache_is_empty(self) -> None:
if len(self._ref_resolver.cache) != 0:
raise AssertionError(
f"Expected empty cache, got {len(self._ref_resolver.cache)} entries"
)
# ── instantiation context with registry awareness (issue #27) ──────
def create_context_with_resolver(self) -> None:
self._ref_resolver = PackageContentResolver()
self._reg_ctx = InstantiationContext(reference_resolver=self._ref_resolver)
def context_has_component_type(self, comp_type: str) -> None:
plural = f"{comp_type}s"
if plural not in self._reg_ctx.components:
raise AssertionError(
f"Context missing component type {plural!r}: {sorted(self._reg_ctx.components.keys())}"
)
def context_component_count(self, comp_type: str, expected: str) -> None:
plural = f"{comp_type}s"
count = len(self._reg_ctx.components.get(plural, {}))
if count != int(expected):
raise AssertionError(
f"Expected {expected} components for {plural}, got {count}"
)
# ── _original_reference propagation (issue #27) ────────────────────
def resolve_local_ref_with_original_reference(self, ref_string: str) -> None:
pr = PackageReference.from_string(ref_string)
self._ref_resolver = PackageContentResolver()
result = self._ref_resolver.resolve(pr)
self._resolved = result
def resolved_original_reference_equals(self, expected: str) -> None:
value = self._resolved.get("_original_reference")
if str(value) != expected:
raise AssertionError(
f"_original_reference: expected {expected!r}, got {value!r}"
)
# ── template registry ref detection (issue #27) ────────────────────
def instantiate_config_with_ref_and_resolver(
self, template_ref: str, params_json: str = "{}"
) -> None:
params = json.loads(params_json)
self._ref_resolver = PackageContentResolver()
ctx = InstantiationContext(reference_resolver=self._ref_resolver)
config = {"template": template_ref, "params": params}
self._reg_instantiated = self._registry.instantiate_from_config(config, ctx)
def instantiated_has_key(self, key: str) -> None:
if key not in self._reg_instantiated:
raise AssertionError(
f"Instantiated result missing key {key!r}: {sorted(self._reg_instantiated.keys())}"
)
def instantiated_key_value_equals(self, key: str, expected: str) -> None:
value = str(self._reg_instantiated.get(key))
if value != expected:
raise AssertionError(
f"Instantiated[{key}]: expected {expected!r}, got {value!r}"
)
+37 -2
View File
@@ -39,6 +39,15 @@ class FakeRegistryHandler(SimpleHTTPRequestHandler):
"version_count": 1,
"created_at": "2026-02-01T00:00:00Z",
},
{
"id": "pkg_tpl_1111111111111111111111111111111111111111",
"type": "template",
"name": "test-template",
"description": "A test template package",
"namespace": "example",
"version_count": 1,
"created_at": "2026-03-01T00:00:00Z",
},
]
_ACTOR_VERSIONS: dict[str, str] = {
@@ -114,7 +123,16 @@ class FakeRegistryHandler(SimpleHTTPRequestHandler):
{
"name": "Fake Package Registry",
"version": "1.0.0",
"supported_types": ["actor", "graph", "stream", "agent"],
"supported_types": [
"actor",
"graph",
"stream",
"agent",
"template",
"skill",
"mcp",
"lsp",
],
"authentication": ["anonymous"],
"features": ["semantic_versioning", "mutable_aliases"],
},
@@ -129,9 +147,26 @@ class FakeRegistryHandler(SimpleHTTPRequestHandler):
"package_id": package_id,
},
)
elif package_id.startswith("pkg_tpl_"):
self._json(
200,
{
"content": "name: test-template\ntype: template",
"package_id": package_id,
},
)
else:
self._json(404, {"message": "Package not found"})
elif path.startswith("/actor/") or path.startswith("/graph/"):
elif (
path.startswith("/actor/")
or path.startswith("/graph/")
or path.startswith("/stream/")
or path.startswith("/agent/")
or path.startswith("/template/")
or path.startswith("/skill/")
or path.startswith("/mcp/")
or path.startswith("/lsp/")
):
parts = path.strip("/").split("/")
if len(parts) >= 3:
ver = params.get("version", ["v1.0.0"])[0]
+208
View File
@@ -0,0 +1,208 @@
*** Settings ***
Documentation Integration tests for issue #27: TemplateType extension
... and PackageReference integration into the template system.
... Validates end-to-end: TemplateType enum, ComponentReference
... with package_ref, TemplateRegistry/EnhancedTemplateRegistry
... with 8 types, PackageContentResolver, _original_reference,
... and registry-aware InstantiationContext.
Library OperatingSystem
Library CleverActorsLib.py
*** Test Cases ***
TemplateType Enum Has All 8 Values
[Documentation] Verify TemplateType contains all 8 members.
Template Type Count Equals 8
Template Type Has Member AGENT
Template Type Has Member GRAPH
Template Type Has Member STREAM
Template Type Has Member TEMPLATE
Template Type Has Member SKILL
Template Type Has Member ACTOR
Template Type Has Member MCP
Template Type Has Member LSP
TemplateType Values Match Standard
[Documentation] Verify each TemplateType enum value is correct.
Template Type Value Equals AGENT agent
Template Type Value Equals GRAPH graph
Template Type Value Equals STREAM stream
Template Type Value Equals TEMPLATE template
Template Type Value Equals SKILL skill
Template Type Value Equals ACTOR actor
Template Type Value Equals MCP mcp
Template Type Value Equals LSP lsp
ComponentReference Created Without PackageRef
[Documentation] ComponentReference with no package_ref has it as None.
Create Component Reference template my_tpl
Component Ref Field Equals ref_type template
Component Ref Field Equals ref_name my_tpl
Component Ref Package Ref Is None
ComponentReference With PackageRef Set
[Documentation] ComponentReference can carry a parsed PackageReference.
Create Component Reference template helper
Set Component Ref Package Ref git.cleverthis.com:acme/helper@v1.0.0
Component Ref Field Equals ref_type template
Component Ref Field Equals ref_name helper
Component Ref Package Ref Has Attr server git.cleverthis.com
Component Ref Package Ref Has Attr namespace acme
Component Ref Package Ref Has Attr name helper
Component Ref Package Ref Has Attr version v1.0.0
TemplateRegistry Registers New Agent Template
[Documentation] Existing AGENT type still registers correctly.
Create Template Registry
Register Template By Type Name AGENT test_agent {"type": "llm", "version": "1.0"}
Template Registry Has Template AGENT test_agent
Template Registry Count For Type AGENT 1
TemplateRegistry Registers New TEMPLATE Type
[Documentation] New TEMPLATE type registers and has_template works.
Create Template Registry
Register Template By Type Name TEMPLATE my_template {"type": "tpl", "version": "2.0"}
Template Registry Has Template TEMPLATE my_template
Template Registry Count For Type TEMPLATE 1
TemplateRegistry Registers New SKILL Type
[Documentation] New SKILL type registers and has_template works.
Create Template Registry
Register Template By Type Name SKILL my_skill {"type": "skill_def", "tools": ["echo"]}
Template Registry Has Template SKILL my_skill
Template Registry Count For Type SKILL 1
TemplateRegistry Registers New ACTOR Type
[Documentation] New ACTOR type registers and has_template works.
Create Template Registry
Register Template By Type Name ACTOR my_actor {"type": "actor_def", "name": "test"}
Template Registry Has Template ACTOR my_actor
Template Registry Count For Type ACTOR 1
TemplateRegistry Registers New MCP Type
[Documentation] New MCP type registers and has_template works.
Create Template Registry
Register Template By Type Name MCP my_mcp {"type": "mcp_def", "server": "test"}
Template Registry Has Template MCP my_mcp
Template Registry Count For Type MCP 1
TemplateRegistry Registers New LSP Type
[Documentation] New LSP type registers and has_template works.
Create Template Registry
Register Template By Type Name LSP my_lsp {"type": "lsp_def", "language": "python"}
Template Registry Has Template LSP my_lsp
Template Registry Count For Type LSP 1
TemplateRegistry List All Returns 8 Types
[Documentation] list_templates returns 8 keys when all types added.
Create Template Registry
Register Template By Type Name AGENT a1 {"type": "agent"}
Register Template By Type Name GRAPH g1 {"type": "graph"}
Register Template By Type Name STREAM s1 {"type": "stream"}
Register Template By Type Name TEMPLATE t1 {"type": "tpl"}
Register Template By Type Name SKILL k1 {"type": "skl"}
Register Template By Type Name ACTOR c1 {"type": "act"}
Register Template By Type Name MCP m1 {"type": "mcp"}
Register Template By Type Name LSP l1 {"type": "lsp"}
Template Registry List All Count 8
EnhancedTemplateRegistry Registers New TEMPLATE Type
[Documentation] Enhanced registry accepts the new TEMPLATE type.
Create Enhanced Registry
Register Enhanced Template By Type Name TEMPLATE enh_tpl {"type": "enhanced_template", "version": "3.0"}
Enhanced Registry Has Template TEMPLATE enh_tpl
EnhancedTemplateRegistry Registers New SKILL Type
[Documentation] Enhanced registry accepts the new SKILL type.
Create Enhanced Registry
Register Enhanced Template By Type Name SKILL enh_skill {"type": "enhanced_skill", "tools": ["search"]}
Enhanced Registry Has Template SKILL enh_skill
EnhancedTemplateRegistry Registers New ACTOR Type
[Documentation] Enhanced registry accepts the new ACTOR type.
Create Enhanced Registry
Register Enhanced Template By Type Name ACTOR enh_actor {"type": "enhanced_actor", "name": "actor1"}
Enhanced Registry Has Template ACTOR enh_actor
EnhancedTemplateRegistry Registers New MCP Type
[Documentation] Enhanced registry accepts the new MCP type.
Create Enhanced Registry
Register Enhanced Template By Type Name MCP enh_mcp {"type": "enhanced_mcp", "server": "example"}
Enhanced Registry Has Template MCP enh_mcp
EnhancedTemplateRegistry Registers New LSP Type
[Documentation] Enhanced registry accepts the new LSP type.
Create Enhanced Registry
Register Enhanced Template By Type Name LSP enh_lsp {"type": "enhanced_lsp", "language": "python"}
Enhanced Registry Has Template LSP enh_lsp
PackageContentResolver Resolves LOCAL Reference
[Documentation] LOCAL references resolve to dict with name and type.
Create Reference Resolver
Resolve Reference local:test/file.yaml
Resolved Has Key _original_reference
Resolved Has Key type
Resolved Has Key name
Resolved Key Value Equals type local
Resolved Key Value Equals name test/file.yaml
PackageContentResolver Resolves ID Reference
[Documentation] ID references resolve to dict with id and type.
Create Reference Resolver
Resolve Reference ID:pkg_tpl_a1b2c3d4e5f67890abcdef12345678900abcdef0
Resolved Has Key _original_reference
Resolved Has Key id
Resolved Has Key type
Resolved Key Value Equals type id
PackageContentResolver Caches Resolved References
[Documentation] Resolving the same reference twice returns cached result.
Create Reference Resolver
Resolve Reference local:cache_test.yaml
Resolve Reference local:cache_test.yaml
Resolver Cache Contains local:cache_test.yaml
PackageContentResolver Empty Cache
[Documentation] clear_cache removes all cached entries.
Create Reference Resolver
Resolve Reference local:to_clear.yaml
Resolver Clear Cache
Resolver Cache Is Empty
PackageContentResolver Incomplete Registry Ref Returns None
[Documentation] A reference without namespace/name returns None.
Create Reference Resolver
Resolve Should Return None bad.server.com:/
PackageContentResolver Client Pool Stores Clients
[Documentation] Creating a client for a server stores it in the client pool.
Create Reference Resolver
Create Ref Resolver Client For Server https://test.example.com
Resolver Client Pool Has Server https://test.example.com
Original Reference Propagated Through Resolution
[Documentation] _original_reference matches the input reference string.
Resolve Local Ref With Original Reference local:my/package.yaml
Resolved Original Reference Equals local:my/package.yaml
InstantiationContext Has All 8 Component Types
[Documentation] InstantiationContext.components dict has 8 type keys.
Create Context With Resolver
Context Has Component Type agent
Context Has Component Type graph
Context Has Component Type stream
Context Has Component Type template
Context Has Component Type skill
Context Has Component Type actor
Context Has Component Type mcp
Context Has Component Type lsp
Context Component Count agent 0
Context Component Count mcp 0
TemplateRegistry Local Template Resolves Unchanged Through InstantiationFromConfig
[Documentation] instantiate_from_config resolves a locally-registered template.
Create Template Registry
Register Template By Type Name TEMPLATE local_test_tpl {"type": "test_tpl", "name": "integration_test"}
Instantiate Config With Ref And Resolver local_test_tpl
Instantiated Has Key type
Instantiated Has Key name
Instantiated Key Value Equals name integration_test
+21 -9
View File
@@ -834,19 +834,16 @@ class ReactiveCleverAgentsApp: # pylint: disable=too-many-instance-attributes
if not isinstance(templates, dict):
continue
for name, template_def in templates.items():
t_type = self._section_to_template_type(template_type)
if t_type is None:
self.logger.warning(
"Unknown template section '%s', skipping", template_type
)
continue
if isinstance(template_def, dict) and template_def.get(
"_needs_preprocessing"
):
# Raw template string
t_type = (
TemplateType.AGENT
if template_type == "agents"
else (
TemplateType.GRAPH
if template_type == "graphs"
else TemplateType.STREAM
)
)
self.template_registry.register_template_string(
t_type, name, template_def["_raw_template"]
)
@@ -897,6 +894,21 @@ class ReactiveCleverAgentsApp: # pylint: disable=too-many-instance-attributes
stream_count,
)
@staticmethod
def _section_to_template_type(section: str) -> Optional[TemplateType]:
"""Map a plural template-section name to its TemplateType."""
mapping: dict[str, TemplateType] = {
"agents": TemplateType.AGENT,
"graphs": TemplateType.GRAPH,
"streams": TemplateType.STREAM,
"templates": TemplateType.TEMPLATE,
"skills": TemplateType.SKILL,
"actors": TemplateType.ACTOR,
"mcps": TemplateType.MCP,
"lsps": TemplateType.LSP,
}
return mapping.get(section)
def _create_agents(self) -> None:
"""Create all configured agents."""
if not self.agent_factory or not self.config:
+3 -1
View File
@@ -15,6 +15,7 @@ from cleveractors.registry.exceptions import (
ValidationError,
VersionNotFoundError,
)
from cleveractors.registry.reference_resolver import PackageContentResolver
from cleveractors.registry.resolver import (
ReferenceResolver,
is_concrete_version,
@@ -31,6 +32,8 @@ from cleveractors.registry.types import (
__all__ = [
"Canonicalizer",
"PackageContentResolver",
"ReferenceResolver",
"RegistryClient",
"RegistryError",
"PackageNotFoundError",
@@ -48,7 +51,6 @@ __all__ = [
"PackageReference",
"PackageType",
"ReferenceType",
"ReferenceResolver",
"resolve_version",
"is_concrete_version",
"is_version_alias",
+20 -1
View File
@@ -14,6 +14,7 @@ from __future__ import annotations
import logging
from typing import Any, Optional
from urllib.parse import quote
import httpx
@@ -43,6 +44,7 @@ class RegistryClient:
base_url: str,
api_key: Optional[str] = None,
timeout: float = 30.0,
allow_insecure: bool = False,
) -> None:
if not base_url:
raise ValueError("base_url must not be empty")
@@ -50,6 +52,19 @@ class RegistryClient:
self.api_key: Optional[str] = api_key
self.timeout: float = timeout
self._client: Optional[httpx.AsyncClient] = None
if not allow_insecure and not self.base_url.startswith("https://"):
if api_key is not None:
logger.warning(
"RegistryClient using HTTP with an API key — "
"the key may be transmitted in cleartext. "
"Use HTTPS or pass allow_insecure=True to suppress this warning."
)
else:
logger.info(
"RegistryClient using HTTP without TLS. "
"The Package Registry Standard v1.0.0 requires HTTPS "
"in production (§12.2)."
)
async def _get_client(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
@@ -159,7 +174,11 @@ class RegistryClient:
params: dict[str, str] = {}
if version:
params["version"] = version
return await self._request(f"/{package_type}/{namespace}/{name}", params=params)
encoded_ns = quote(namespace, safe="")
encoded_name = quote(name, safe="")
return await self._request(
f"/{package_type}/{encoded_ns}/{encoded_name}", params=params
)
# ------------------------------------------------------------------
# §8.4.1 GET /browse
@@ -0,0 +1,339 @@
"""Reference resolver with multi-server RegistryClient pool.
Resolves PackageReference objects to content dictionaries by maintaining
one RegistryClient per server URL, with caching of fetched packages.
"""
from __future__ import annotations
import asyncio
import copy
import logging
import threading
from collections import OrderedDict
from typing import Any, Optional
from cleveractors.registry.client import RegistryClient
from cleveractors.registry.types import PackageReference, ReferenceType
logger = logging.getLogger(__name__)
_MAX_CACHE_SIZE = 128
# Maps TemplateType.value to PackageType.value (e.g. "template" → "tpl")
_TEMPLATE_TYPE_TO_PACKAGE_TYPE: dict[str, str] = {
"agent": "agt",
"graph": "grh",
"stream": "str",
"template": "tpl",
"skill": "skl",
"actor": "act",
"mcp": "mcp",
"lsp": "lsp",
}
class PackageContentResolver:
"""Resolves package references using a pool of RegistryClient instances.
Maintains one ``RegistryClient`` per server URL, reusing connections.
Caches fetched package content by reference string and package type.
The cache is an LRU with a maximum of 128 entries. Cache operations
are synchronised ``threading.Lock`` guards synchronous code paths
and ``asyncio.Lock`` guards asynchronous code paths. The two locks
are independent; mixed sync/async concurrent use of a single instance
is not supported.
The async lock is lazily initialised to avoid creating an
``asyncio.Lock`` outside a running event loop.
Attributes:
clients: Map of server URL to RegistryClient.
cache: Cache of resolved content dicts keyed by reference+type.
api_key: Optional API key for authenticated operations.
"""
def __init__(self, api_key: Optional[str] = None) -> None:
self.clients: dict[str, RegistryClient] = {}
self.cache: OrderedDict[str, dict[str, Any]] = OrderedDict()
self.api_key: Optional[str] = api_key
self._lock: threading.Lock = threading.Lock()
self._async_lock: Optional[asyncio.Lock] = None
def _get_async_lock(self) -> asyncio.Lock:
if self._async_lock is None:
self._async_lock = asyncio.Lock()
return self._async_lock
async def _get_client(self, server: str) -> RegistryClient:
async with self._get_async_lock():
if server in self.clients:
return self.clients[server]
client = RegistryClient(base_url=server, api_key=self.api_key)
self.clients[server] = client
return client
def resolve(
self,
ref: PackageReference,
package_type: Optional[str] = None,
) -> Optional[dict[str, Any]]:
"""Synchronous resolve entry point for template usage.
For registry references, runs the async fetch in a new event loop.
For local and ID references, returns a simple placeholder dict.
Args:
ref: A parsed PackageReference.
package_type: Package type code (e.g. ``\"tpl\"``) for
REGISTRY references. Required when ref type is REGISTRY
and the type is known. Maps ``TemplateType.value`` to
``PackageType.value`` automatically.
Returns:
A dict with fetched content and ``_original_reference``, or None.
Raises:
RuntimeError: If called from an already-running event loop.
Use :meth:`aresolve` instead.
"""
mapped_type = (
_TEMPLATE_TYPE_TO_PACKAGE_TYPE.get(package_type, package_type)
if package_type and ref.reference_type == ReferenceType.REGISTRY
else None
)
cache_key = (
f"{ref.original_reference}:{mapped_type}"
if mapped_type
else ref.original_reference
)
with self._lock:
if cache_key in self.cache:
logger.debug("Returning cached result for %s", cache_key)
self.cache.move_to_end(cache_key)
return copy.deepcopy(self.cache[cache_key])
if ref.reference_type == ReferenceType.LOCAL:
result: dict[str, Any] = {
"name": ref.name,
"type": "local",
"_original_reference": ref.original_reference,
}
with self._lock:
self._put_cache(cache_key, result)
return copy.deepcopy(result)
if ref.reference_type == ReferenceType.ID:
result = {
"id": ref.id_string,
"type": "id",
"_original_reference": ref.original_reference,
}
with self._lock:
self._put_cache(cache_key, result)
return copy.deepcopy(result)
if ref.reference_type == ReferenceType.REGISTRY:
if not ref.server or not ref.namespace or not ref.name:
logger.warning("Incomplete registry reference: %s", ref)
return None
pkg_type = package_type or ref.package_type
if pkg_type is None:
logger.error(
"Cannot resolve REGISTRY reference %s without package_type; "
"provide package_type=... or set ref.package_type",
ref,
)
return None
resolved_type = _TEMPLATE_TYPE_TO_PACKAGE_TYPE.get(pkg_type, pkg_type)
try:
asyncio.get_running_loop()
except RuntimeError:
return asyncio.run(
self._resolve_registry_async(
ref.server,
ref.namespace,
ref.name,
ref.version or "latest",
cache_key,
resolved_type,
ref.original_reference,
)
)
raise RuntimeError(
f"asyncio.run() cannot be called from a running event loop. "
f"Use await resolver.aresolve({ref!r}, package_type={pkg_type!r}) instead."
)
return None
async def aresolve(
self,
ref: PackageReference,
package_type: Optional[str] = None,
) -> Optional[dict[str, Any]]:
"""Async entry point — preferred when running inside an event loop.
Args:
ref: A parsed PackageReference.
package_type: Package type code for REGISTRY references.
Returns:
A dict with fetched content and ``_original_reference``, or None.
"""
mapped_type = (
_TEMPLATE_TYPE_TO_PACKAGE_TYPE.get(package_type, package_type)
if package_type and ref.reference_type == ReferenceType.REGISTRY
else None
)
cache_key = (
f"{ref.original_reference}:{mapped_type}"
if mapped_type
else ref.original_reference
)
async with self._get_async_lock():
if cache_key in self.cache:
logger.debug("Returning cached result for %s", cache_key)
self.cache.move_to_end(cache_key)
return copy.deepcopy(self.cache[cache_key])
if ref.reference_type == ReferenceType.LOCAL:
result: dict[str, Any] = {
"name": ref.name,
"type": "local",
"_original_reference": ref.original_reference,
}
async with self._get_async_lock():
self._put_cache(cache_key, result)
return copy.deepcopy(result)
if ref.reference_type == ReferenceType.ID:
result = {
"id": ref.id_string,
"type": "id",
"_original_reference": ref.original_reference,
}
async with self._get_async_lock():
self._put_cache(cache_key, result)
return copy.deepcopy(result)
if ref.reference_type == ReferenceType.REGISTRY:
if not ref.server or not ref.namespace or not ref.name:
logger.warning("Incomplete registry reference: %s", ref)
return None
pkg_type = package_type or ref.package_type
if pkg_type is None:
logger.error(
"Cannot resolve REGISTRY reference %s without package_type; "
"provide package_type=... or set ref.package_type",
ref,
)
return None
resolved_type = _TEMPLATE_TYPE_TO_PACKAGE_TYPE.get(pkg_type, pkg_type)
return await self._resolve_registry_async(
ref.server,
ref.namespace,
ref.name,
ref.version or "latest",
cache_key,
resolved_type,
ref.original_reference,
)
return None
def _put_cache(self, key: str, value: dict[str, Any]) -> None:
"""Insert into cache, evicting oldest if at max size.
Caller must hold the lock that guards ``self.cache``
(``self._lock`` for sync paths, ``self._get_async_lock()``
for async paths).
"""
if key in self.cache:
self.cache.move_to_end(key)
elif len(self.cache) >= _MAX_CACHE_SIZE:
self.cache.popitem(last=False)
self.cache[key] = value
async def _resolve_registry_async(
self,
server: str,
namespace: str,
name: str,
version: str,
cache_key: str,
package_type: str,
original_reference: str,
) -> dict[str, Any]:
"""Resolve a REGISTRY reference against a live registry server.
Fetches package content via ``RegistryClient.resolve_package``,
builds a content dict with trace metadata, and caches the result.
Args:
server: Registry server base URL.
namespace: Package namespace.
name: Package name.
version: Version string or alias.
cache_key: Normalised cache key.
package_type: Mapped package-type code (e.g. ``\"tpl\"``).
original_reference: The verbatim original reference string.
Returns:
Dict with ``_original_reference``, ``_server``,
``_namespace``, ``_name``, ``_version`` merged into the
resolved content.
"""
client = await self._get_client(server)
resolved = await client.resolve_package(
package_type=package_type,
namespace=namespace,
name=name,
version=version,
)
content: dict[str, Any] = {
"_original_reference": original_reference,
"_server": server,
"_namespace": namespace,
"_name": name,
"_version": version,
}
if isinstance(resolved, dict):
content.update(resolved)
async with self._get_async_lock():
self._put_cache(cache_key, content)
return copy.deepcopy(content)
async def close_all(self) -> None:
"""Close all open RegistryClient connections.
Closes each client under the async lock so that concurrent
``_get_client()`` calls cannot create a new client while the
close operation is in progress.
"""
async with self._get_async_lock():
clients = list(self.clients.values())
for client in clients:
try:
await client.close()
except Exception:
logger.debug(
"Error closing client for %s", client.base_url, exc_info=True
)
self.clients.clear()
def clear_cache(self) -> None:
"""Clear the resolution cache.
Only takes ``self._lock``. Callers that may run concurrently
with async paths should use ``aclear_cache()`` instead.
"""
with self._lock:
self.cache.clear()
+1
View File
@@ -128,6 +128,7 @@ class PackageReference:
name: str | None = None
version: str | None = None
id_string: str | None = None
package_type: str | None = None
@classmethod
def from_string(cls, raw_ref: str) -> PackageReference:
+4 -2
View File
@@ -2,10 +2,10 @@
Template system for CleverAgents.
This module provides a unified template system for creating reusable,
parameterizable components including agents, graphs, and streams.
parameterizable components including agents, graphs, streams, templates,
skills, actors, MCP servers, and LSP services.
"""
# Import only base types to avoid circular imports
from cleveractors.templates.base import (
BaseTemplate,
ComponentReference,
@@ -13,9 +13,11 @@ from cleveractors.templates.base import (
TemplateParameter,
TemplateType,
)
from cleveractors.templates.generic_template import GenericTemplate
__all__ = [
"BaseTemplate",
"GenericTemplate",
"TemplateParameter",
"TemplateType",
"ComponentReference",
+121 -8
View File
@@ -13,11 +13,16 @@ from typing import TYPE_CHECKING, Any, List, Optional
from jinja2 import Template
from cleveractors.core.exceptions import TemplateError
logger = logging.getLogger(__name__)
if TYPE_CHECKING:
from typing import Protocol
from cleveractors.registry.reference_resolver import PackageContentResolver
from cleveractors.registry.types import PackageReference
class TemplateRegistryProtocol(Protocol):
"""Protocol for template registries."""
@@ -42,6 +47,16 @@ class TemplateType(Enum):
AGENT = "agent"
GRAPH = "graph"
STREAM = "stream"
TEMPLATE = "template"
SKILL = "skill"
ACTOR = "actor"
MCP = "mcp"
LSP = "lsp"
@property
def plural_name(self) -> str:
"""The plural section name (e.g. ``\"agents\"`` for ``TemplateType.AGENT``)."""
return self.value + "s"
@dataclass
@@ -92,9 +107,12 @@ class TemplateParameter:
class ComponentReference:
"""Reference to a component that can be resolved during instantiation."""
ref_type: str # 'agent', 'graph', 'stream'
ref_type: (
str # 'agent', 'graph', 'stream', 'template', 'skill', 'actor', 'mcp', 'lsp'
)
ref_name: str
ref_params: dict[str, Any] = field(default_factory=dict)
package_ref: Optional["PackageReference"] = None
def resolve(self, context: "InstantiationContext") -> Any:
"""Resolve this reference in the given context."""
@@ -104,42 +122,78 @@ class ComponentReference:
class InstantiationContext:
"""Context for resolving references during template instantiation."""
def __init__(self, parent: Optional["InstantiationContext"] = None):
def __init__(
self,
parent: Optional["InstantiationContext"] = None,
reference_resolver: Optional["PackageContentResolver"] = None,
):
self.parent = parent
self.reference_resolver = reference_resolver
self.components: dict[str, dict[str, Any]] = {
"agents": {},
"graphs": {},
"streams": {},
"templates": {},
"skills": {},
"actors": {},
"mcps": {},
"lsps": {},
}
self._pending_resolutions: List[tuple[ComponentReference, str]] = []
def add_component(self, comp_type: str, name: str, instance: Any) -> None:
"""Add a component to this context."""
self.components[f"{comp_type}s"][name] = instance
key = f"{comp_type}s" if not comp_type.endswith("s") else comp_type
if key not in self.components:
self.components[key] = {}
self.components[key][name] = instance
logger.debug("Added %s '%s' to context", comp_type, name)
def resolve_reference(self, ref: ComponentReference) -> Any:
"""Resolve a component reference."""
"""Resolve a component reference, checking registry if needed."""
comp_type_plural = f"{ref.ref_type}s"
# First check local context
if ref.package_ref is not None:
return self._resolve_package_reference(ref)
if ref.ref_name in self.components.get(comp_type_plural, {}):
return self.components[comp_type_plural][ref.ref_name]
# Then check parent context
if self.parent:
return self.parent.resolve_reference(ref)
# If not found, it might be pending - add to pending list
self._pending_resolutions.append((ref, comp_type_plural))
return None
def _resolve_package_reference(self, ref: ComponentReference) -> Any:
if ref.package_ref is None:
return None
if self.reference_resolver is None:
raise TemplateError(
f"Cannot resolve package reference '{ref.package_ref}' "
"without a PackageContentResolver"
)
try:
resolved = self.reference_resolver.resolve(
ref.package_ref, package_type=ref.ref_type
)
except Exception as exc:
raise TemplateError(
f"Failed to resolve package reference '{ref.package_ref}': {exc}"
) from exc
if resolved is None:
raise TemplateError(
f"Failed to resolve package reference '{ref.package_ref}'"
)
if isinstance(resolved, dict) and "_original_reference" not in resolved:
resolved["_original_reference"] = ref.package_ref.original_reference
return resolved
def resolve_pending(self) -> None:
"""Attempt to resolve any pending references."""
unresolved = []
for ref, comp_type_plural in self._pending_resolutions:
if ref.ref_name in self.components.get(comp_type_plural, {}):
# Now it's available
continue
unresolved.append((ref, comp_type_plural))
@@ -312,3 +366,62 @@ class BaseTemplate(ABC):
result[key] = processed_value
return result
def _resolve_registry_ref(
template_name: str,
params: dict[str, Any],
package_type: str,
context: Optional[InstantiationContext],
) -> Any:
"""Shared helper for resolving a registry reference.
Used by both ``TemplateRegistry`` and ``EnhancedTemplateRegistry``
to avoid duplicated resolve/error-handling/cleanup logic.
Returns the resolved content dict. Raises ``TemplateError`` on failure.
"""
import asyncio
from cleveractors.core.exceptions import TemplateError
from cleveractors.registry.reference_resolver import PackageContentResolver
from cleveractors.registry.types import PackageReference
try:
package_ref = PackageReference.from_string(template_name)
except ValueError as exc:
raise TemplateError(
f"Invalid registry reference '{template_name}': {exc}"
) from exc
resolver = context.reference_resolver if context else None
close_resolver = False
if resolver is None:
resolver = PackageContentResolver()
close_resolver = True
try:
resolved = resolver.resolve(package_ref, package_type=package_type)
except Exception as exc:
raise TemplateError(
f"Failed to resolve registry reference '{template_name}': {exc}"
) from exc
finally:
if close_resolver:
try:
asyncio.run(resolver.close_all())
except Exception:
logging.getLogger(__name__).debug(
"Error closing ad-hoc resolver for %s",
template_name,
exc_info=True,
)
if resolved is None:
raise TemplateError(f"Failed to resolve registry reference '{template_name}'")
if params and isinstance(resolved, dict):
resolved = dict(resolved)
resolved.update(params)
return resolved
+65 -11
View File
@@ -13,7 +13,7 @@ from cleveractors.templates.template_store import TemplateStore
from cleveractors.templates.yaml_preprocessor import YAMLTemplateProcessor
if TYPE_CHECKING:
pass
from cleveractors.registry.types import PackageReference
logger = logging.getLogger(__name__)
@@ -27,11 +27,15 @@ class EnhancedTemplateRegistry:
self.store = TemplateStore()
self.processor = YAMLTemplateProcessor()
# Cache for instantiated template classes
self._template_cache: dict[TemplateType, dict[str, BaseTemplate]] = {
TemplateType.AGENT: {},
TemplateType.GRAPH: {},
TemplateType.STREAM: {},
TemplateType.TEMPLATE: {},
TemplateType.SKILL: {},
TemplateType.ACTOR: {},
TemplateType.MCP: {},
TemplateType.LSP: {},
}
logger.debug("Initialized enhanced template registry")
@@ -48,7 +52,7 @@ class EnhancedTemplateRegistry:
template_yaml: YAML string containing the template
"""
# Store the raw template
type_str = template_type.value + "s" # Convert to plural
type_str = template_type.plural_name
self.store.add_template(type_str, name, template_yaml)
logger.info(
"Registered %s template '%s' from string", template_type.value, name
@@ -104,10 +108,10 @@ class EnhancedTemplateRegistry:
AgentTemplate,
CompositeAgentTemplate,
)
from cleveractors.templates.generic_template import GenericTemplate
from cleveractors.templates.graph_templates import GraphTemplate
from cleveractors.templates.stream_templates import StreamTemplate
# Create appropriate template instance
template: BaseTemplate
if template_type == TemplateType.AGENT:
if definition.get("type") == "composite":
@@ -118,6 +122,14 @@ class EnhancedTemplateRegistry:
template = GraphTemplate(name, template_type, definition)
elif template_type == TemplateType.STREAM:
template = StreamTemplate(name, template_type, definition)
elif template_type in (
TemplateType.TEMPLATE,
TemplateType.SKILL,
TemplateType.ACTOR,
TemplateType.MCP,
TemplateType.LSP,
):
template = GenericTemplate(name, template_type, definition)
else:
raise ValueError(f"Unknown template type: {template_type}")
@@ -130,7 +142,7 @@ class EnhancedTemplateRegistry:
return self._template_cache[template_type][name]
# Check if it exists in the store
type_str = template_type.value + "s"
type_str = template_type.plural_name
if self.store.get_template(type_str, name):
# For complex templates in the store, we need to create a BaseTemplate wrapper
# This is a limitation - complex templates don't have a BaseTemplate representation
@@ -155,20 +167,26 @@ class EnhancedTemplateRegistry:
Args:
template_type: Type of template
name: Template name
name: Template name or registry reference
params: Template parameters
context: Optional instantiation context
Returns:
Instantiated configuration
"""
registry_ref = self._try_parse_registry_ref(name)
if registry_ref is not None:
return self._instantiate_from_registry_ref(
name, params, template_type, context
)
# Check cache first
if name in self._template_cache.get(template_type, {}):
template = self._template_cache[template_type][name]
return template.instantiate(params, self, context or InstantiationContext())
# Check store for complex templates
type_str = template_type.value + "s"
type_str = template_type.plural_name
raw_template = self.store.get_template(type_str, name)
if raw_template:
@@ -186,6 +204,32 @@ class EnhancedTemplateRegistry:
f"{template_type.value.capitalize()} template '{name}' not found"
)
@staticmethod
def _try_parse_registry_ref(template_name: str) -> Optional["PackageReference"]:
"""Return a PackageReference if *template_name* is a REGISTRY reference."""
from cleveractors.registry.types import PackageReference, ReferenceType
try:
ref = PackageReference.from_string(template_name)
except ValueError:
return None
if ref.reference_type != ReferenceType.REGISTRY:
return None
return ref
def _instantiate_from_registry_ref(
self,
template_name: str,
params: dict[str, Any],
template_type: TemplateType,
context: Optional[InstantiationContext],
) -> Any:
from cleveractors.templates.base import _resolve_registry_ref
return _resolve_registry_ref(
template_name, params, template_type.value, context
)
def register_all_templates(
self, templates_config: dict[str, dict[str, dict[str, Any]]]
) -> None: # pylint: disable=line-too-long
@@ -203,6 +247,16 @@ class EnhancedTemplateRegistry:
template_type = TemplateType.GRAPH
elif type_name == "streams":
template_type = TemplateType.STREAM
elif type_name == "templates":
template_type = TemplateType.TEMPLATE
elif type_name == "skills":
template_type = TemplateType.SKILL
elif type_name == "actors":
template_type = TemplateType.ACTOR
elif type_name == "mcps":
template_type = TemplateType.MCP
elif type_name == "lsps":
template_type = TemplateType.LSP
else:
logger.warning("Unknown template type: %s", type_name)
continue
@@ -218,7 +272,7 @@ class EnhancedTemplateRegistry:
return True
# Check store
type_str = template_type.value + "s"
type_str = template_type.plural_name
return self.store.get_template(type_str, name) is not None
def get_template_metadata(
@@ -236,7 +290,7 @@ class EnhancedTemplateRegistry:
}
# Check store
type_str = template_type.value + "s"
type_str = template_type.plural_name
metadata = self.store.get_metadata(type_str, name)
if metadata:
return metadata
@@ -253,14 +307,14 @@ class EnhancedTemplateRegistry:
if template_type:
# Single type
type_str = template_type.value + "s"
type_str = template_type.plural_name
names = list(self._template_cache.get(template_type, {}).keys())
names.extend(self.store.raw_templates.get(type_str, {}).keys())
result[type_str] = list(set(names)) # Remove duplicates
else:
# All types
for t_type in TemplateType:
type_str = t_type.value + "s"
type_str = t_type.plural_name
names = list(self._template_cache.get(t_type, {}).keys())
names.extend(self.store.raw_templates.get(type_str, {}).keys())
result[type_str] = list(set(names))
@@ -0,0 +1,32 @@
"""Generic template for package types without specialized template classes."""
from typing import TYPE_CHECKING, Any
from cleveractors.templates.base import (
BaseTemplate,
InstantiationContext,
TemplateType,
)
if TYPE_CHECKING:
from cleveractors.templates.base import TemplateRegistryProtocol
class GenericTemplate(BaseTemplate):
"""A fallback template for types without specialised rendering logic.
Used for TEMPLATE, SKILL, ACTOR, MCP, and LSP types. Instantiates
by deep-copying the definition and applying parameter substitution.
"""
def instantiate(
self,
params: dict[str, Any],
registry: "TemplateRegistryProtocol",
context: InstantiationContext,
) -> dict[str, Any]:
validated_params = self.validate_params(params)
result = self._apply_template_vars(self.definition, validated_params)
if isinstance(result, dict):
return result
return {"definition": result}
+74 -17
View File
@@ -3,12 +3,13 @@ Template registry for managing all template types.
"""
import logging
from typing import TYPE_CHECKING, Any, List, Optional
from typing import TYPE_CHECKING, Any, Optional
from cleveractors.core.exceptions import TemplateError
from cleveractors.templates.base import BaseTemplate, InstantiationContext, TemplateType
if TYPE_CHECKING:
pass
from cleveractors.registry.types import PackageReference
logger = logging.getLogger(__name__)
@@ -21,6 +22,11 @@ class TemplateRegistry:
TemplateType.AGENT: {},
TemplateType.GRAPH: {},
TemplateType.STREAM: {},
TemplateType.TEMPLATE: {},
TemplateType.SKILL: {},
TemplateType.ACTOR: {},
TemplateType.MCP: {},
TemplateType.LSP: {},
}
logger.debug("Initialized template registry")
@@ -34,10 +40,10 @@ class TemplateRegistry:
AgentTemplate,
CompositeAgentTemplate,
)
from cleveractors.templates.generic_template import GenericTemplate
from cleveractors.templates.graph_templates import GraphTemplate
from cleveractors.templates.stream_templates import StreamTemplate
# Create appropriate template instance
template: BaseTemplate
if template_type == TemplateType.AGENT:
if definition.get("type") == "composite":
@@ -48,6 +54,14 @@ class TemplateRegistry:
template = GraphTemplate(name, template_type, definition)
elif template_type == TemplateType.STREAM:
template = StreamTemplate(name, template_type, definition)
elif template_type in (
TemplateType.TEMPLATE,
TemplateType.SKILL,
TemplateType.ACTOR,
TemplateType.MCP,
TemplateType.LSP,
):
template = GenericTemplate(name, template_type, definition)
else:
raise ValueError(f"Unknown template type: {template_type}")
@@ -84,21 +98,32 @@ class TemplateRegistry:
self, config: dict[str, Any], context: Optional[InstantiationContext] = None
) -> Any:
"""Create an instance from configuration."""
# Handle None config
if config is None:
raise ValueError("Cannot instantiate from None configuration")
# Create context if not provided
if context is None:
context = InstantiationContext()
# Determine what we're instantiating
if "template" in config:
# Agent with template
template_name = config["template"]
params = config.get("params", {})
# Try to find template in order: agent, graph, stream
registry_ref = self._try_parse_registry_ref(template_name)
if registry_ref is not None:
for template_type in TemplateType:
try:
resolved = self._instantiate_from_registry_ref(
template_name, params, context, template_type
)
if resolved is not None:
return resolved
except TemplateError:
continue
raise TemplateError(
f"Failed to resolve registry reference '{template_name}' "
"across all template types"
)
for template_type in TemplateType:
if self.has_template(template_type, template_name):
template = self.get_template(template_type, template_name)
@@ -118,42 +143,74 @@ class TemplateRegistry:
template = self.get_template(TemplateType.STREAM, config["stream_template"])
return template.instantiate(config.get("params", {}), self, context)
# Direct definition (backward compatibility)
# pylint: disable=import-outside-toplevel
if "type" in config:
# Direct agent definition - will be handled by agent factory
return None
if "nodes" in config and "edges" in config:
# Direct graph definition
from cleveractors.langgraph.graph import GraphConfig
return GraphConfig(**config)
if "operators" in config:
# Direct stream definition
from cleveractors.reactive.stream_router import StreamConfig
return StreamConfig(**config)
raise ValueError("Cannot determine instance type from configuration")
@staticmethod
def _try_parse_registry_ref(template_name: str) -> Optional["PackageReference"]:
"""Return a PackageReference if *template_name* is a REGISTRY reference."""
from cleveractors.registry.types import PackageReference, ReferenceType
try:
ref = PackageReference.from_string(template_name)
except ValueError:
return None
if ref.reference_type != ReferenceType.REGISTRY:
return None
return ref
def _instantiate_from_registry_ref(
self,
template_name: str,
params: dict[str, Any],
context: InstantiationContext,
template_type: Optional[TemplateType] = None,
) -> Any:
from cleveractors.templates.base import _resolve_registry_ref
pkg_type = template_type.value if template_type else "template"
return _resolve_registry_ref(template_name, params, pkg_type, context)
def register_all_templates(
self, templates_config: dict[str, dict[str, dict[str, Any]]]
) -> None:
"""Register all templates from configuration."""
# Register agents
for name, definition in templates_config.get("agents", {}).items():
self.register_template(TemplateType.AGENT, name, definition)
# Register graphs
for name, definition in templates_config.get("graphs", {}).items():
self.register_template(TemplateType.GRAPH, name, definition)
# Register streams
for name, definition in templates_config.get("streams", {}).items():
self.register_template(TemplateType.STREAM, name, definition)
for name, definition in templates_config.get("templates", {}).items():
self.register_template(TemplateType.TEMPLATE, name, definition)
for name, definition in templates_config.get("skills", {}).items():
self.register_template(TemplateType.SKILL, name, definition)
for name, definition in templates_config.get("actors", {}).items():
self.register_template(TemplateType.ACTOR, name, definition)
for name, definition in templates_config.get("mcps", {}).items():
self.register_template(TemplateType.MCP, name, definition)
for name, definition in templates_config.get("lsps", {}).items():
self.register_template(TemplateType.LSP, name, definition)
def list_templates(
self, template_type: Optional[TemplateType] = None
) -> dict[str, List[str]]:
) -> dict[str, list[str]]:
"""List all registered templates."""
if template_type:
return {template_type.value: list(self.templates[template_type].keys())}
+12 -1
View File
@@ -27,6 +27,11 @@ class TemplateStore:
"agents": {},
"graphs": {},
"streams": {},
"templates": {},
"skills": {},
"actors": {},
"mcps": {},
"lsps": {},
}
# Store parsed metadata (parameters, type, etc.)
@@ -34,6 +39,11 @@ class TemplateStore:
"agents": {},
"graphs": {},
"streams": {},
"templates": {},
"skills": {},
"actors": {},
"mcps": {},
"lsps": {},
}
def add_template(
@@ -43,7 +53,8 @@ class TemplateStore:
Add a template definition.
Args:
template_type: Type of template ('agents', 'graphs', 'streams')
template_type: Type of template ('agents', 'graphs', 'streams',
'templates', 'skills', 'actors', 'mcps', 'lsps')
name: Template name
definition: Template definition as string or dict
"""