diff --git a/CHANGELOG.md b/CHANGELOG.md index ee79c0e..8c98b97 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,17 +7,27 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm ## [Unreleased] -### Fixed - -- **Registry Resolution Bugfixes (PR Review rui.hu)**: Fixed `TemplateRegistry._instantiate_from_registry_ref` hardcoding `package_type="template"` instead of threading the template type, which broke all non-template registry resolutions. `_try_parse_registry_ref` now filters to `ReferenceType.REGISTRY` only, preserving the "local templates unaffected" acceptance criterion. `_original_reference` in resolved results now stores the verbatim original reference string, not the cache-key with type suffix. `application.py` template-type mapping extended to cover all 8 types (previously only AGENT/GRAPH/STREAM — the other 5 silently misclassified as STREAM). Duplicated `_instantiate_from_registry_ref`/`_try_parse_registry_ref` logic extracted into shared `_resolve_registry_ref` helper in `base.py`. `RegistryClient` now URL-encodes namespace/name path components to prevent injection. HTTPS enforcement with `allow_insecure` flag per spec §12.2. `ReferenceResolver.close_all()` closes clients under the async lock to prevent concurrent client leak. Async lock lazily initialised to avoid deprecation warning. Added `plural_name` property to `TemplateType`. - -- **Registry Error Hierarchy Fixes (PR Review rui.hu Round 5)**: Fixed `details` dropping in non-5xx fallback path of `_request()` — `details` extracted from structured error bodies are now propagated to `RegistryError` for all unrecognised error types. Added `asyncio.Lock` to `RegistryClient._get_client()` to match the project-standardised check-and-create pattern used by `ReferenceResolver._get_client()`. Fixed a no-op BDD scenario ("404 without error type in body falls back to PackageNotFoundError") that had no `Then` step. Fixed HTTPS logging step definitions that constructed a new `RegistryClient` internally, defeating test isolation. Enhanced the "Client with API key" scenario to verify the `Authorization` header is present. Added end-to-end BDD scenario for `details` propagation from structured error bodies. Added `quote(package_id)` URL-encoding to `get_package()`. Replaced misleading `if exc.request is not None:` with `if getattr(exc, "_request", None) is not None:`. Simplified `RegistryNetworkError.__str__` using parts list + join. Replaced deprecated `asyncio.get_event_loop().run_until_complete()` with `asyncio.run()`. Tightened `_CLASS_MAP` type annotation to `dict[str, type[CleverAgentsException]]`. Added comment to `typing.cast(int, code)` explaining intentional non-int test. - - -- **Registry Reference Resolution**: Fixed `ReferenceResolver._resolve_registry_async` passing the package name instead of the type code to `RegistryClient.resolve_package`, which would construct invalid API paths and break all registry integrations. Added `package_type` parameter to `resolve()` and `aresolve()`, with automatic mapping from `TemplateType` values to `PackageType` codes. Replaced `ValueError` with `TemplateError` for all registry fetch failure paths per acceptance criteria. Cache and client-pool operations now use `asyncio.Lock` in async coroutines (`aresolve`, `_resolve_registry_async`, `_get_client`, `close_all`) to prevent event-loop blocking, while retaining `threading.Lock` for the synchronous `resolve()` path. Use bounded `OrderedDict` with LRU eviction and consistently return deep copies to prevent cache poisoning. Fixed `_get_client()` race condition by protecting the full check-and-create under a single lock acquisition. Ad-hoc `ReferenceResolver` instances created during `_instantiate_from_registry_ref` are now properly closed in both success and error paths. Registry reference detection now uses proper `PackageReference.from_string()` parsing instead of a fragile `":" in template_name` check, and returns PackageReference for all reference types (LOCAL, ID, REGISTRY) enabling resolution of local and ID references through `instantiate_from_config` and `EnhancedTemplateRegistry.instantiate()`. Template `params` from registry-based configs are now applied to resolved content instead of being silently discarded. Added async `aresolve()` entry point to prevent `RuntimeError` when called from running event loops; `resolve()` now detects running loops and directs callers to `aresolve()`. Bare `except Exception: pass` blocks replaced with debug-level logging. - ### Added +- **Router-facing API documented in README** (`README.md`): New "Router-facing API" section documents all 8 router-facing exports (`validate_dict`, `merge_configs`, `create_executor`, `Executor`, `ActorResult`, `NodeUsage`, `ExecutionError`, `ConfigurationError`) with a complete end-to-end usage example, individual subsections for each export, streaming documentation for `execute_stream()` (including `try/except ExecutionError` billing example), and a reference table for `ExecutionError.kind`/`.reason` values. "Package structure" and "Key exports" tables updated to reflect the v2.1.0 API surface. `TemplateRegistry` re-exported from `cleveractors.templates` so the Key exports import block is runnable without `ImportError`. (issue #17, ADR-2024, ADR-2026, ADR-2027, ADR-2029) +- **Registry Error Hierarchy** (`cleveractors.registry.exceptions`): Typed exception hierarchy per Package Registry Standard §13.2. `RegistryError(CleverAgentsException)` base carries `message`, optional `details: dict`, and optional `original_reference: str`; `__str__` includes the reference when present. Nine leaf exceptions: `PackageNotFoundError` (404), `InvalidPackageIdError` (400), `InvalidPackageReferenceError` (400), `VersionNotFoundError` (404), `ValidationError` (400), `AuthenticationRequiredError` (401), `AccessDeniedError` (403), `ConflictError` (409), and `RegistryNetworkError` (5xx / connection / timeout) which additionally carries `status_code` and `url`. `exception_for_status()` maps HTTP codes to typed exceptions; `_ERROR_TYPE_MAP` enables error-type parsing from structured JSON error bodies. 23 Behave BDD scenarios and 12 Robot Framework integration tests; exceptions.py achieves 100% coverage. +- **`Executor.execute_stream()` — token-by-token streaming delivery** (`cleveractors.runtime.Executor`, `cleveractors.agents.llm.LLMAgent`, `cleveractors.langgraph.nodes.Node`, `cleveractors.langgraph.pure_graph.PureLangGraph`, `cleveractors.runtime_dispatch`): `LLMAgent` gains `stream_message(message, context)` using `self.chat_model.astream(messages)`, yielding token chunks via the LangChain streaming API with `_safe_int()` / fallback chain for token counts from the final chunk's `usage_metadata`. `Node` gains `stream_agent(state)` which delegates to `stream_message()` for `LLMAgent` instances or falls back to `process_message()` for non-LLM agents. `PureLangGraph.execute_stream()` mirrors `execute()` but uses `_stream_from_node()`, which buffers tokens for intermediate AGENT nodes (only yielding from the terminal node) while running non-AGENT nodes with `ainvoke()`. `Executor` gains `last_result: ActorResult | None = None` (populated after stream exhaustion for billing) and `execute_stream(message)` dispatching to `_execute_llm_stream()` (LLM actors) or `_execute_graph_stream()` (graph actors). All existing execution limits (`timeout_ms`, `max_model_calls`, `max_tool_calls`) are enforced in the streaming path. `execute_stream()` raises `ConfigurationError` for unsupported actor types (`tool`, `multi_actor`). No new top-level package export required. (issue #16) +- **Structured `ExecutionError` fields + 5-limit enforcement in `PureLangGraph`** (`cleveractors.core.exceptions.ExecutionError`, `cleveractors.langgraph.pure_graph.PureLangGraph`): `ExecutionError` gains `kind` (categorical: `depth`, `model_calls`, `tool_calls`, `timeout`, `cost`) and `reason` (sub-code: `budget_exhausted` or `missing_pricing_entry`) fields, both defaulting to `""` for backward compatibility with existing `raise ExecutionError(msg)` call sites. `PureLangGraph` now accepts `limits` and `pricing` constructor arguments. When `limits["max_depth"]` is supplied, a depth breach raises `ExecutionError(kind="depth")` instead of silently returning the current message. `max_model_calls` is checked before each AGENT-type node; `max_tool_calls` before each TOOL-type node. `execute()` wraps `_execute_from_node()` in `asyncio.wait_for()` when `limits["timeout_ms"]` is set, mapping `asyncio.TimeoutError` to `ExecutionError(kind="timeout")`. After each LLM node, cost is computed from the supplied `pricing` table (rates are USD per million tokens per ADR-2029); breach raises `ExecutionError(kind="cost", reason="budget_exhausted")`; a missing provider or model entry raises `ExecutionError(kind="cost", reason="missing_pricing_entry")`. `runtime_dispatch._execute_graph()` now passes `executor.limits` and `executor.pricing` to `PureLangGraph`. `ExecutionError` is exported from `cleveractors.__init__` and `__all__`. (ADR-2029, issue #15) +- **`create_executor()` router-facing API** (`cleveractors.create_executor`): New module-level factory function that constructs an `Executor` wrapping `PureLangGraph` and `AgentFactory`. Accepts `config_dict` (validated actor configuration), `credentials` (per-provider credential dict for per-request injection), `limits` (execution budget), and `pricing` (per-model cost table). `executor.execute(message)` runs the actor graph and returns an `ActorResult(response, prompt_tokens, completion_tokens, nodes)`. All four execution paths are supported — `llm`, `graph`, `tool`, and `multi_actor`. Credentials are passed to `AgentFactory` and never injected into the stored `config_dict` (ADR-2026 AC8). Exported from `cleveractors.__init__` and `__all__` (ADR-2024, ADR-2026, ADR-2029). +- **`ActorResult` and `NodeUsage` types** (`cleveractors.result`): Canonical dataclasses for the router-facing result API, now defined in `cleveractors.result` (ADR-2027). `ActorResult` carries the response string, aggregated `prompt_tokens`/`completion_tokens`, a non-empty `nodes: list[NodeUsage]` breakdown for per-model billing, and an optional opaque `state` blob for stateless graph resumption (ADR-2026). Re-exported from `cleveractors.runtime` for backward compatibility. +- **Real LangChain token extraction** (`LLMAgent`): `process_message()` now reads token counts from `response.usage_metadata` (primary path) with fallback to `response.response_metadata["token_usage"]`. Both paths are guarded with `isinstance(dict)` checks to prevent `AttributeError` on truthy non-dict provider values. Non-numeric token values are coerced via `_safe_int()` with fallback to 0 and a warning log. A warning distinguishing the failure cause (empty `usage_metadata`, missing `response_metadata`, or missing `token_usage` key) is emitted when no usage data is available. +- **Per-node token accumulation** (`PureLangGraph`, `Node`): `Node._execute_agent()` reads `_last_token_usage`, `provider`, and `model` from each `LLMAgent` after invocation and includes a `_node_token_usage` dict in the state-updates return. `PureLangGraph` accumulates these into a `_node_usages` list and returns a 3-tuple `(response, final_state, node_usages)` from `execute()`. +- **Registry Client-Side Cache** (`cleveractors.registry.RegistryCache` + `CacheFactory` + `PackageContentResolver` integration): LRU cache with TTL for `RegistryClient` responses implementing Package Registry Standard v1.0.0 §10.3 client-side caching requirements. Wraps a `RegistryClient` as a transparent caching layer with configurable max size and TTL. Features SHA-1 content validation via `Canonicalizer` for tamper detection, LRU eviction policy, singleflight coalescing for concurrent-miss thundering-herd prevention, per-entry invalidation (`invalidate`), manual warm insertion (`put` — used by `PackageContentResolver` to seed the content cache from `resolve_package` results), full cache clear, `CacheStats` for observability (hits, misses, evictions), per-entry logging, async context manager support, and `__contains__` lookup. All cache operations are protected by an `asyncio.Lock` for thread-safe concurrent access. Returns defensive copies of stored content to prevent mutable-dict cache poisoning. Validates `package_id` format via `PackageId.from_string` as the first guard in all public methods. SHA-1 validation can be toggled off via `validate_content=False` for environments where the upstream content format does not directly match the stored PackageId (e.g., integration testing against incomplete mock servers). Boolean `max_size` guard rejects `True`/`False`. **CacheFactory** (Factory Method + Dependency Injection patterns) centralises cache creation and isolates cache configuration from consumers. **PackageContentResolver** now integrates a two-tier caching architecture: a resolution cache (`OrderedDict`, reference→enriched content) and per-server content caches (`RegistryCache`, package_id→raw content) created by `CacheFactory`. +- **Per-request credential injection** (`AgentFactory` + `LLMAgent`): `AgentFactory` now accepts an optional `credentials: dict[str, dict[str, str]] | None` parameter. When supplied, each credential entry (keyed by provider name) is forwarded to the corresponding `LLMAgent`. The LangChain client is constructed lazily on first access of the `chat_model` property using the injected credentials, so API keys are never baked into the stored actor config dict (ADR-2026). +- **Extended provider routing** (`LLMAgent`): Any provider not in `{openai, anthropic, google}` (e.g., `groq`, `fireworks`, `together`, `mistral`, `openrouter`, or the generic `openai_compatible` extension) is now routed to `ChatOpenAI(base_url=..., api_key=...)` using the `base_url` supplied in the credentials entry for that provider. Named providers and `openai_compatible` are treated identically under this routing path (ADR-2028). +- **`ReactiveAgentFactory` backward-compatibility alias**: A module-level type alias `ReactiveAgentFactory: type[AgentFactory] = AgentFactory` is provided in `cleveractors.agents.factory` for backward compatibility with existing code that references the old name from v2.0.0. +- **Registry Template Integration**: Extended `TemplateType` enum with five new members (`TEMPLATE`, `SKILL`, `ACTOR`, `MCP`, `LSP`) to align with the Package Registry Standard v1.0.0 package types (§3.2). Added optional `package_ref: PackageReference | None` field to `ComponentReference` for external registry references. Created `ReferenceResolver` class with multi-server `RegistryClient` pool and resolution caching for local, ID, and registry reference formats (§5.3). Updated `InstantiationContext` with registry-aware resolution and `_original_reference` propagation through component metadata. Extended `TemplateRegistry`, `EnhancedTemplateRegistry`, and `TemplateStore` to support all eight template types. Registry-style template references (e.g. `"git.cleverthis.com:acme/helper@v1.0.0"`) are now detected and resolved via `ReferenceResolver` during `instantiate_from_config`. 25 Robot Framework integration tests, 24 Behave BDD scenarios, and full coverage of all new modules. +- **`merge_configs()` public API** (`cleveractors.merge_configs`): New module-level function implementing the Actor Configuration Standard §3.1 deep-merge algorithm. Accepts an arbitrary number of `dict[str, Any]` arguments and returns a fresh merged dict without mutating any input. Merge semantics: absent key → add; both mappings → deep-merge recursively; both sequences → append; otherwise → replace. Zero-argument call returns `{}`. Exported from `cleveractors.__init__` and `__all__`. +- **`validate_dict()` public API** (`cleveractors.validate_dict`): New router-facing validation function that validates a spec-conformant Actor Configuration Standard v1.0.0 dict against the full schema and enforces platform-level structural constraints. Validates top-level key presence (`agents`, `routes`), agent types, LLM provider allowlists, route/edge field names (`source`/`target` only — legacy `from`/`to` rejected), node types, stream operator types, and structural limits (`max_graph_depth`, `max_subgraph_depth`, `max_total_nodes`) from the supplied `platform_limits` dict. Returns the dict unchanged when valid; raises `ConfigurationError` on any violation. Pure static validator with no file I/O, env-var reads, or app construction. Exported from `cleveractors.__init__` and listed in `__all__`. (ADR-2024, ADR-2025, ADR-2029) +- **Registry HTTP Client** (`cleveractors.registry`): Async registry client implementing the Package Registry Standard v1.0.0 HTTP API endpoints (§8) using `httpx.AsyncClient`. Supports all four endpoints: `GET /packages/{id}` (retrieve by PackageId), `GET /{type}/{ns}/{name}?version=` (reference resolution with version alias resolution per §4.2), `GET /browse` (discovery with type/namespace filters), and `GET /.well-known/cleverthis-packages` (registry metadata). Maps all 8 standard error types (§13.2) to typed exceptions via error-body type parsing with status-code fallback. Supports anonymous reads (§9.1) and optional API key authentication (§9.2). Includes async context manager support, 32 Behave BDD scenarios, 7 Robot Framework integration tests against a fake server, and ASV benchmarks. +- **Canonicalizer** (`cleveractors.registry.Canonicalizer`): Deterministic canonicalization engine implementing Package Registry Standard v1.0.0 §6. Produces content-addressed SHA-1 hashes via `compute_package_id()` for all 8 package types. Applies Unicode NFC normalization to all string values, lexicographic key sorting for deterministic output, lifecycle field stripping (`version`, `release_date`), and RFC-8785 canonical JSON serialization. Internal reference resolution is integrated into the canonicalization pipeline (§6.2 step 5) — when configured, references are resolved before transformation. Includes `max_depth` recursion protection, argument validation on all public methods, `allow_nan=False` for valid JSON output, and ASV performance benchmarks. 31 Behave BDD scenarios (including error-path and integration tests) and 10 Robot Framework integration tests; canonical.py achieves 100% coverage. +- **Package Registry Core Types** (`cleveractors.registry.types`): Content-addressed `PackageId` in `pkg__<40-hex-sha1>` format (§5.1-5.2) with validation, `PackageType` enum covering all 8 standard types (§3.2), `PackageReference` parsing for all three reference formats (§5.3): registry (`server:ns/name@version`), ID (`ID:pkg__`), and local (`local:`), and `PackageContent` wrapping fetched packages with resolved identity and metadata. Validates namespace/name against safe character set `[a-zA-Z0-9_.-]+` and rejects `@` in identifier portions. 24 Behave BDD scenarios. +- **Reference Resolver** (`cleveractors.registry.resolver`): `ReferenceResolver` class implementing Package Registry Standard §5.3 and §4.2. `parse()` wraps `PackageReference.from_string` returning structured `PackageReference` objects with `InvalidPackageReferenceError` on invalid input (including `None`). `resolve()` resolves references to concrete `PackageId`: ID references resolve directly, registry references use the `RegistryClient` for HTTP resolution with version alias support and a configurable `package_type` parameter (default `"actor"`), local references raise informative error. Registry client errors and malformed `PackageId` values from the registry are both wrapped in `InvalidPackageReferenceError`. Supports async context manager (`__aenter__`/`__aexit__`) and `close()` for clean resource management. `resolve_version()` standalone function resolves version aliases (`latest`, `vx`, `x`, `vX.x`, `vX.Y.x`) against available concrete versions per §4.2 semantics using O(n) `max()` instead of O(n log n) sort with `@functools.lru_cache` on the semver parser. `is_concrete_version()` and `is_version_alias()` classifiers (the latter now explicitly matches known alias patterns instead of using complement-of-concrete). Package reference parsing validates namespace/name against module-level `[a-zA-Z0-9_.-]+` regex and rejects `@` in identifier portions. Local references reject directory traversal and absolute paths. Fake registry server uses semantic version sorting (§4.2) and returns proper 404 on version-not-found (§13.2). 43 Behave BDD scenarios, 42 Robot Framework integration tests against a fake server with version alias resolution, and 7 ASV benchmark classes covering parse, resolve, version resolution at scale, and registry reference resolution. - **Local Namespace Reference Resolution** (`cleveractors.registry.local_store`): Implements the `local:` reference scheme per Package Registry Standard §5.3. **Core types:** `LocalPackageStore` reads YAML package files from a configurable `base_dir` (accepting `Path | str`), canonicalizes content, and assigns content-addressed `PackageId`s via SHA-1. The `LocalPackage` frozen dataclass carries the `package_id`, `content` (resolved dict), `file_path`, and `original_reference` string (the verbatim `local:`). Package type is auto-detected via `_TYPE_DETECTORS` covering all 9 resource types: `GRAPH` (`graph` key), `STREAM` (`stream` key), `AGENT` (`agents` key), `SKILL` (`skill` key), `MCP` (`mcp` key), `LSP` (`lsp` key), `ACTOR` (`actor` key), `COMPOSITE` (`nodes` and `edges`), falling back to `TEMPLATE`. @@ -40,24 +50,6 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm **Module:** `src/cleveractors/registry/local_store.py`, `exceptions.py` (`CircularLocalReferenceError`), `canonical.py` (`_resolving` propagation), `reference_resolver.py` (LOCAL resolution + caching), `resolver.py` (async `asyncio.to_thread` for LOCAL refs), `__init__.py`. 22 files changed. -- **Registry Error Hierarchy** (`cleveractors.registry.exceptions`): Typed exception hierarchy per Package Registry Standard §13.2. `RegistryError(CleverAgentsException)` base carries `message`, optional `details: dict`, and optional `original_reference: str`; `__str__` includes the reference when present. Nine leaf exceptions: `PackageNotFoundError` (404), `InvalidPackageIdError` (400), `InvalidPackageReferenceError` (400), `VersionNotFoundError` (404), `ValidationError` (400), `AuthenticationRequiredError` (401), `AccessDeniedError` (403), `ConflictError` (409), and `RegistryNetworkError` (5xx / connection / timeout) which additionally carries `status_code` and `url`. `exception_for_status()` maps HTTP codes to typed exceptions; `_ERROR_TYPE_MAP` enables error-type parsing from structured JSON error bodies. 23 Behave BDD scenarios and 12 Robot Framework integration tests; exceptions.py achieves 100% coverage. -- **`Executor.execute_stream()` — token-by-token streaming delivery** (`cleveractors.runtime.Executor`, `cleveractors.agents.llm.LLMAgent`, `cleveractors.langgraph.nodes.Node`, `cleveractors.langgraph.pure_graph.PureLangGraph`, `cleveractors.runtime_dispatch`): `LLMAgent` gains `stream_message(message, context)` using `self.chat_model.astream(messages)`, yielding token chunks via the LangChain streaming API with `_safe_int()` / fallback chain for token counts from the final chunk's `usage_metadata`. `Node` gains `stream_agent(state)` which delegates to `stream_message()` for `LLMAgent` instances or falls back to `process_message()` for non-LLM agents. `PureLangGraph.execute_stream()` mirrors `execute()` but uses `_stream_from_node()`, which buffers tokens for intermediate AGENT nodes (only yielding from the terminal node) while running non-AGENT nodes with `ainvoke()`. `Executor` gains `last_result: ActorResult | None = None` (populated after stream exhaustion for billing) and `execute_stream(message)` dispatching to `_execute_llm_stream()` (LLM actors) or `_execute_graph_stream()` (graph actors). All existing execution limits (`timeout_ms`, `max_model_calls`, `max_tool_calls`) are enforced in the streaming path. `execute_stream()` raises `ConfigurationError` for unsupported actor types (`tool`, `multi_actor`). No new top-level package export required. (issue #16) -- **Structured `ExecutionError` fields + 5-limit enforcement in `PureLangGraph`** (`cleveractors.core.exceptions.ExecutionError`, `cleveractors.langgraph.pure_graph.PureLangGraph`): `ExecutionError` gains `kind` (categorical: `depth`, `model_calls`, `tool_calls`, `timeout`, `cost`) and `reason` (sub-code: `budget_exhausted` or `missing_pricing_entry`) fields, both defaulting to `""` for backward compatibility with existing `raise ExecutionError(msg)` call sites. `PureLangGraph` now accepts `limits` and `pricing` constructor arguments. When `limits["max_depth"]` is supplied, a depth breach raises `ExecutionError(kind="depth")` instead of silently returning the current message. `max_model_calls` is checked before each AGENT-type node; `max_tool_calls` before each TOOL-type node. `execute()` wraps `_execute_from_node()` in `asyncio.wait_for()` when `limits["timeout_ms"]` is set, mapping `asyncio.TimeoutError` to `ExecutionError(kind="timeout")`. After each LLM node, cost is computed from the supplied `pricing` table (rates are USD per million tokens per ADR-2029); breach raises `ExecutionError(kind="cost", reason="budget_exhausted")`; a missing provider or model entry raises `ExecutionError(kind="cost", reason="missing_pricing_entry")`. `runtime_dispatch._execute_graph()` now passes `executor.limits` and `executor.pricing` to `PureLangGraph`. `ExecutionError` is exported from `cleveractors.__init__` and `__all__`. (ADR-2029, issue #15) -- **`create_executor()` router-facing API** (`cleveractors.create_executor`): New module-level factory function that constructs an `Executor` wrapping `PureLangGraph` and `AgentFactory`. Accepts `config_dict` (validated actor configuration), `credentials` (per-provider credential dict for per-request injection), `limits` (execution budget), and `pricing` (per-model cost table). `executor.execute(message)` runs the actor graph and returns an `ActorResult(response, prompt_tokens, completion_tokens, nodes)`. All four execution paths are supported — `llm`, `graph`, `tool`, and `multi_actor`. Credentials are passed to `AgentFactory` and never injected into the stored `config_dict` (ADR-2026 AC8). Exported from `cleveractors.__init__` and `__all__` (ADR-2024, ADR-2026, ADR-2029). -- **`ActorResult` and `NodeUsage` types** (`cleveractors.result`): Canonical dataclasses for the router-facing result API, now defined in `cleveractors.result` (ADR-2027). `ActorResult` carries the response string, aggregated `prompt_tokens`/`completion_tokens`, a non-empty `nodes: list[NodeUsage]` breakdown for per-model billing, and an optional opaque `state` blob for stateless graph resumption (ADR-2026). Re-exported from `cleveractors.runtime` for backward compatibility. -- **Real LangChain token extraction** (`LLMAgent`): `process_message()` now reads token counts from `response.usage_metadata` (primary path) with fallback to `response.response_metadata["token_usage"]`. Both paths are guarded with `isinstance(dict)` checks to prevent `AttributeError` on truthy non-dict provider values. Non-numeric token values are coerced via `_safe_int()` with fallback to 0 and a warning log. A warning distinguishing the failure cause (empty `usage_metadata`, missing `response_metadata`, or missing `token_usage` key) is emitted when no usage data is available. -- **Per-node token accumulation** (`PureLangGraph`, `Node`): `Node._execute_agent()` reads `_last_token_usage`, `provider`, and `model` from each `LLMAgent` after invocation and includes a `_node_token_usage` dict in the state-updates return. `PureLangGraph` accumulates these into a `_node_usages` list and returns a 3-tuple `(response, final_state, node_usages)` from `execute()`. -- **Registry Client-Side Cache** (`cleveractors.registry.RegistryCache` + `CacheFactory` + `PackageContentResolver` integration): LRU cache with TTL for `RegistryClient` responses implementing Package Registry Standard v1.0.0 §10.3 client-side caching requirements. Wraps a `RegistryClient` as a transparent caching layer with configurable max size and TTL. Features SHA-1 content validation via `Canonicalizer` for tamper detection, LRU eviction policy, singleflight coalescing for concurrent-miss thundering-herd prevention, per-entry invalidation (`invalidate`), manual warm insertion (`put` — used by `PackageContentResolver` to seed the content cache from `resolve_package` results), full cache clear, `CacheStats` for observability (hits, misses, evictions), per-entry logging, async context manager support, and `__contains__` lookup. All cache operations are protected by an `asyncio.Lock` for thread-safe concurrent access. Returns defensive copies of stored content to prevent mutable-dict cache poisoning. Validates `package_id` format via `PackageId.from_string` as the first guard in all public methods. SHA-1 validation can be toggled off via `validate_content=False` for environments where the upstream content format does not directly match the stored PackageId (e.g., integration testing against incomplete mock servers). Boolean `max_size` guard rejects `True`/`False`. **CacheFactory** (Factory Method + Dependency Injection patterns) centralises cache creation and isolates cache configuration from consumers. **PackageContentResolver** now integrates a two-tier caching architecture: a resolution cache (`OrderedDict`, reference→enriched content) and per-server content caches (`RegistryCache`, package_id→raw content) created by `CacheFactory`. -- **Per-request credential injection** (`AgentFactory` + `LLMAgent`): `AgentFactory` now accepts an optional `credentials: dict[str, dict[str, str]] | None` parameter. When supplied, each credential entry (keyed by provider name) is forwarded to the corresponding `LLMAgent`. The LangChain client is constructed lazily on first access of the `chat_model` property using the injected credentials, so API keys are never baked into the stored actor config dict (ADR-2026). -- **Extended provider routing** (`LLMAgent`): Any provider not in `{openai, anthropic, google}` (e.g., `groq`, `fireworks`, `together`, `mistral`, `openrouter`, or the generic `openai_compatible` extension) is now routed to `ChatOpenAI(base_url=..., api_key=...)` using the `base_url` supplied in the credentials entry for that provider. Named providers and `openai_compatible` are treated identically under this routing path (ADR-2028). -- **`ReactiveAgentFactory` backward-compatibility alias**: A module-level type alias `ReactiveAgentFactory: type[AgentFactory] = AgentFactory` is provided in `cleveractors.agents.factory` for backward compatibility with existing code that references the old name from v2.0.0. -- **Registry Template Integration**: Extended `TemplateType` enum with five new members (`TEMPLATE`, `SKILL`, `ACTOR`, `MCP`, `LSP`) to align with the Package Registry Standard v1.0.0 package types (§3.2). Added optional `package_ref: PackageReference | None` field to `ComponentReference` for external registry references. Created `ReferenceResolver` class with multi-server `RegistryClient` pool and resolution caching for local, ID, and registry reference formats (§5.3). Updated `InstantiationContext` with registry-aware resolution and `_original_reference` propagation through component metadata. Extended `TemplateRegistry`, `EnhancedTemplateRegistry`, and `TemplateStore` to support all eight template types. Registry-style template references (e.g. `"git.cleverthis.com:acme/helper@v1.0.0"`) are now detected and resolved via `ReferenceResolver` during `instantiate_from_config`. 25 Robot Framework integration tests, 24 Behave BDD scenarios, and full coverage of all new modules. -- **`merge_configs()` public API** (`cleveractors.merge_configs`): New module-level function implementing the Actor Configuration Standard §3.1 deep-merge algorithm. Accepts an arbitrary number of `dict[str, Any]` arguments and returns a fresh merged dict without mutating any input. Merge semantics: absent key → add; both mappings → deep-merge recursively; both sequences → append; otherwise → replace. Zero-argument call returns `{}`. Exported from `cleveractors.__init__` and `__all__`. -- **`validate_dict()` public API** (`cleveractors.validate_dict`): New router-facing validation function that validates a spec-conformant Actor Configuration Standard v1.0.0 dict against the full schema and enforces platform-level structural constraints. Validates top-level key presence (`agents`, `routes`), agent types, LLM provider allowlists, route/edge field names (`source`/`target` only — legacy `from`/`to` rejected), node types, stream operator types, and structural limits (`max_graph_depth`, `max_subgraph_depth`, `max_total_nodes`) from the supplied `platform_limits` dict. Returns the dict unchanged when valid; raises `ConfigurationError` on any violation. Pure static validator with no file I/O, env-var reads, or app construction. Exported from `cleveractors.__init__` and listed in `__all__`. (ADR-2024, ADR-2025, ADR-2029) -- **Registry HTTP Client** (`cleveractors.registry`): Async registry client implementing the Package Registry Standard v1.0.0 HTTP API endpoints (§8) using `httpx.AsyncClient`. Supports all four endpoints: `GET /packages/{id}` (retrieve by PackageId), `GET /{type}/{ns}/{name}?version=` (reference resolution with version alias resolution per §4.2), `GET /browse` (discovery with type/namespace filters), and `GET /.well-known/cleverthis-packages` (registry metadata). Maps all 8 standard error types (§13.2) to typed exceptions via error-body type parsing with status-code fallback. Supports anonymous reads (§9.1) and optional API key authentication (§9.2). Includes async context manager support, 32 Behave BDD scenarios, 7 Robot Framework integration tests against a fake server, and ASV benchmarks. -- **Canonicalizer** (`cleveractors.registry.Canonicalizer`): Deterministic canonicalization engine implementing Package Registry Standard v1.0.0 §6. Produces content-addressed SHA-1 hashes via `compute_package_id()` for all 8 package types. Applies Unicode NFC normalization to all string values, lexicographic key sorting for deterministic output, lifecycle field stripping (`version`, `release_date`), and RFC-8785 canonical JSON serialization. Internal reference resolution is integrated into the canonicalization pipeline (§6.2 step 5) — when configured, references are resolved before transformation. Includes `max_depth` recursion protection, argument validation on all public methods, `allow_nan=False` for valid JSON output, and ASV performance benchmarks. 31 Behave BDD scenarios (including error-path and integration tests) and 10 Robot Framework integration tests; canonical.py achieves 100% coverage. -- **Package Registry Core Types** (`cleveractors.registry.types`): Content-addressed `PackageId` in `pkg__<40-hex-sha1>` format (§5.1-5.2) with validation, `PackageType` enum covering all 8 standard types (§3.2), `PackageReference` parsing for all three reference formats (§5.3): registry (`server:ns/name@version`), ID (`ID:pkg__`), and local (`local:`), and `PackageContent` wrapping fetched packages with resolved identity and metadata. Validates namespace/name against safe character set `[a-zA-Z0-9_.-]+` and rejects `@` in identifier portions. 24 Behave BDD scenarios. -- **Reference Resolver** (`cleveractors.registry.resolver`): `ReferenceResolver` class implementing Package Registry Standard §5.3 and §4.2. `parse()` wraps `PackageReference.from_string` returning structured `PackageReference` objects with `InvalidPackageReferenceError` on invalid input (including `None`). `resolve()` resolves references to concrete `PackageId`: ID references resolve directly, registry references use the `RegistryClient` for HTTP resolution with version alias support and a configurable `package_type` parameter (default `"actor"`), local references raise informative error. Registry client errors and malformed `PackageId` values from the registry are both wrapped in `InvalidPackageReferenceError`. Supports async context manager (`__aenter__`/`__aexit__`) and `close()` for clean resource management. `resolve_version()` standalone function resolves version aliases (`latest`, `vx`, `x`, `vX.x`, `vX.Y.x`) against available concrete versions per §4.2 semantics using O(n) `max()` instead of O(n log n) sort with `@functools.lru_cache` on the semver parser. `is_concrete_version()` and `is_version_alias()` classifiers (the latter now explicitly matches known alias patterns instead of using complement-of-concrete). Package reference parsing validates namespace/name against module-level `[a-zA-Z0-9_.-]+` regex and rejects `@` in identifier portions. Local references reject directory traversal and absolute paths. Fake registry server uses semantic version sorting (§4.2) and returns proper 404 on version-not-found (§13.2). 43 Behave BDD scenarios, 42 Robot Framework integration tests against a fake server with version alias resolution, and 7 ASV benchmark classes covering parse, resolve, version resolution at scale, and registry reference resolution. - **Core CleverActors Framework**: New agent-based LLM orchestration framework implementing the Actor Configuration Standard (§1-4). Includes agent base class, factory pattern for agent creation, configuration management, template rendering engine, and exception hierarchy. - **LLM Agent** (`type: llm`, §4.4): Agent backed by language models with support for OpenAI, Anthropic, and Google Gemini providers. Configurable temperature, max_tokens, system prompts, memory/history, and structured output (json_mode/response_format). - **Tool Agent** (`type: tool`, §4.5): Deterministic agent executing built-in tools (echo, math, json_parse, http_request, file_read, file_write, progress_bar) and custom inline code tools. Supports safe/unsafe execution modes, shell command filtering, and file operation sandboxing. @@ -105,6 +97,10 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm ### Fixed +- **Root-logger-level and handler-level test isolation** (`features/environment.py`): `before_scenario` now saves the root logger's current level and all handler levels; `after_scenario` restores both. `ReactiveCleverAgentsApp.__init__` sets `logging.getLogger().setLevel(log_level)` globally and also sets each handler's level based on the `verbose` parameter (`verbose=0` → CRITICAL, `verbose=1` → ERROR), which contaminated the logging state for later scenarios in the same slipcover process. This caused 5 BDD scenarios in `execute_stream.feature` to fail in `nox -s coverage_report` (while passing in `nox -s unit_tests` due to process isolation differences). The fix follows the same save/restore pattern used by all other per-logger blocks in the file. (issue #17) +- **Registry Resolution Bugfixes (PR Review rui.hu)**: Fixed `TemplateRegistry._instantiate_from_registry_ref` hardcoding `package_type="template"` instead of threading the template type, which broke all non-template registry resolutions. `_try_parse_registry_ref` now filters to `ReferenceType.REGISTRY` only, preserving the "local templates unaffected" acceptance criterion. `_original_reference` in resolved results now stores the verbatim original reference string, not the cache-key with type suffix. `application.py` template-type mapping extended to cover all 8 types (previously only AGENT/GRAPH/STREAM — the other 5 silently misclassified as STREAM). Duplicated `_instantiate_from_registry_ref`/`_try_parse_registry_ref` logic extracted into shared `_resolve_registry_ref` helper in `base.py`. `RegistryClient` now URL-encodes namespace/name path components to prevent injection. HTTPS enforcement with `allow_insecure` flag per spec §12.2. `ReferenceResolver.close_all()` closes clients under the async lock to prevent concurrent client leak. Async lock lazily initialised to avoid deprecation warning. Added `plural_name` property to `TemplateType`. +- **Registry Error Hierarchy Fixes (PR Review rui.hu Round 5)**: Fixed `details` dropping in non-5xx fallback path of `_request()` — `details` extracted from structured error bodies are now propagated to `RegistryError` for all unrecognised error types. Added `asyncio.Lock` to `RegistryClient._get_client()` to match the project-standardised check-and-create pattern used by `ReferenceResolver._get_client()`. Fixed a no-op BDD scenario ("404 without error type in body falls back to PackageNotFoundError") that had no `Then` step. Fixed HTTPS logging step definitions that constructed a new `RegistryClient` internally, defeating test isolation. Enhanced the "Client with API key" scenario to verify the `Authorization` header is present. Added end-to-end BDD scenario for `details` propagation from structured error bodies. Added `quote(package_id)` URL-encoding to `get_package()`. Replaced misleading `if exc.request is not None:` with `if getattr(exc, "_request", None) is not None:`. Simplified `RegistryNetworkError.__str__` using parts list + join. Replaced deprecated `asyncio.get_event_loop().run_until_complete()` with `asyncio.run()`. Tightened `_CLASS_MAP` type annotation to `dict[str, type[CleverAgentsException]]`. Added comment to `typing.cast(int, code)` explaining intentional non-int test. +- **Registry Reference Resolution**: Fixed `ReferenceResolver._resolve_registry_async` passing the package name instead of the type code to `RegistryClient.resolve_package`, which would construct invalid API paths and break all registry integrations. Added `package_type` parameter to `resolve()` and `aresolve()`, with automatic mapping from `TemplateType` values to `PackageType` codes. Replaced `ValueError` with `TemplateError` for all registry fetch failure paths per acceptance criteria. Cache and client-pool operations now use `asyncio.Lock` in async coroutines (`aresolve`, `_resolve_registry_async`, `_get_client`, `close_all`) to prevent event-loop blocking, while retaining `threading.Lock` for the synchronous `resolve()` path. Use bounded `OrderedDict` with LRU eviction and consistently return deep copies to prevent cache poisoning. Fixed `_get_client()` race condition by protecting the full check-and-create under a single lock acquisition. Ad-hoc `ReferenceResolver` instances created during `_instantiate_from_registry_ref` are now properly closed in both success and error paths. Registry reference detection now uses proper `PackageReference.from_string()` parsing instead of a fragile `":" in template_name` check, and returns PackageReference for all reference types (LOCAL, ID, REGISTRY) enabling resolution of local and ID references through `instantiate_from_config` and `EnhancedTemplateRegistry.instantiate()`. Template `params` from registry-based configs are now applied to resolved content instead of being silently discarded. Added async `aresolve()` entry point to prevent `RuntimeError` when called from running event loops; `resolve()` now detects running loops and directs callers to `aresolve()`. Bare `except Exception: pass` blocks replaced with debug-level logging. - **Canonicalizer depth guard**: Fixed DoS vector where deeply nested lists bypassed the `max_depth` recursion protection. The depth check now fires in `_transform()` before any recursion, covering dicts, lists, and all value types uniformly. - **Canonicalizer benchmark OOM**: Fixed exponential tree fixture in `benchmarks/canonicalizer_benchmark.py` where `_make_nested(depth=30, width=2)` created ~2 billion nodes. Changed to `width=1` for a true 30-level linear chain. - **Canonicalizer float normalization**: Added negative-zero normalization (`-0.0` → `0.0`) in `_transform()` for strict RFC-8785 compliance. diff --git a/README.md b/README.md index b9b9bfd..da466de 100644 --- a/README.md +++ b/README.md @@ -66,22 +66,294 @@ stream = router.create_stream({"name": "pipeline", "type": StreamType.HOT}) router.send_message("pipeline", "Process this message") ``` +## Router-facing API + +CleverActors v2.1.0 exposes a stable router-facing API surface that lets the +[CleverRouter](https://git.cleverthis.com/cleverrouter/cleverrouter) validate +actor configurations, merge layered configs, and execute actors without +importing LangChain or LangGraph directly (see ADR-2024). + +### Exports at a glance + +| Name | Kind | One-line description | +|------|------|---------------------| +| `validate_dict` | function | Validate an actor config dict against the Actor Configuration Standard; raise `ConfigurationError` on failure. | +| `merge_configs` | function | Deep-merge zero or more config dicts per the §3.1 merge algorithm (absent→add, mapping→recurse, list→append, else→replace). | +| `create_executor` | function | Build an `Executor` from a validated config dict, per-request credentials, and optional execution limits and pricing table. | +| `Executor` | class | Runnable actor handle returned by `create_executor()`; call `execute()` or `execute_stream()`. | +| `ActorResult` | dataclass | Returned by `Executor.execute()`; carries `response`, `prompt_tokens`, `completion_tokens`, and per-node `nodes`. | +| `NodeUsage` | dataclass | One element of `ActorResult.nodes`; carries `node_id`, `provider`, `model`, `prompt_tokens`, `completion_tokens`. | +| `ExecutionError` | exception | Raised by `execute()` / `execute_stream()` on limit breaches; has `.kind` and `.reason` attributes. | +| `ConfigurationError` | exception | Raised by `validate_dict()` and `create_executor()` on invalid configuration. | + +### End-to-end usage example + +```python +import json +import yaml +from cleveractors import ( + validate_dict, + merge_configs, + create_executor, + ActorResult, + NodeUsage, + ExecutionError, + ConfigurationError, +) + +# ── Upload time: validate and store ───────────────────────────────────────── +platform_defaults = {"max_graph_depth": 5, "max_subgraph_depth": 3} +platform_limits = merge_configs(platform_defaults, {"max_total_nodes": 50}) + +actor_yaml_text = "..." # YAML string from the upload request +raw_config = yaml.safe_load(actor_yaml_text) # parsed by the router +validated = validate_dict(raw_config, platform_limits) # raises ConfigurationError on failure +compiled_ir = json.dumps(validated) # store in actors.compiled_ir + +# ── Request time: execute ──────────────────────────────────────────────────── +config_dict = json.loads(compiled_ir) +credentials = { + "openai": {"api_key": "sk-..."}, + "anthropic": {"api_key": "sk-ant-..."}, +} +limits = { + "max_depth": 5, + "max_model_calls": 10, + "max_tool_calls": 10, + "timeout_ms": 60_000, + "max_cost_usd": 1.00, +} +pricing = { + "openai": { + "gpt-4.1-mini": {"prompt": 0.15, "completion": 0.60}, + }, +} + +executor = create_executor(config_dict, credentials, limits, pricing) + +try: + result: ActorResult = await executor.execute(message="What is 2+2?") +except ExecutionError as exc: + # exc.kind → "depth" | "model_calls" | "tool_calls" | "timeout" | "cost" | "" + # exc.reason → "budget_exhausted" | "missing_pricing_entry" | "" + handle_error(exc) + raise + +# Use the result +print(result.response) # "4" +print(result.prompt_tokens) # total input tokens, all nodes +print(result.completion_tokens) # total output tokens, all nodes + +for node in result.nodes: # ActorResult.nodes: list[NodeUsage] + assert isinstance(node, NodeUsage) + node_cost = ( + pricing[node.provider][node.model]["prompt"] * node.prompt_tokens / 1_000_000 + + pricing[node.provider][node.model]["completion"] * node.completion_tokens / 1_000_000 + ) + print(f"{node.node_id} ({node.provider}/{node.model}): ${node_cost:.6f}") +``` + +### `validate_dict(config_dict, platform_limits)` + +Validates a Python dict (already loaded from YAML) against the +Actor Configuration Standard and enforces platform-level structural constraints. + +Accepts two config shapes: + +- **Spec-level v1.0** (`agents` key present) — full Actor Configuration Standard + checks: agent types, LLM provider allowlist, route/edge field names, graph node + types, stream operators, and structural depth/count limits. +- **Actor-level (CleverAgents v2.0 / runtime format)** (no `agents` key) — validates + a single actor runtime config; the `type` field is inferred from structure if + absent (`routes` → `graph`, `actors` → `multi_actor`). + +Returns `config_dict` unchanged on success; raises `ConfigurationError` on any failure. + +```python +from cleveractors import validate_dict, ConfigurationError + +try: + validate_dict(config_dict, {"max_total_nodes": 50}) +except ConfigurationError as exc: + print(f"Invalid config: {exc}") +``` + +### `merge_configs(*dicts)` + +Deep-merges zero or more config dicts left-to-right using the §3.1 algorithm. +Inputs are never mutated; a fresh dict is returned. + +Merge rules: +- Key absent from accumulated result → key-value pair added as-is. +- Both values are mappings → recursively deep-merged. +- Both values are sequences → new sequence appended to existing. +- Any other combination → new value replaces the accumulated value. + +```python +from cleveractors import merge_configs + +base = {"model": "gpt-4o", "tools": ["search"]} +override = {"model": "gpt-4.1-mini", "tools": ["calc"], "temperature": 0.7} +merged = merge_configs(base, override) +# → {"model": "gpt-4.1-mini", "tools": ["search", "calc"], "temperature": 0.7} +``` + +### `create_executor(config_dict, credentials, limits=None, pricing=None)` and `Executor` + +`create_executor()` wraps `PureLangGraph`, `AgentFactory`, and all agent +implementations behind a single stable handle. Credentials, limits, and the +pricing table are per-request; construct a **fresh executor per request** +(ADR-2024). + +> ⚠️ **Always pass `limits` and `pricing` in production.** When omitted, no +> execution-limit enforcement is performed (no timeout, depth, cost, or call-count +> checks). The router MUST pass both on every request per ADR-2024. + +```python +from cleveractors import create_executor + +executor = create_executor( + config_dict, + credentials={"openai": {"api_key": "sk-..."}}, + limits={"timeout_ms": 30_000, "max_cost_usd": 0.50}, + pricing={"openai": {"gpt-4.1-mini": {"prompt": 0.15, "completion": 0.60}}}, +) +result = await executor.execute("Summarise this document.") +``` + +#### Streaming with `execute_stream()` + +For long-running LLM calls, stream tokens progressively: + +```python +from cleveractors import ExecutionError + +try: + async for token in executor.execute_stream("Tell me a story."): + send_sse_token(token) + # After the async iterator is exhausted, billing data is available: + result = executor.last_result # ActorResult | None + if result: + bill_from_result(result) +except ExecutionError as exc: + # last_result is populated even on limit breaches — bill for partial usage. + partial = executor.last_result + if partial: + bill_from_result(partial) + raise +``` + +**`last_result` semantics:** + +- **Happy path** — iterator fully exhausted: `last_result` is a complete `ActorResult` + with all token counts and `nodes` populated. +- **Early abandonment** (client disconnect, `break`): the iterator is not exhausted, + so `last_result` remains `None` — no billing data is available. +- **`ExecutionError` (limit breach — including `timeout`, `depth`, `cost`, etc.)**: + `last_result` is populated with a *partial* `ActorResult` before the exception + propagates — billing still works for partial consumption. Always check `last_result` + in your `except ExecutionError` handler. + +Supported actor types: `"llm"` and `"graph"`. Raises `ConfigurationError` +for `"tool"` and `"multi_actor"`. + +### `ActorResult` and `NodeUsage` + +```python +from cleveractors import ActorResult, NodeUsage + +# ActorResult fields: +# response: str — final assistant message +# prompt_tokens: int — total input tokens across all LLM nodes +# completion_tokens: int — total output tokens across all LLM nodes +# nodes: list[NodeUsage] +# state: dict[str, Any] | None — opaque client-carried graph state +# (graph actors only; for stateless resumption per ADR-2026) + +# Stateless resumption (graph actors): pass the previous result's state back +# as the `state` keyword argument to resume the graph from where it left off: +# next_result = await executor.execute(message, state=prev_result.state) + +# NodeUsage fields: +# node_id: str — node identifier from the actor graph +# provider: str — e.g. "openai", "anthropic", "openai_compatible" +# model: str — e.g. "gpt-4.1-mini", "claude-3-5-haiku" +# prompt_tokens: int — input tokens for this node +# completion_tokens: int — output tokens for this node +``` + +`result.nodes` always contains at least one element. For single-LLM actors it +has exactly one entry. For graph actors it has one entry per LLM node that was +executed. If the graph contains no LLM nodes a zero-usage synthetic placeholder +is inserted. + +### `ExecutionError` and `ConfigurationError` + +`ExecutionError` is raised when an execution limit is breached or an actor +runtime error occurs. It carries two attributes in addition to the message: + +| Attribute | Values | Meaning | +|-----------|--------|---------| +| `.kind` | `"depth"` | Actor graph exceeded `max_depth` iterations. | +| | `"model_calls"` | Number of LLM invocations exceeded `max_model_calls`. | +| | `"tool_calls"` | Number of tool invocations exceeded `max_tool_calls`. | +| | `"timeout"` | Wall-clock time exceeded `timeout_ms`. | +| | `"cost"` | Accumulated cost exceeded `max_cost_usd`, or pricing entry missing. | +| | `""` | Non-limit runtime error (default). | +| `.reason` | `"budget_exhausted"` | Accumulated cost exceeded `max_cost_usd`. Return HTTP 429. | +| | `"missing_pricing_entry"` | Pricing table lacks an entry for the provider/model used. Return HTTP 500. | +| | `""` | All non-cost errors (default). | + +```python +from cleveractors import ExecutionError, ConfigurationError + +try: + result = await executor.execute(message) +except ExecutionError as exc: + if exc.kind == "cost" and exc.reason == "budget_exhausted": + return http_429("Budget exhausted") + if exc.kind == "cost" and exc.reason == "missing_pricing_entry": + return http_500("Pricing entry missing") + return http_500(str(exc)) +except ConfigurationError as exc: + return http_400(str(exc)) +``` + +`ConfigurationError` is raised by `validate_dict()` when the supplied config +dict fails any Actor Configuration Standard check, and by `create_executor()` +when the config dict or constructor arguments are invalid. + ## Package structure | Module | Purpose | |--------|---------| -| `cleveractors` | Top-level exports: `Agent`, `ContextManager`, `ReactiveCleverAgentsApp`, `CleverAgentsException` | +| `cleveractors` | Top-level exports (v2.1+): `validate_dict`, `merge_configs`, `create_executor`, `Executor`, `ActorResult`, `NodeUsage`, `ExecutionError`, `ConfigurationError`, `Agent`, `ContextManager`, `ReactiveCleverAgentsApp`, `CleverAgentsException` | | `cleveractors.agents` | Agent implementations: `LLMAgent`, `ToolAgent`, `CompositeAgent`, `ChainAgent`, `AgentFactory` | | `cleveractors.core` | Core framework: `ReactiveCleverAgentsApp`, `ConfigurationManager`, `ProgressBarManager`, exceptions | -| `cleveractors.langgraph` | LangGraph integration: `LangGraph`, `PureLangGraph`, `Node`, `GraphState`, `StateManager`, `RxPyLangGraphBridge` | +| `cleveractors.langgraph` | LangGraph integration: `LangGraph`, `PureLangGraph`, `Node`, `NodeType`, `GraphState`, `StateManager`, `RxPyLangGraphBridge` | | `cleveractors.reactive` | RxPy streams: `ReactiveStreamRouter`, `StreamMessage`, `RouteConfig`, `ReactiveConfigParser` | -| `cleveractors.templates` | Jinja2+YAML template system: `BaseTemplate`, `TemplateRegistry`, `AgentTemplate`, `GraphTemplate`, `StreamTemplate` | +| `cleveractors.templates` | Jinja2+YAML template system: `BaseTemplate`, `GenericTemplate`, `TemplateType`, `TemplateParameter`, `TemplateRegistry`, `ComponentReference`, `InstantiationContext` | ## Key exports ```python +# Router-facing API (v2.1+) — single-import surface +from cleveractors import ( + validate_dict, + merge_configs, + create_executor, + Executor, + ActorResult, + NodeUsage, + ExecutionError, + ConfigurationError, +) + +# Legacy CLI-facing surface from cleveractors import Agent, ContextManager, ReactiveCleverAgentsApp, CleverAgentsException -from cleveractors.core.exceptions import ConfigurationError, TemplateError, RoutingError, ExecutionError + +# Sub-module imports for advanced use +from cleveractors.core.exceptions import TemplateError, RoutingError from cleveractors.core.config import ConfigurationManager from cleveractors.agents.factory import AgentFactory from cleveractors.langgraph import LangGraph, Node, NodeType, GraphState, StateManager diff --git a/features/environment.py b/features/environment.py index f7babb6..480c852 100644 --- a/features/environment.py +++ b/features/environment.py @@ -3,6 +3,7 @@ BDD test environment setup for reactive CleverAgents. """ import asyncio +import logging import os import shutil import sys @@ -52,6 +53,23 @@ def after_all(context): def before_scenario(context, scenario): """Set up before each scenario.""" + # Save root logger level, handler levels, and the handler set as the very + # first action so the restore in after_scenario is guaranteed to have saved + # values even if any of the subsequent setup lines raise. + # ReactiveCleverAgentsApp.__init__ sets the root logger level AND each + # handler's level globally based on the verbose parameter; without this + # save/restore, a test that creates the app with verbose=0 contaminates + # subsequent tests that rely on WARNING-level log records propagating. + # The handler-set snapshot also guards against handlers added during the + # scenario (e.g. StreamHandler added by ReactiveCleverAgentsApp.__init__) + # leaking into subsequent scenarios. + _root_logger = logging.getLogger() + context._root_log_original_level = _root_logger.level + context._root_log_original_handler_levels = [ + (h, h.level) for h in list(_root_logger.handlers) + ] + context._root_log_original_handlers = set(_root_logger.handlers) + context.config_files = [] context.app = None context.result = None @@ -90,6 +108,41 @@ def before_scenario(context, scenario): def after_scenario(context, scenario): """Clean up after each scenario.""" + # Restore root logger level and handler levels as the very first action so + # the isolation guarantee is independent of any exception in the cleanup + # blocks below. + if hasattr(context, "_root_log_original_level"): + try: + logging.getLogger().setLevel(context._root_log_original_level) + except Exception: + # Swallow intentionally: test isolation must not fail the test run. + pass + if hasattr(context, "_root_log_original_handler_levels"): + for h, lvl in context._root_log_original_handler_levels: + try: + h.setLevel(lvl) + except Exception: + # Swallow intentionally: test isolation must not fail the test run. + pass + # Remove any handlers that were added during the scenario (e.g. StreamHandler + # added by ReactiveCleverAgentsApp.__init__) to prevent handler leaks across + # scenarios. This makes the workaround fully hermetic against the + # production-side bug tracked in #49. + if hasattr(context, "_root_log_original_handlers"): + _root_logger = logging.getLogger() + try: + current_handlers = set(_root_logger.handlers) + leaked = current_handlers - context._root_log_original_handlers + for h in leaked: + try: + _root_logger.removeHandler(h) + except Exception: + # Swallow intentionally: test isolation must not fail the test run. + pass + except Exception: + # Swallow intentionally: test isolation must not fail the test run. + pass + # Clean up bridge resources if present if hasattr(context, "bridge") and context.bridge: try: diff --git a/features/runtime_coverage.feature b/features/runtime_coverage.feature index 8a8b62d..8f0567e 100644 --- a/features/runtime_coverage.feature +++ b/features/runtime_coverage.feature @@ -26,6 +26,35 @@ Feature: Runtime Executor API And the executor limits should be an empty dictionary And the executor pricing should be an empty dictionary + # --------------------------------------------------------------------------- + # Executor.__init__ None-handling: direct construction (not via factory) + # These scenarios exercise the None→{} code paths in Executor.__init__ + # directly, without going through create_executor (which would pre-convert + # None to {} before the constructor is called). + # --------------------------------------------------------------------------- + + Scenario: Executor constructed directly without limits or pricing defaults to empty dicts + Given a valid actor config dict with type "llm" + And credentials dict with openai provider + When I construct an Executor directly without limits or pricing kwargs + Then an Executor instance should be returned + And the executor limits should be an empty dictionary + And the executor pricing should be an empty dictionary + + Scenario: Executor constructed directly with limits=None defaults limits to empty dict + Given a valid actor config dict with type "llm" + And credentials dict with openai provider + When I construct an Executor directly with limits=None + Then an Executor instance should be returned + And the executor limits should be an empty dictionary + + Scenario: Executor constructed directly with pricing=None defaults pricing to empty dict + Given a valid actor config dict with type "llm" + And credentials dict with openai provider + When I construct an Executor directly with pricing=None + Then an Executor instance should be returned + And the executor pricing should be an empty dictionary + Scenario: execute dispatches to LLM agent when type is "llm" Given a valid actor config dict with type "llm" and openai provider And credentials dict with openai api key diff --git a/features/steps/runtime_coverage_steps.py b/features/steps/runtime_coverage_steps.py index 8607b10..a0902ed 100644 --- a/features/steps/runtime_coverage_steps.py +++ b/features/steps/runtime_coverage_steps.py @@ -46,6 +46,35 @@ def step_call_create_executor_none(context: Any) -> None: ) +@when("I construct an Executor directly without limits or pricing kwargs") +def step_construct_executor_no_kwargs(context: Any) -> None: + """Construct Executor directly, omitting limits and pricing entirely.""" + context.test_executor = Executor( + config_dict=context.config_dict, + credentials=context.credentials, + ) + + +@when("I construct an Executor directly with limits=None") +def step_construct_executor_limits_none(context: Any) -> None: + """Construct Executor directly with limits=None to exercise the None→{} path.""" + context.test_executor = Executor( + config_dict=context.config_dict, + credentials=context.credentials, + limits=None, + ) + + +@when("I construct an Executor directly with pricing=None") +def step_construct_executor_pricing_none(context: Any) -> None: + """Construct Executor directly with pricing=None to exercise the None→{} path.""" + context.test_executor = Executor( + config_dict=context.config_dict, + credentials=context.credentials, + pricing=None, + ) + + @when("I execute the actor with message {msg}") @async_run_until_complete async def step_execute_actor(context: Any, msg: str) -> None: diff --git a/features/steps/templates_steps.py b/features/steps/templates_steps.py index 6e6d372..5d67ae1 100644 --- a/features/steps/templates_steps.py +++ b/features/steps/templates_steps.py @@ -438,3 +438,46 @@ def step_advanced_yaml_features_work(context): def step_schema_validation_applied(context): """Verify schema validation is applied.""" assert hasattr(context, "smart_loader_module") + + +# ── TemplateRegistry top-level re-export smoke test (issue #17) ───────────── + + +@when("I import TemplateRegistry from cleveractors.templates") +def step_tc_import_template_registry(context) -> None: + """Import TemplateRegistry from the cleveractors.templates package.""" + try: + from cleveractors.templates import TemplateRegistry as ImportedTR + + context.tc_imported_class = ImportedTR + context.tc_import_error = None + except ImportError as exc: + context.tc_imported_class = None + context.tc_import_error = exc + + +@then("the import should succeed without error") +def step_tc_import_succeeds(context) -> None: + """Verify the import raised no exception.""" + assert context.tc_import_error is None, ( + f"Import of TemplateRegistry from cleveractors.templates raised: " + f"{context.tc_import_error!r}" + ) + assert context.tc_imported_class is not None, ( + "TemplateRegistry imported as None" + ) + + +@then( + "the imported TemplateRegistry should be the same object as " + "cleveractors.templates.registry.TemplateRegistry" +) +def step_tc_same_object(context) -> None: + """Verify the re-exported class is identical to the canonical definition.""" + from cleveractors.templates.registry import TemplateRegistry as CanonicalTR + + assert context.tc_imported_class is CanonicalTR, ( + f"cleveractors.templates.TemplateRegistry is not the same object as " + f"cleveractors.templates.registry.TemplateRegistry: " + f"{context.tc_imported_class!r} vs {CanonicalTR!r}" + ) diff --git a/features/templates_coverage.feature b/features/templates_coverage.feature index c5acd50..0e03c67 100644 --- a/features/templates_coverage.feature +++ b/features/templates_coverage.feature @@ -65,3 +65,8 @@ Feature: Template System Integration When I use the smart YAML loader Then advanced YAML features should work And schema validation should be applied + + Scenario: TemplateRegistry is importable from cleveractors.templates and is the same class as cleveractors.templates.registry.TemplateRegistry + When I import TemplateRegistry from cleveractors.templates + Then the import should succeed without error + And the imported TemplateRegistry should be the same object as cleveractors.templates.registry.TemplateRegistry diff --git a/src/cleveractors/runtime.py b/src/cleveractors/runtime.py index c75d957..d452b84 100644 --- a/src/cleveractors/runtime.py +++ b/src/cleveractors/runtime.py @@ -1,7 +1,11 @@ """Router-facing runtime API for cleveractors-core. This module provides the public API that the CleverThis router consumes: - - create_executor(config_dict, credentials, limits, pricing) + - create_executor(config_dict, credentials, limits=None, pricing=None) + +Omitting ``limits`` or ``pricing`` disables all execution-limit enforcement +(no timeout, depth, cost, or call-count checks). The router MUST pass both +on every request per ADR-2024. The ``ActorResult`` and ``NodeUsage`` dataclasses live in ``cleveractors.result`` (AC1 of issue #14) and are re-exported here for @@ -53,21 +57,21 @@ class Executor: self, config_dict: dict[str, Any], credentials: dict[str, Any] | None, - limits: dict[str, Any], - pricing: dict[str, Any], + limits: dict[str, Any] | None = None, + pricing: dict[str, Any] | None = None, ): if not isinstance(config_dict, dict): raise ConfigurationError("config_dict must be a dict") if credentials is not None and not isinstance(credentials, dict): raise ConfigurationError("credentials must be a dict") - if not isinstance(limits, dict): + if limits is not None and not isinstance(limits, dict): raise ConfigurationError("limits must be a dict") - if not isinstance(pricing, dict): + if pricing is not None and not isinstance(pricing, dict): raise ConfigurationError("pricing must be a dict") self.config = config_dict self.credentials = credentials - self.limits = limits - self.pricing = pricing + self.limits: dict[str, Any] = limits or {} + self.pricing: dict[str, Any] = pricing or {} # Populated by execute_stream() after the async iterator is exhausted. # Remains None until the stream completes (AC4, issue #16). self.last_result: ActorResult | None = None @@ -240,7 +244,11 @@ def create_executor( (e.g. ``{"openai": {"api_key": "sk-..."}}``). limits: Optional execution limits (max_depth, max_model_calls, max_tool_calls, timeout_ms, max_cost_usd). + ⚠️ Per ADR-2024, the router MUST pass both on every request. + Omitting either disables timeout / depth / cost / call-count enforcement. pricing: Optional pricing table for cost enforcement. + ⚠️ Per ADR-2024, the router MUST pass both on every request. + Omitting either disables timeout / depth / cost / call-count enforcement. Returns: An :class:`Executor` instance. Call ``await executor.execute(message)`` @@ -249,6 +257,6 @@ def create_executor( return Executor( config_dict=config_dict, credentials=credentials, - limits=limits or {}, - pricing=pricing or {}, + limits=limits, + pricing=pricing, ) diff --git a/src/cleveractors/templates/__init__.py b/src/cleveractors/templates/__init__.py index f8b275b..b52f6aa 100644 --- a/src/cleveractors/templates/__init__.py +++ b/src/cleveractors/templates/__init__.py @@ -14,12 +14,14 @@ from cleveractors.templates.base import ( TemplateType, ) from cleveractors.templates.generic_template import GenericTemplate +from cleveractors.templates.registry import TemplateRegistry __all__ = [ "BaseTemplate", - "GenericTemplate", - "TemplateParameter", - "TemplateType", "ComponentReference", + "GenericTemplate", "InstantiationContext", + "TemplateParameter", + "TemplateRegistry", + "TemplateType", ]