diff --git a/CHANGELOG.md b/CHANGELOG.md index 7706848..fa47cf0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm - **`merge_configs()` public API** (`cleveractors.merge_configs`): New module-level function implementing the Actor Configuration Standard §3.1 deep-merge algorithm. Accepts an arbitrary number of `dict[str, Any]` arguments and returns a fresh merged dict without mutating any input. Merge semantics: absent key → add; both mappings → deep-merge recursively; both sequences → append; otherwise → replace. Zero-argument call returns `{}`. Exported from `cleveractors.__init__` and `__all__`. - **`validate_dict()` public API** (`cleveractors.validate_dict`): New router-facing validation function that validates a spec-conformant Actor Configuration Standard v1.0.0 dict against the full schema and enforces platform-level structural constraints. Validates top-level key presence (`agents`, `routes`), agent types, LLM provider allowlists, route/edge field names (`source`/`target` only — legacy `from`/`to` rejected), node types, stream operator types, and structural limits (`max_graph_depth`, `max_subgraph_depth`, `max_total_nodes`) from the supplied `platform_limits` dict. Returns the dict unchanged when valid; raises `ConfigurationError` on any violation. Pure static validator with no file I/O, env-var reads, or app construction. Exported from `cleveractors.__init__` and listed in `__all__`. (ADR-2024, ADR-2025, ADR-2029) - **Registry HTTP Client** (`cleveractors.registry`): Async registry client implementing the Package Registry Standard v1.0.0 HTTP API endpoints (§8) using `httpx.AsyncClient`. Supports all four endpoints: `GET /packages/{id}` (retrieve by PackageId), `GET /{type}/{ns}/{name}?version=` (reference resolution with version alias resolution per §4.2), `GET /browse` (discovery with type/namespace filters), and `GET /.well-known/cleverthis-packages` (registry metadata). Maps all 8 standard error types (§13.2) to typed exceptions via error-body type parsing with status-code fallback. Supports anonymous reads (§9.1) and optional API key authentication (§9.2). Includes async context manager support, 32 Behave BDD scenarios, 7 Robot Framework integration tests against a fake server, and ASV benchmarks. All registry modules achieve 100% coverage. +- **Canonicalizer** (`cleveractors.registry.Canonicalizer`): Deterministic canonicalization engine implementing Package Registry Standard v1.0.0 §6. Produces content-addressed SHA-1 hashes via `compute_package_id()` for all 8 package types. Applies Unicode NFC normalization to all string values, lexicographic key sorting for deterministic output, lifecycle field stripping (`version`, `release_date`), and RFC-8785 canonical JSON serialization. Internal reference resolution is integrated into the canonicalization pipeline (§6.2 step 5) — when configured, references are resolved before transformation. Includes `max_depth` recursion protection, argument validation on all public methods, `allow_nan=False` for valid JSON output, and ASV performance benchmarks. 31 Behave BDD scenarios (including error-path and integration tests) and 10 Robot Framework integration tests; canonical.py achieves 100% coverage. - **Core CleverActors Framework**: New agent-based LLM orchestration framework implementing the Actor Configuration Standard (§1-4). Includes agent base class, factory pattern for agent creation, configuration management, template rendering engine, and exception hierarchy. - **LLM Agent** (`type: llm`, §4.4): Agent backed by language models with support for OpenAI, Anthropic, and Google Gemini providers. Configurable temperature, max_tokens, system prompts, memory/history, and structured output (json_mode/response_format). - **Tool Agent** (`type: tool`, §4.5): Deterministic agent executing built-in tools (echo, math, json_parse, http_request, file_read, file_write, progress_bar) and custom inline code tools. Supports safe/unsafe execution modes, shell command filtering, and file operation sandboxing. @@ -57,9 +58,19 @@ and this project adheres to [Clever Semantic Versioning](https://www.w3.org/subm ### Fixed +- **Canonicalizer depth guard**: Fixed DoS vector where deeply nested lists bypassed the `max_depth` recursion protection. The depth check now fires in `_transform()` before any recursion, covering dicts, lists, and all value types uniformly. +- **Canonicalizer benchmark OOM**: Fixed exponential tree fixture in `benchmarks/canonicalizer_benchmark.py` where `_make_nested(depth=30, width=2)` created ~2 billion nodes. Changed to `width=1` for a true 30-level linear chain. +- **Canonicalizer float normalization**: Added negative-zero normalization (`-0.0` → `0.0`) in `_transform()` for strict RFC-8785 compliance. +- **Canonicalizer argument validation**: Added `max_depth` range validation in `__init__` and `package_type` type validation in `compute_package_id()`. +- **Canonicalizer variable shadowing**: Fixed parameter shadowing in `canonicalize()` — replaced `content = self.resolve_references(content)` with a separate `resolved_content` variable. - **Config safety in `Executor` dispatch methods**: ``_execute_llm`` replaces ``setdefault`` with direct assignment to correctly apply top-level config overrides (previously ``setdefault`` silently ignored user-specified ``provider``/``model``/etc. when the nested ``config:`` block had a conflicting value). ``_execute_graph`` uses ``copy.deepcopy(executor.config)`` passed to ``AgentFactory`` for defense-in-depth against nested-dict mutation (AC7 immutability invariant). ``_execute_multi_actor`` uses ``copy.deepcopy(sub_config)`` for nested-dict safety in sub-executor creation. All paths remain compliant with ADR-2026 AC8 (stored ``config_dict`` is never modified). - **Immutability in `Executor._execute_multi_actor`**: Replaced in-place `NodeUsage.node_id` mutation with `dataclasses.replace()` to avoid aliasing issues when the sub-executor result is reused. - **Mutable state hygiene in `Executor.execute()`**: Added `self._usage_log.clear()` at the start of `execute()` so no mutable state from a previous invocation persists between calls (AC5 compliance). +- **Version string**: Updated `__version__` from `"2.0.0"` to `"2.1.0"` to match `pyproject.toml`. +- **Robot Framework keywords**: Renamed `Result Contains Nfc Normalized Text` to `Result Contains NFC Normalized Text` for consistent acronym casing. +- **CI lint fix**: Removed triple blank line (ruff E303) in `features/steps/registry_canonicalization_steps.py`. +- **Integration test fix**: Updated `robot/version.robot` to expect version `2.1.0` matching the `pyproject.toml` bump. +- **Project metadata**: Updated `pyproject.toml` description string to reference `v2.1.0` instead of `v2.0.0`. - **Double Response Bug**: Fixed stream subscription duplication after merge/split operations that caused messages to be processed twice. - **Double Input Echo**: Fixed terminal echoing user input twice in interactive mode by correcting LangGraph bridge async handling with mutual exclusion locks. - **Graph Context Propagation**: Fixed context not flowing properly between graph nodes; LLM agents now receive conversation history from graph state. diff --git a/benchmarks/canonicalizer_benchmark.py b/benchmarks/canonicalizer_benchmark.py new file mode 100644 index 0000000..931eaec --- /dev/null +++ b/benchmarks/canonicalizer_benchmark.py @@ -0,0 +1,84 @@ +"""ASV benchmarks for the Canonicalizer. + +Measures the performance of canonicalize() and compute_package_id() +across small, medium, and large/deeply-nested content dictionaries. +""" + +from __future__ import annotations + +from typing import Any + +from cleveractors.registry.canonical import Canonicalizer +from cleveractors.registry.types import PackageType + + +class CanonicalizerBenchmark: + """Benchmark suite for Canonicalizer.canonicalize() and compute_package_id().""" + + def setup(self) -> None: + self.canon = Canonicalizer() + + self.small: dict[str, Any] = { + "name": "simple-actor", + "description": "A minimal actor for benchmarking", + } + + self.medium: dict[str, Any] = { + "name": "complex-actor", + "description": "An actor with many properties", + "type": "llm", + "model": "gpt-4", + "temperature": 0.7, + "max_tokens": 4096, + "metadata": { + "author": "Team", + "tags": ["ai", "nlp", "conversation"], + "license": "MIT", + }, + "parameters": { + "top_p": 0.9, + "frequency_penalty": 0.0, + "presence_penalty": 0.0, + }, + } + + self.large: dict[str, Any] = self._make_nested(depth=5, width=8, value=1) + + self.deep: dict[str, Any] = self._make_nested(depth=30, width=1, value=1) + + @staticmethod + def _make_nested(depth: int, width: int, value: int) -> dict[str, Any]: + if depth == 0: + return {f"leaf_{k}": value for k in range(width)} + return { + f"key_{k}": CanonicalizerBenchmark._make_nested(depth - 1, width, value) + for k in range(width) + } + + def time_canonicalize_small(self) -> None: + """Benchmark canonicalize() on a small content dict.""" + self.canon.canonicalize(self.small) + + def time_canonicalize_medium(self) -> None: + """Benchmark canonicalize() on a medium content dict.""" + self.canon.canonicalize(self.medium) + + def time_canonicalize_large(self) -> None: + """Benchmark canonicalize() on a large/deeply-nested dict.""" + self.canon.canonicalize(self.large) + + def time_canonicalize_deep(self) -> None: + """Benchmark canonicalize() on a deep (30-level) narrow dict.""" + self.canon.canonicalize(self.deep) + + def time_compute_package_id_small(self) -> None: + """Benchmark compute_package_id() on a small content dict.""" + self.canon.compute_package_id(self.small, PackageType.ACTOR) + + def time_compute_package_id_medium(self) -> None: + """Benchmark compute_package_id() on a medium content dict.""" + self.canon.compute_package_id(self.medium, PackageType.ACTOR) + + def time_compute_package_id_large(self) -> None: + """Benchmark compute_package_id() on a large/deeply-nested dict.""" + self.canon.compute_package_id(self.large, PackageType.ACTOR) diff --git a/features/registry_canonicalization.feature b/features/registry_canonicalization.feature new file mode 100644 index 0000000..196a5b5 --- /dev/null +++ b/features/registry_canonicalization.feature @@ -0,0 +1,216 @@ +Feature: Registry Canonicalization Engine + As a developer + I want a Canonicalizer that produces deterministic RFC-8785 JSON from package content + So that packages are content-addressed and the same content always produces the same PackageId + + Background: + Given I have imported the Canonicalizer + + # ── Canonicalization determinism ────────────────────────────────────────── + + Scenario: Same content always produces the same canonical JSON + Given a content dictionary with keys "name" and "description" + When I canonicalize the content + Then the canonical JSON should be the same on every invocation + + Scenario: Canonicalize produces valid JSON + Given a content dictionary with name "test-package" and description "A test package" + When I canonicalize the content + Then the canonical output should be valid JSON + + Scenario: Empty dict canonicalizes to empty object + Given an empty content dictionary + When I canonicalize the content + Then the canonical output should be an empty JSON object + + # ── Key sorting ─────────────────────────────────────────────────────────── + + Scenario: Key ordering does not affect the canonical form + Given a content dictionary with keys in order "b", "a" + When I canonicalize the content + Then the canonical output should have key "a" before key "b" + + Scenario: Nested dictionaries have sorted keys + Given a content dictionary with nested unsorted keys + When I canonicalize the content + Then the nested dictionary keys should be sorted in the output + + # ── Lifecycle field stripping ───────────────────────────────────────────── + + Scenario: Lifecycle field "version" is stripped from canonical form + Given a content dictionary with a "version" field set to "1.2.3" + When I canonicalize the content + Then the canonical output should not contain "version" + + Scenario: Lifecycle field "release_date" is stripped from canonical form + Given a content dictionary with a "release_date" field set to "2026-01-01" + When I canonicalize the content + Then the canonical output should not contain "release_date" + + Scenario: Lifecycle fields are stripped from nested dicts + Given a content dictionary with nested "version" and "release_date" fields + When I canonicalize the content + Then the canonical output should not contain any lifecycle fields + + # ── NFC normalization ───────────────────────────────────────────────────── + + Scenario: NFC normalization normalizes composed characters + Given a content dictionary with NFC-decomposed string value + When I canonicalize the content + Then the output should use the composed form + + Scenario: NFC normalization does not change already-composed strings + Given a content dictionary with NFC-composed string value + When I canonicalize the content + Then the output should contain the same composed form + + Scenario: NFC normalization is applied to nested string values + Given a content dictionary with nested decomposed string values + When I canonicalize the content + Then all string values should be NFC-normalized in the output + + # ── SHA-1 hashing and PackageId ─────────────────────────────────────────── + + Scenario: compute_package_id produces a valid PackageId + Given a content dictionary with name "hello-world" + When I compute the PackageId with package type ACTOR + Then the PackageId should have type ACTOR + And the PackageId should have a 40-character sha1_hex + And the PackageId id_string should start with "pkg_act_" + + Scenario: Same content produces the same PackageId + Given a content dictionary with name "immutable-test" + When I compute the PackageId with package type SKILL + And I compute the PackageId again with the same content and type + Then both PackageIds should be identical + + Scenario: Different content produces different PackageIds + Given a content dictionary with name "package-a" + And another content dictionary with name "package-b" + When I compute the PackageId for both with package type ACTOR + Then the two PackageIds should be different + + Scenario: Different package types produce different prefixes + Given a content dictionary with name "multi-type" + When I compute the PackageId with package types ACTOR and GRAPH + Then the Actor PackageId should start with "pkg_act_" + And the Graph PackageId should start with "pkg_grh_" + + # ── Non-string values ───────────────────────────────────────────────────── + + Scenario: Integer values are preserved in canonical form + Given a content dictionary with an integer field + When I canonicalize the content + Then the integer value should be preserved in the output + + Scenario: Boolean values are preserved in canonical form + Given a content dictionary with a boolean field + When I canonicalize the content + Then the boolean value should be preserved in the output + + Scenario: Null values are preserved in canonical form + Given a content dictionary with a null field + When I canonicalize the content + Then the null value should be preserved in the output + + Scenario: List values are preserved in canonical form + Given a content dictionary with a list field + When I canonicalize the content + Then the list should be preserved in order in the output + + # ── Reference resolution ────────────────────────────────────────────────── + + Scenario: resolve_references without a resolver returns unchanged content + Given a content dictionary with an internal ID reference + When I resolve references without a resolver + Then the resolved content should be unchanged + + Scenario: resolve_references with a resolver transforms ID references + Given a content dictionary with an internal ID reference + When I resolve references with a simple resolver + Then all ID references should be resolved + + Scenario: resolve_references resolves references in nested dicts + Given a content dictionary with nested ID references + When I resolve references with a simple resolver + Then all ID references in nested structures should be resolved + + Scenario: resolve_references resolves references in lists + Given a content dictionary with ID references in a list + When I resolve references with a simple resolver + Then all ID references in the list should be resolved + + Scenario: Canonicalization resolves references before computing canonical form + Given a content dictionary with an internal ID reference + When I canonicalize the content with a reference resolver + Then the canonical output should contain the resolved reference + + # ── Error paths ──────────────────────────────────────────────────────────── + + Scenario: canonicalize raises TypeError on None input + Given I have imported the Canonicalizer + When I attempt to canonicalize None content + Then canonicalize raises TypeError + + Scenario: canonicalize raises TypeError on non-dict input + Given I have imported the Canonicalizer + When I attempt to canonicalize a string "not-a-dict" + Then canonicalize raises TypeError + + Scenario: canonicalize raises TypeError on list input + Given I have imported the Canonicalizer + When I attempt to canonicalize a list + Then canonicalize raises TypeError + + Scenario: compute_package_id raises TypeError on None input + Given I have imported the Canonicalizer + When I attempt to compute_package_id with None content + Then compute_package_id raises TypeError + + Scenario: canonicalize raises ValueError when max depth exceeded + Given a Canonicalizer with max_depth 3 + When I canonicalize a deeply nested dict with depth 10 + Then canonicalize raises ValueError with message containing "Maximum nesting depth" + + Scenario: resolve_references raises TypeError on non-dict input + Given I have imported the Canonicalizer + When I attempt to resolve_references with None content + Then resolve_references raises TypeError + + Scenario: resolve_references raises ValueError when max depth exceeded + Given a Canonicalizer with max_depth 3 and a reference resolver + When I resolve references in a deeply nested dict with depth 10 + Then resolve_references raises ValueError with message containing "Maximum nesting depth" + + # ── Constructor argument validation ────────────────────────────────────── + + Scenario: Canonicalizer rejects negative max_depth + Given I have imported the Canonicalizer + When I attempt to create a Canonicalizer with max_depth -1 + Then canonicalize raises ValueError with message containing "max_depth must be a positive int" + + Scenario: Canonicalizer rejects non-int max_depth + Given I have imported the Canonicalizer + When I attempt to create a Canonicalizer with max_depth "not-an-int" + Then canonicalize raises ValueError with message containing "max_depth must be a positive int" + + # ── PackageType validation ──────────────────────────────────────────────── + + Scenario: compute_package_id raises TypeError on non-PackageType + Given I have imported the Canonicalizer + When I attempt to compute_package_id with content "some-package" and package_type None + Then compute_package_id raises TypeError + + # ── Deeply nested list max_depth enforcement ────────────────────────────── + + Scenario: canonicalize raises ValueError on deeply nested list exceeding max depth + Given a Canonicalizer with max_depth 3 + When I canonicalize a deeply nested list with depth 10 + Then canonicalize raises ValueError with message containing "Maximum nesting depth" + + # ── Negative zero float normalization ───────────────────────────────────── + + Scenario: Negative zero is normalized to positive zero in canonical output + Given I have imported the Canonicalizer + When I canonicalize content containing negative zero + Then the canonical output should serialize negative zero as positive zero diff --git a/features/steps/registry_canonicalization_steps.py b/features/steps/registry_canonicalization_steps.py new file mode 100644 index 0000000..1e8b50a --- /dev/null +++ b/features/steps/registry_canonicalization_steps.py @@ -0,0 +1,688 @@ +""" +Step definitions for Registry Canonicalization Engine BDD tests. +""" + +from __future__ import annotations + +import json +import unicodedata +from typing import Any + +from behave import given, then, when +from behave.runner import Context + +from cleveractors.registry.canonical import Canonicalizer +from cleveractors.registry.types import PackageType + +# ── Background ───────────────────────────────────────────────────────────── + + +@given("I have imported the Canonicalizer") +def step_import_canonicalizer(context: Context) -> None: + context.Canonicalizer = Canonicalizer + + +# ── Given steps ───────────────────────────────────────────────────────────── + + +@given('a content dictionary with keys "name" and "description"') +def step_content_dict_name_description(context: Context) -> None: + context.content = {"name": "test", "description": "A test package"} + + +@given('a content dictionary with name "test-package" and description "A test package"') +def step_content_dict_test_package(context: Context) -> None: + context.content = {"name": "test-package", "description": "A test package"} + + +@given("an empty content dictionary") +def step_empty_content_dict(context: Context) -> None: + context.content = {} + + +@given('a content dictionary with keys in order "b", "a"') +def step_content_dict_keys_ba(context: Context) -> None: + from collections import OrderedDict + + context.content = OrderedDict([("b", 1), ("a", 2)]) + + +@given("a content dictionary with nested unsorted keys") +def step_content_dict_nested_unsorted(context: Context) -> None: + from collections import OrderedDict + + context.content = { + "outer": OrderedDict([("z", 1), ("x", 2), ("y", 3)]), + } + + +@given('a content dictionary with a "version" field set to "1.2.3"') +def step_content_dict_with_version(context: Context) -> None: + context.content = {"name": "pkg", "version": "1.2.3"} + + +@given('a content dictionary with a "release_date" field set to "2026-01-01"') +def step_content_dict_with_release_date(context: Context) -> None: + context.content = {"name": "pkg", "release_date": "2026-01-01"} + + +@given('a content dictionary with nested "version" and "release_date" fields') +def step_content_dict_nested_lifecycle(context: Context) -> None: + context.content = { + "name": "pkg", + "meta": {"version": "1.0.0", "release_date": "2026-01-01"}, + } + + +@given("a content dictionary with NFC-decomposed string value") +def step_content_dict_decomposed(context: Context) -> None: + decomposed = unicodedata.normalize("NFD", "café résumé naïveté") + context.content = {"name": "test", "description": decomposed} + + +@given("a content dictionary with NFC-composed string value") +def step_content_dict_composed(context: Context) -> None: + composed = unicodedata.normalize("NFC", "café résumé naïveté") + context.content = {"name": "test", "description": composed} + + +@given("a content dictionary with nested decomposed string values") +def step_content_dict_nested_decomposed(context: Context) -> None: + decomposed = unicodedata.normalize("NFD", "voilà garçon") + context.content = { + "name": "test", + "strings": [decomposed, "normal"], + "nested": {"deep": decomposed}, + } + + +@given('a content dictionary with name "hello-world"') +def step_content_dict_hello_world(context: Context) -> None: + context.content = {"name": "hello-world", "description": "A test"} + + +@given('a content dictionary with name "immutable-test"') +def step_content_dict_immutable(context: Context) -> None: + context.content = {"name": "immutable-test", "value": 42} + + +@given('a content dictionary with name "package-a"') +def step_content_dict_package_a(context: Context) -> None: + context.content_a = {"name": "package-a", "value": 1} + + +@given('another content dictionary with name "package-b"') +def step_content_dict_package_b(context: Context) -> None: + context.content_b = {"name": "package-b", "value": 2} + + +@given('a content dictionary with name "multi-type"') +def step_content_dict_multi_type(context: Context) -> None: + context.content = {"name": "multi-type"} + + +@given("a content dictionary with an integer field") +def step_content_dict_integer(context: Context) -> None: + context.content = {"name": "test", "count": 42} + + +@given("a content dictionary with a boolean field") +def step_content_dict_boolean(context: Context) -> None: + context.content = {"name": "test", "enabled": True} + + +@given("a content dictionary with a null field") +def step_content_dict_null(context: Context) -> None: + context.content = {"name": "test", "optional": None} + + +@given("a content dictionary with a list field") +def step_content_dict_list(context: Context) -> None: + context.content = {"name": "test", "tags": ["alpha", "beta", "gamma"]} + + +# ── When steps ────────────────────────────────────────────────────────────── + + +@when("I canonicalize the content") +def step_when_canonicalize(context: Context) -> None: + canon = Canonicalizer() + context.canonical_output = canon.canonicalize(context.content) + + +@when("I compute the PackageId with package type {pkg_type}") +def step_when_compute_package_id(context: Context, pkg_type: str) -> None: + canon = Canonicalizer() + context.package_id = canon.compute_package_id( + context.content, PackageType[pkg_type.upper()] + ) + + +@when("I compute the PackageId again with the same content and type") +def step_when_compute_package_id_again(context: Context) -> None: + canon = Canonicalizer() + context.package_id_2 = canon.compute_package_id(context.content, PackageType.SKILL) + + +@when("I compute the PackageId for both with package type {pkg_type}") +def step_when_compute_both_package_ids(context: Context, pkg_type: str) -> None: + canon = Canonicalizer() + context.package_id_a = canon.compute_package_id( + context.content_a, PackageType[pkg_type.upper()] + ) + context.package_id_b = canon.compute_package_id( + context.content_b, PackageType[pkg_type.upper()] + ) + + +@when("I compute the PackageId with package types {pkg_type1} and {pkg_type2}") +def step_when_compute_both_types( + context: Context, pkg_type1: str, pkg_type2: str +) -> None: + canon = Canonicalizer() + context.package_id_actor = canon.compute_package_id( + context.content, PackageType[pkg_type1.upper()] + ) + context.package_id_graph = canon.compute_package_id( + context.content, PackageType[pkg_type2.upper()] + ) + + +# ── Then steps ────────────────────────────────────────────────────────────── + + +@then("the canonical JSON should be the same on every invocation") +def step_then_deterministic(context: Context) -> None: + canon = Canonicalizer() + output2 = canon.canonicalize(context.content) + assert context.canonical_output == output2, ( + f"Expected deterministic output but got different results:\n" + f"First: {context.canonical_output!r}\n" + f"Second: {output2!r}" + ) + + +@then("the canonical output should be valid JSON") +def step_then_valid_json(context: Context) -> None: + parsed = json.loads(context.canonical_output) + assert isinstance(parsed, dict), f"Expected dict, got {type(parsed).__name__}" + + +@then("the canonical output should be an empty JSON object") +def step_then_empty_object(context: Context) -> None: + assert context.canonical_output == "{}", ( + f"Expected '{{}}' but got {context.canonical_output!r}" + ) + + +@then('the canonical output should have key "a" before key "b"') +def step_then_keys_sorted(context: Context) -> None: + idx_a = context.canonical_output.index('"a"') + idx_b = context.canonical_output.index('"b"') + assert idx_a < idx_b, ( + f"Key 'a' should appear before 'b', but found 'a' at {idx_a} and 'b' at {idx_b}" + ) + + +@then("the nested dictionary keys should be sorted in the output") +def step_then_nested_keys_sorted(context: Context) -> None: + parsed = json.loads(context.canonical_output) + outer = parsed.get("outer", {}) + keys = list(outer.keys()) + assert keys == sorted(keys), f"Nested keys not sorted: {keys}" + + +@then('the canonical output should not contain "version"') +def step_then_no_version(context: Context) -> None: + assert '"version"' not in context.canonical_output, ( + f"Found 'version' in output: {context.canonical_output!r}" + ) + + +@then('the canonical output should not contain "release_date"') +def step_then_no_release_date(context: Context) -> None: + assert '"release_date"' not in context.canonical_output, ( + f"Found 'release_date' in output: {context.canonical_output!r}" + ) + + +@then("the canonical output should not contain any lifecycle fields") +def step_then_no_lifecycle_fields(context: Context) -> None: + output = context.canonical_output + for field in Canonicalizer.LIFECYCLE_FIELDS: + assert f'"{field}"' not in output, ( + f"Found lifecycle field {field!r} in output: {output!r}" + ) + + +@then("the output should use the composed form") +def step_then_composed_form(context: Context) -> None: + composed = unicodedata.normalize("NFC", "café résumé naïveté") + escaped_composed = json.dumps(composed, ensure_ascii=False) + assert escaped_composed in context.canonical_output, ( + f"Expected composed form in output: {context.canonical_output!r}" + ) + + +@then("the output should contain the same composed form") +def step_then_same_composed_form(context: Context) -> None: + composed = unicodedata.normalize("NFC", "café résumé naïveté") + escaped_composed = json.dumps(composed, ensure_ascii=False) + assert escaped_composed in context.canonical_output, ( + f"Expected same composed form: {context.canonical_output!r}" + ) + + +@then("all string values should be NFC-normalized in the output") +def step_then_all_nfc_normalized(context: Context) -> None: + parsed = json.loads(context.canonical_output) + + def check_nfc(value: Any) -> None: + if isinstance(value, str): + nfc = unicodedata.normalize("NFC", value) + assert value == nfc, f"String not NFC-normalized: {value!r} != {nfc!r}" + elif isinstance(value, dict): + for v in value.values(): + check_nfc(v) + elif isinstance(value, list): + for item in value: + check_nfc(item) + + check_nfc(parsed) + + +@then("the PackageId should have a 40-character sha1_hex") +def step_then_sha1_length(context: Context) -> None: + sha1 = context.package_id.sha1_hex + assert len(sha1) == 40, f"Expected 40 chars, got {len(sha1)}: {sha1!r}" + assert all(c in "0123456789abcdef" for c in sha1), ( + f"Expected all hex chars: {sha1!r}" + ) + + +@then('the PackageId id_string should start with "{prefix}"') +def step_then_id_string_prefix(context: Context, prefix: str) -> None: + assert context.package_id.id_string.startswith(prefix), ( + f"Expected id_string to start with {prefix!r}, " + f"got {context.package_id.id_string!r}" + ) + + +@then("both PackageIds should be identical") +def step_then_package_ids_identical(context: Context) -> None: + assert context.package_id == context.package_id_2, ( + f"Expected identical PackageIds, got:\n" + f" {context.package_id}\n" + f" {context.package_id_2}" + ) + + +@then("the two PackageIds should be different") +def step_then_package_ids_different(context: Context) -> None: + assert context.package_id_a != context.package_id_b, ( + f"Expected different PackageIds, but both were: {context.package_id_a}" + ) + + +@then('the Actor PackageId should start with "pkg_act_"') +def step_then_actor_prefix(context: Context) -> None: + assert context.package_id_actor.id_string.startswith("pkg_act_"), ( + f"Expected pkg_act_ prefix: {context.package_id_actor}" + ) + + +@then('the Graph PackageId should start with "pkg_grh_"') +def step_then_graph_prefix(context: Context) -> None: + assert context.package_id_graph.id_string.startswith("pkg_grh_"), ( + f"Expected pkg_grh_ prefix: {context.package_id_graph}" + ) + + +@then("the integer value should be preserved in the output") +def step_then_integer_preserved(context: Context) -> None: + parsed = json.loads(context.canonical_output) + assert parsed.get("count") == 42, f"Expected 42, got {parsed.get('count')!r}" + + +@then("the boolean value should be preserved in the output") +def step_then_boolean_preserved(context: Context) -> None: + parsed = json.loads(context.canonical_output) + assert parsed.get("enabled") is True, ( + f"Expected True, got {parsed.get('enabled')!r}" + ) + + +@then("the null value should be preserved in the output") +def step_then_null_preserved(context: Context) -> None: + parsed = json.loads(context.canonical_output) + assert parsed.get("optional") is None, ( + f"Expected None, got {parsed.get('optional')!r}" + ) + + +@then("the list should be preserved in order in the output") +def step_then_list_preserved(context: Context) -> None: + parsed = json.loads(context.canonical_output) + assert parsed.get("tags") == ["alpha", "beta", "gamma"], ( + f"Expected ['alpha', 'beta', 'gamma'], got {parsed.get('tags')!r}" + ) + + +# ── Reference resolution steps ──────────────────────────────────────────── + + +@given("a content dictionary with an internal ID reference") +def step_content_dict_id_ref(context: Context) -> None: + context.content = { + "name": "test", + "template_ref": "ID:pkg_tpl_a1b2c3d4e5f67890abcdef1234567890abcdef", + } + + +@given("a content dictionary with nested ID references") +def step_content_dict_nested_id_refs(context: Context) -> None: + context.content = { + "name": "test", + "deps": { + "primary": "ID:pkg_act_0000000000000000000000000000000000000001", + }, + } + + +@given("a content dictionary with ID references in a list") +def step_content_dict_list_id_refs(context: Context) -> None: + context.content = { + "name": "test", + "refs": [ + "ID:pkg_skl_1111111111111111111111111111111111111111", + "ID:pkg_mcp_2222222222222222222222222222222222222222", + ], + } + + +@when("I resolve references without a resolver") +def step_when_resolve_without_resolver(context: Context) -> None: + canon = Canonicalizer() + context.resolved = canon.resolve_references(context.content) + + +@when("I resolve references with a simple resolver") +def step_when_resolve_with_resolver(context: Context) -> None: + def simple_resolver(raw: str) -> str: + return raw + "-resolved" + + canon = Canonicalizer(reference_resolver=simple_resolver) + context.resolved = canon.resolve_references(context.content) + + +@then("the resolved content should be unchanged") +def step_then_resolved_unchanged(context: Context) -> None: + assert context.resolved == context.content, ( + f"Expected unchanged content, got {context.resolved!r}" + ) + + +@then("all ID references should be resolved") +def step_then_all_id_refs_resolved(context: Context) -> None: + ref_val = context.resolved.get("template_ref") + assert ref_val is not None, "Expected template_ref in resolved content" + assert ref_val.endswith("-resolved"), f"Expected resolved suffix, got {ref_val!r}" + + +@then("all ID references in nested structures should be resolved") +def step_then_nested_id_refs_resolved(context: Context) -> None: + deps = context.resolved.get("deps", {}) + primary = deps.get("primary") + assert primary is not None, "Expected deps.primary in resolved content" + assert primary.endswith("-resolved"), f"Expected resolved suffix, got {primary!r}" + + +@then("all ID references in the list should be resolved") +def step_then_list_id_refs_resolved(context: Context) -> None: + refs = context.resolved.get("refs", []) + assert len(refs) == 2, f"Expected 2 refs, got {len(refs)}" + for ref in refs: + assert ref.endswith("-resolved"), ( + f"Expected resolved suffix in list item, got {ref!r}" + ) + + +# ── Resolve + canonicalize integration steps ───────────────────────── + + +@when("I canonicalize the content with a reference resolver") +def step_when_canonicalize_with_resolver(context: Context) -> None: + def simple_resolver(raw: str) -> str: + return raw + "-resolved" + + canon = Canonicalizer(reference_resolver=simple_resolver) + context.canonical_output = canon.canonicalize(context.content) + + +@then("the canonical output should contain the resolved reference") +def step_then_canonical_has_resolved_ref(context: Context) -> None: + assert "-resolved" in context.canonical_output, ( + f"Expected resolved reference in output: {context.canonical_output!r}" + ) + + +# ── Error-path steps ─────────────────────────────────────────────────── + + +@when("I attempt to canonicalize None content") +def step_when_canonicalize_none(context: Context) -> None: + canon = Canonicalizer() + try: + canon.canonicalize(None) + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +@when('I attempt to canonicalize a string "not-a-dict"') +def step_when_canonicalize_string(context: Context) -> None: + canon = Canonicalizer() + try: + canon.canonicalize("not-a-dict") + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +@when("I attempt to canonicalize a list") +def step_when_canonicalize_list(context: Context) -> None: + canon = Canonicalizer() + try: + canon.canonicalize([1, 2, 3]) + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +@then("canonicalize raises TypeError") +def step_then_canonicalize_typeerror(context: Context) -> None: + assert context.canonicalize_error is not None, ( + "Expected TypeError, but no exception was raised" + ) + assert isinstance(context.canonicalize_error, TypeError), ( + f"Expected TypeError, got {type(context.canonicalize_error).__name__}" + ) + + +@when("I attempt to compute_package_id with None content") +def step_when_compute_id_none(context: Context) -> None: + canon = Canonicalizer() + try: + canon.compute_package_id(None, PackageType.ACTOR) + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +@then("compute_package_id raises TypeError") +def step_then_compute_id_typeerror(context: Context) -> None: + assert context.canonicalize_error is not None, ( + "Expected TypeError, but no exception was raised" + ) + assert isinstance(context.canonicalize_error, TypeError), ( + f"Expected TypeError, got {type(context.canonicalize_error).__name__}" + ) + + +@given("a Canonicalizer with max_depth 3") +def step_given_shallow_canonicalizer(context: Context) -> None: + context.shallow_canon = Canonicalizer(max_depth=3) + + +@when("I canonicalize a deeply nested dict with depth 10") +def step_when_canonicalize_deep(context: Context) -> None: + def _build_nested(depth: int): + if depth == 0: + return {"value": 1} + return {"nested": _build_nested(depth - 1)} + + deep_content = _build_nested(10) + try: + context.shallow_canon.canonicalize(deep_content) + context.canonicalize_error = None + except ValueError as exc: + context.canonicalize_error = exc + + +@then('canonicalize raises ValueError with message containing "{msg_fragment}"') +def step_then_valueerror_with_msg(context: Context, msg_fragment: str) -> None: + assert context.canonicalize_error is not None, ( + "Expected ValueError, but no exception was raised" + ) + assert isinstance(context.canonicalize_error, ValueError), ( + f"Expected ValueError, got {type(context.canonicalize_error).__name__}" + ) + assert msg_fragment in str(context.canonicalize_error), ( + f"Expected error to contain {msg_fragment!r}, " + f"got {context.canonicalize_error!r}" + ) + + +@when("I attempt to resolve_references with None content") +def step_when_resolve_none(context: Context) -> None: + canon = Canonicalizer(reference_resolver=lambda r: r) + try: + canon.resolve_references(None) + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +@then("resolve_references raises TypeError") +def step_then_resolve_typeerror(context: Context) -> None: + assert context.canonicalize_error is not None, ( + "Expected TypeError, but no exception was raised" + ) + assert isinstance(context.canonicalize_error, TypeError), ( + f"Expected TypeError, got {type(context.canonicalize_error).__name__}" + ) + + +@given("a Canonicalizer with max_depth 3 and a reference resolver") +def step_given_shallow_canonicalizer_with_resolver(context: Context) -> None: + context.shallow_canon = Canonicalizer(max_depth=3, reference_resolver=lambda r: r) + + +@when("I resolve references in a deeply nested dict with depth 10") +def step_when_resolve_deep(context: Context) -> None: + def _build_nested(depth: int): + if depth == 0: + return {"value": "1"} + return {"nested": _build_nested(depth - 1)} + + deep_content = _build_nested(10) + try: + context.shallow_canon.resolve_references(deep_content) + context.canonicalize_error = None + except ValueError as exc: + context.canonicalize_error = exc + + +@then('resolve_references raises ValueError with message containing "{msg_fragment}"') +def step_then_resolve_valueerror_with_msg(context: Context, msg_fragment: str) -> None: + assert context.canonicalize_error is not None, ( + "Expected ValueError, but no exception was raised" + ) + assert isinstance(context.canonicalize_error, ValueError), ( + f"Expected ValueError, got {type(context.canonicalize_error).__name__}" + ) + assert msg_fragment in str(context.canonicalize_error), ( + f"Expected error to contain {msg_fragment!r}, " + f"got {context.canonicalize_error!r}" + ) + + +# ── Constructor argument validation ────────────────────────────────────────── + + +@when("I attempt to create a Canonicalizer with max_depth {value}") +def step_when_create_canonicalizer_with_bad_max_depth( + context: Context, value: str +) -> None: + try: + import ast + + depth = ast.literal_eval(value) + Canonicalizer(max_depth=depth) + context.canonicalize_error = None + except ValueError as exc: + context.canonicalize_error = exc + + +# ── PackageType validation ─────────────────────────────────────────────────── + + +@when( + 'I attempt to compute_package_id with content "{content_name}" and package_type None' +) +def step_when_compute_with_none_type(context: Context, content_name: str) -> None: + content = {"name": content_name, "description": "test"} + canon = context.Canonicalizer() + try: + canon.compute_package_id(content, None) + context.canonicalize_error = None + except TypeError as exc: + context.canonicalize_error = exc + + +# ── Deeply nested list max_depth enforcement ────────────────────────────────── + + +@when("I canonicalize a deeply nested list with depth 10") +def step_when_canonicalize_deep_list(context: Context) -> None: + def _build_nested_list(depth: int): + if depth == 0: + return [1] + return [_build_nested_list(depth - 1)] + + deep_list = _build_nested_list(10) + try: + context.shallow_canon.canonicalize({"key": deep_list}) + context.canonicalize_error = None + except ValueError as exc: + context.canonicalize_error = exc + + +# ── Negative zero float normalization ───────────────────────────────────────── + + +@when("I canonicalize content containing negative zero") +def step_when_canonicalize_negative_zero(context: Context) -> None: + content = {"value": -0.0} + canon = context.Canonicalizer() + context.canonical_output = canon.canonicalize(content) + + +@then("the canonical output should serialize negative zero as positive zero") +def step_then_negative_zero_normalized(context: Context) -> None: + assert "-0.0" not in context.canonical_output, ( + f"Canonical JSON should not contain -0.0: {context.canonical_output!r}" + ) + assert ":0.0" in context.canonical_output or ":0," in context.canonical_output, ( + f"Canonical JSON should contain 0.0: {context.canonical_output!r}" + ) diff --git a/pyproject.toml b/pyproject.toml index 04a6c74..21253d9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,8 +4,8 @@ build-backend = "hatchling.build" [project] name = "cleveractors" -version = "2.0.0" -description = "CleverActors library - Agent-based LLM tool framework (v2.0.0 snapshot)" +version = "2.1.0" +description = "CleverActors library - Agent-based LLM tool framework (v2.1.0)" readme = "README.md" requires-python = ">=3.13" license = {text = "MIT"} diff --git a/robot/CanonicalizerLib.py b/robot/CanonicalizerLib.py new file mode 100644 index 0000000..0ff01f7 --- /dev/null +++ b/robot/CanonicalizerLib.py @@ -0,0 +1,268 @@ +"""Robot Framework keyword library for Canonicalizer integration tests.""" + +from __future__ import annotations + +import json +from typing import Any + +from cleveractors.registry.canonical import Canonicalizer +from cleveractors.registry.types import PackageType + +ROBOT_LIBRARY_SCOPE = "TEST SUITE" # pragma: no cover - integration test library + + +class CanonicalizerLib: # pragma: no cover - integration test library + """Keyword library for Canonicalizer Robot Framework integration tests.""" + + def __init__(self) -> None: + self._canonicalizer: Canonicalizer = Canonicalizer() + self._result: str = "" + self._content: dict[str, Any] = {} + self._package_id_string: str = "" + self._package_id_string_2: str = "" + self._canonical_output_2: str = "" + self._content_original: dict[str, Any] = {} + + def create_canonicalizer(self) -> None: + """Create a fresh Canonicalizer instance for a test case.""" + self._canonicalizer = Canonicalizer() + + def set_content_from_yaml_dict(self, yaml_string: str) -> None: + """Parse a YAML string into the internal content dict.""" + import yaml + + self._content = yaml.safe_load(yaml_string) + + def set_complex_content(self) -> None: + """Set content to a complex nested dict with lifecycle fields.""" + self._content = { + "name": "complex-actor", + "description": "An actor with many properties", + "type": "llm", + "model": "gpt-4", + "temperature": 0.7, + "max_tokens": 4096, + "version": "2.1.0", + "release_date": "2026-06-07", + "metadata": { + "author": "Luis Mendes", + "tags": ["ai", "nlp", "conversation"], + "license": "MIT", + "version": "1.0.0", + }, + "parameters": { + "top_p": 0.9, + "frequency_penalty": 0.0, + "presence_penalty": 0.0, + }, + } + + def set_unicode_content(self) -> None: + """Set content containing accented Unicode strings for NFC testing.""" + self._content = { + "name": "unicode-test", + "description": "café résumé naïve garçon voilà", + "labels": ["café", "résumé", "naïve"], + "nested": {"key": "garçon voilà"}, + } + + def set_content_with_unsorted_keys(self) -> None: + """Set content with intentionally unsorted keys for sorting tests.""" + from collections import OrderedDict + + self._content = OrderedDict( + [ + ("zulu", "last"), + ("alpha", "first"), + ("mike", "middle"), + ("version", "should-be-stripped"), + ] + ) + + def canonicalize_content(self) -> None: + """Canonicalize the current content and store the result.""" + self._result = self._canonicalizer.canonicalize(self._content) + + def canonicalize_content_twice(self) -> None: + """Canonicalize content twice to enable determinism assertions.""" + first = self._canonicalizer.canonicalize(self._content) + second = self._canonicalizer.canonicalize(self._content) + self._result = first + self._canonical_output_2 = second + + def compute_package_id_for_type(self, pkg_type: str) -> None: + """Compute a PackageId for the current content by type name.""" + pt = PackageType[pkg_type.upper()] + self._package_id_string = self._canonicalizer.compute_package_id( + self._content, pt + ).id_string + + def compute_package_id_twice_same_type(self, pkg_type: str) -> None: + """Compute PackageId twice with same type to verify determinism.""" + pt = PackageType[pkg_type.upper()] + pid1 = self._canonicalizer.compute_package_id(self._content, pt) + pid2 = self._canonicalizer.compute_package_id(self._content, pt) + self._package_id_string = str(pid1) + self._package_id_string_2 = str(pid2) + + def compute_package_ids_for_types(self, type1: str, type2: str) -> None: + """Compute PackageIds for two different package types.""" + pt1 = PackageType[type1.upper()] + pt2 = PackageType[type2.upper()] + self._package_id_string = self._canonicalizer.compute_package_id( + self._content, pt1 + ).id_string + self._package_id_string_2 = self._canonicalizer.compute_package_id( + self._content, pt2 + ).id_string + + def content_differs_by_one_field(self) -> None: + """Save original content then modify the current content.""" + self._content_original = dict(self._content) + self._content["modified"] = True + + def canonicalize_both_contents(self) -> None: + """Canonicalize both original and modified content for comparison.""" + self._result = self._canonicalizer.canonicalize(self._content_original) + self._canonical_output_2 = self._canonicalizer.canonicalize(self._content) + + # ── Assertions ────────────────────────────────────────────────────── + + def result_is_valid_json(self) -> None: + """Assert the canonicalization result parses as valid JSON dict.""" + parsed = json.loads(self._result) + assert isinstance(parsed, dict), ( + f"Expected dict from JSON, got {type(parsed).__name__}" + ) + + def result_has_key(self, key: str) -> None: + """Assert the result JSON contains the given top-level key.""" + parsed = json.loads(self._result) + assert key in parsed, f"Key {key!r} not found in result" + + def result_does_not_have_key(self, key: str) -> None: + """Assert the result JSON does NOT contain the given top-level key.""" + parsed = json.loads(self._result) + assert key not in parsed, f"Key {key!r} should not be in result" + + def result_keys_are_sorted(self) -> None: + """Assert top-level keys in the result are lexicographically sorted.""" + parsed = json.loads(self._result) + keys = list(parsed.keys()) + assert keys == sorted(keys), f"Keys not sorted: {keys}" + + def result_keys_are_sorted_recursively(self) -> None: + """Assert all keys at every nesting level are lexicographically sorted.""" + parsed = json.loads(self._result) + + def _check_sorted(obj: Any, path: str) -> None: + if isinstance(obj, dict): + keys = list(obj.keys()) + assert keys == sorted(keys), f"Keys not sorted at {path}: {keys}" + for k, v in obj.items(): + _check_sorted(v, f"{path}.{k}") + elif isinstance(obj, list): + for i, item in enumerate(obj): + _check_sorted(item, f"{path}[{i}]") + + _check_sorted(parsed, "root") + + def canonicalization_is_deterministic(self) -> None: + """Assert two canonicalization calls produced identical output.""" + assert self._result == self._canonical_output_2, ( + f"Canonicalization not deterministic:\n{self._result!r}\n!=\n{self._canonical_output_2!r}" + ) + + def canonicalization_produces_different_output(self) -> None: + """Assert canonicalization of different content produces different output.""" + assert self._result != self._canonical_output_2, ( + "Different content should produce different canonical output" + ) + + def package_id_starts_with(self, prefix: str) -> None: + """Assert the PackageId string starts with the expected type prefix.""" + assert self._package_id_string.startswith(prefix), ( + f"PackageId {self._package_id_string!r} should start with {prefix!r}" + ) + + def package_ids_are_identical(self) -> None: + """Assert two computed PackageIds are identical.""" + assert self._package_id_string == self._package_id_string_2, ( + f"PackageIds should be identical: {self._package_id_string} != {self._package_id_string_2}" + ) + + def package_ids_are_different(self) -> None: + """Assert two computed PackageIds differ.""" + assert self._package_id_string != self._package_id_string_2, ( + f"PackageIds should differ: {self._package_id_string} == {self._package_id_string_2}" + ) + + def package_id_is_40_hex_chars(self) -> None: + """Assert the PackageId SHA-1 portion is 40 lowercase hex characters.""" + sha1 = self._package_id_string.rsplit("_", 1)[-1] + assert len(sha1) == 40, f"SHA1 should be 40 chars: {len(sha1)}" + assert all(c in "0123456789abcdef" for c in sha1), ( + f"SHA1 should be hex: {sha1!r}" + ) + + def result_contains_nfc_normalized_text(self, expected: str) -> None: + """Assert the expected NFC-normalized text exists in the result and all strings are NFC.""" + import unicodedata + + parsed = json.loads(self._result) + + def _collect_strings(obj: Any) -> list[str]: + strings: list[str] = [] + if isinstance(obj, str): + strings.append(obj) + elif isinstance(obj, dict): + for v in obj.values(): + strings.extend(_collect_strings(v)) + elif isinstance(obj, list): + for item in obj: + strings.extend(_collect_strings(item)) + return strings + + all_strings = _collect_strings(parsed) + nfc_expected = unicodedata.normalize("NFC", expected) + assert nfc_expected in all_strings, ( + f"Expected text {nfc_expected!r} not found in strings: {all_strings!r}" + ) + for s in all_strings: + assert s == unicodedata.normalize("NFC", s), ( + f"String not NFC-normalized: {s!r}" + ) + + def result_does_not_contain_lifecycle_field(self, field: str) -> None: + """Assert the given lifecycle field is not present at any nesting level.""" + parsed = json.loads(self._result) + + def _has_field(obj: Any, target: str) -> bool: + if isinstance(obj, dict): + if target in obj: + return True + return any(_has_field(v, target) for v in obj.values()) + if isinstance(obj, list): + return any(_has_field(item, target) for item in obj) + return False + + assert not _has_field(parsed, field), ( + f"Lifecycle field {field!r} found in canonical output" + ) + + def result_has_string_value(self, key: str, expected: str) -> None: + """Assert the result contains the given key with the expected string value.""" + parsed = json.loads(self._result) + + def _find(obj: Any, target: str) -> Any: + if isinstance(obj, dict): + if target in obj: + return obj[target] + for v in obj.values(): + found = _find(v, target) + if found is not None: + return found + return None + + actual = _find(parsed, key) + assert actual == expected, f"Key {key!r}: expected {expected!r}, got {actual!r}" diff --git a/robot/canonicalization.robot b/robot/canonicalization.robot new file mode 100644 index 0000000..0402e10 --- /dev/null +++ b/robot/canonicalization.robot @@ -0,0 +1,109 @@ +*** Settings *** +Documentation Integration tests for the Canonicalizer (Package Registry Standard v1.0.0 §6). +... Exercises complex content structures, determinism, NFC normalization, +... lifecycle field stripping, and PackageId computation across runs. +Library CanonicalizerLib.py + +*** Test Cases *** +Canonicalize Complex Content Produces Valid JSON + [Documentation] Complex nested content with metadata, parameters, and lifecycle fields + ... should produce valid JSON with sorted keys and stripped lifecycle fields. + [Tags] canonicalization smoketest + Create Canonicalizer + Set Complex Content + Canonicalize Content + Result Is Valid JSON + Result Keys Are Sorted Recursively + Result Does Not Have Key version + Result Does Not Have Key release_date + +Canonicalization Is Deterministic Across Multiple Runs + [Documentation] Same content must produce identical canonical output on every invocation. + [Tags] canonicalization determinism + Create Canonicalizer + Set Complex Content + Canonicalize Content Twice + Canonicalization Is Deterministic + +Key Sorting Is Enforced In Canonical Output + [Documentation] Unsorted keys in input must be lexicographically sorted in output, + ... with lifecycle fields stripped. + [Tags] canonicalization key-sorting + Create Canonicalizer + Set Content With Unsorted Keys + Canonicalize Content + Result Keys Are Sorted + Result Does Not Have Key version + +Unicode NFC Normalization Is Applied + [Documentation] Strings with accented characters must be NFC-normalized in output. + [Tags] canonicalization unicode + Create Canonicalizer + Set Unicode Content + Canonicalize Content + Result Contains NFC Normalized Text café résumé naïve garçon voilà + +Package ID Computation With Actor Type + [Documentation] compute_package_id must produce a valid PackageId with correct prefix. + [Tags] canonicalization package-id + Create Canonicalizer + Set Complex Content + Compute Package Id For Type ACTOR + Package Id Starts With pkg_act_ + Package Id Is 40 Hex Chars + +Package ID Computation With All Supported Types + [Documentation] Each package type must produce a PackageId with the correct prefix. + [Tags] canonicalization package-id + Create Canonicalizer + Set Complex Content + Compute Package Id For Type GRAPH + Package Id Starts With pkg_grh_ + Compute Package Id For Type STREAM + Package Id Starts With pkg_str_ + Compute Package Id For Type AGENT + Package Id Starts With pkg_agt_ + Compute Package Id For Type TEMPLATE + Package Id Starts With pkg_tpl_ + Compute Package Id For Type SKILL + Package Id Starts With pkg_skl_ + Compute Package Id For Type MCP + Package Id Starts With pkg_mcp_ + Compute Package Id For Type LSP + Package Id Starts With pkg_lsp_ + +Same Content Produces Identical Package IDs + [Documentation] Computing PackageId twice with same content and type yields identical result. + [Tags] canonicalization package-id determinism + Create Canonicalizer + Set Complex Content + Compute Package Id Twice Same Type ACTOR + Package Ids Are Identical + +Different Package Types Produce Different Prefixes + [Documentation] Same content with different types must yield different IDs. + [Tags] canonicalization package-id + Create Canonicalizer + Set Complex Content + Compute Package Ids For Types ACTOR GRAPH + Package Ids Are Different + +Different Content Produces Different Canonical Output + [Documentation] Modifying content must change the canonical output. + [Tags] canonicalization determinism + Create Canonicalizer + Set Complex Content + Content Differs By One Field + Canonicalize Both Contents + Canonicalization Produces Different Output + +Lifecycle Fields Stripped At All Nesting Levels + [Documentation] Version and release_date must be removed from all nesting levels. + [Tags] canonicalization lifecycle + Create Canonicalizer + Set Complex Content + Canonicalize Content + Result Does Not Contain Lifecycle Field version + Result Does Not Contain Lifecycle Field release_date + Result Has String Value name complex-actor + Result Has String Value model gpt-4 diff --git a/robot/version.robot b/robot/version.robot index d847e64..1e14a3e 100644 --- a/robot/version.robot +++ b/robot/version.robot @@ -5,7 +5,7 @@ Library OperatingSystem Library String *** Variables *** -${EXPECTED_VERSION} 2.0.0 +${EXPECTED_VERSION} 2.1.0 *** Test Cases *** Package Version Is Correct diff --git a/src/cleveractors/__init__.py b/src/cleveractors/__init__.py index 4f9c42f..464ba42 100644 --- a/src/cleveractors/__init__.py +++ b/src/cleveractors/__init__.py @@ -1,12 +1,12 @@ """ -CleverActors - Agent-based LLM tool framework (v2.0.0 snapshot). +CleverActors - Agent-based LLM tool framework (v2.1.0 snapshot). This package provides the core framework for creating agent-based LLM applications using reactive streams (RxPy) and LangGraph, including templates, prompts, and configuration for agent networks. """ -__version__ = "2.0.0" +__version__ = "2.1.0" __author__ = "CleverThis Engineering" from cleveractors.agent import Agent diff --git a/src/cleveractors/registry/__init__.py b/src/cleveractors/registry/__init__.py index 47a733e..ae4e8d5 100644 --- a/src/cleveractors/registry/__init__.py +++ b/src/cleveractors/registry/__init__.py @@ -1,5 +1,6 @@ """Package Registry client and core types (Package Registry Standard v1.0.0).""" +from cleveractors.registry.canonical import Canonicalizer from cleveractors.registry.client import RegistryClient from cleveractors.registry.exceptions import ( AccessDeniedError, @@ -22,6 +23,7 @@ from cleveractors.registry.types import ( ) __all__ = [ + "Canonicalizer", "RegistryClient", "RegistryError", "PackageNotFoundError", diff --git a/src/cleveractors/registry/canonical.py b/src/cleveractors/registry/canonical.py new file mode 100644 index 0000000..b2f7618 --- /dev/null +++ b/src/cleveractors/registry/canonical.py @@ -0,0 +1,192 @@ +"""Canonicalization engine (Package Registry Standard v1.0.0 §6). + +Produces deterministic RFC-8785 canonical JSON from package content +for content-addressed SHA-1 hashing. + +SHA-1 is used solely for content-addressing per §6. It is NOT +suitable for cryptographic integrity verification. +""" + +from __future__ import annotations + +import hashlib +import json +import unicodedata +from collections.abc import Callable +from typing import Any + +from cleveractors.registry.types import PackageId, PackageType + +_DEFAULT_MAX_DEPTH: int = 100 + + +class Canonicalizer: + """Canonicalize package content into deterministic RFC-8785 JSON. + + The canonical form (§6.2) is produced by: + + 1. Resolving internal ``ID:pkg_...`` references when a resolver + callback is configured + 2. NFC-normalizing every string value + 3. Sorting dictionary keys lexicographically + 4. Stripping lifecycle fields (``version``, ``release_date``) + 5. Serializing to RFC-8785 canonical JSON (no whitespace, + sorted keys, UTF-8, no solidus escaping) + + SHA-1 is used solely for content-addressing per §6. It is NOT + suitable for cryptographic integrity verification. + """ + + LIFECYCLE_FIELDS: frozenset[str] = frozenset({"version", "release_date"}) + + def __init__( + self, + *, + reference_resolver: Callable[[str], str] | None = None, + max_depth: int = _DEFAULT_MAX_DEPTH, + ) -> None: + if not isinstance(max_depth, int) or max_depth < 1: + raise ValueError(f"max_depth must be a positive int, got {max_depth!r}") + self._reference_resolver = reference_resolver + self._max_depth = max_depth + + # ------------------------------------------------------------------ + # Public API + # ------------------------------------------------------------------ + + def canonicalize(self, content: dict[str, Any]) -> str: + """Produce deterministic RFC-8785 canonical JSON from *content*. + + When a ``reference_resolver`` is configured, internal references + are resolved before transformation (§6.2 step 5). + + Args: + content: A package document as a parsed dict. + + Returns: + A canonical JSON string (UTF-8, no insignificant whitespace). + + Raises: + TypeError: If *content* is not a ``dict``. + ValueError: If the document exceeds the configured max depth. + """ + if not isinstance(content, dict): + raise TypeError(f"content must be dict, got {type(content).__name__}") + if self._reference_resolver is not None: + resolved_content = self.resolve_references(content) + else: + resolved_content = content + canonical = self._transform(resolved_content, depth=0) + return json.dumps( + canonical, + separators=(",", ":"), + ensure_ascii=False, + sort_keys=True, + allow_nan=False, + ) + + def compute_package_id( + self, content: dict[str, Any], package_type: PackageType + ) -> PackageId: + """Compute a content-addressed PackageId from *content*. + + Args: + content: A package document as a parsed dict. + package_type: The package type (determines the ID prefix). + + Returns: + A PackageId whose SHA-1 is the hash of the canonical form. + + Raises: + TypeError: If *content* is not a ``dict``. + ValueError: If the document exceeds the configured max depth. + """ + if not isinstance(content, dict): + raise TypeError(f"content must be dict, got {type(content).__name__}") + if not isinstance(package_type, PackageType): + raise TypeError( + f"package_type must be PackageType, got {type(package_type).__name__}" + ) + canonical = self.canonicalize(content) + sha1_hex = hashlib.sha1( + canonical.encode("utf-8"), usedforsecurity=False + ).hexdigest() + return PackageId(package_type=package_type, sha1_hex=sha1_hex) + + # ------------------------------------------------------------------ + # Internal transformation + # ------------------------------------------------------------------ + + def _transform(self, value: Any, *, depth: int = 0) -> Any: + """Recursively transform *value* into canonical form. + + - Dicts: strip lifecycle fields, sort keys, recurse into values. + - Lists: recurse into each element. + - Strings: apply NFC normalization. + - Other (int, float, bool, None): pass through unchanged. + """ + if depth > self._max_depth: + raise ValueError(f"Maximum nesting depth ({self._max_depth}) exceeded") + if isinstance(value, dict): + return self._transform_dict(value, depth=depth + 1) + if isinstance(value, list): + return [self._transform(item, depth=depth + 1) for item in value] + if isinstance(value, str): + return unicodedata.normalize("NFC", value) + if isinstance(value, float) and value == 0.0: + return 0.0 + return value + + def _transform_dict(self, dct: dict[str, Any], *, depth: int = 0) -> dict[str, Any]: + """Transform a dict, stripping lifecycle fields and sorting keys.""" + if depth > self._max_depth: + raise ValueError(f"Maximum nesting depth ({self._max_depth}) exceeded") + return { + key: self._transform(val, depth=depth) + for key, val in dct.items() + if key not in self.LIFECYCLE_FIELDS + } + + # ------------------------------------------------------------------ + # Reference resolution + # ------------------------------------------------------------------ + + def resolve_references(self, content: dict[str, Any]) -> dict[str, Any]: + """Resolve internal references in *content* to concrete PackageIds. + + Walks the dict recursively and replaces any string value that + looks like a reference (``ID:pkg_...``) via the configured + *reference_resolver* callback. + + When no resolver is configured, the content is returned unchanged + (references are left in their original form). + + Args: + content: A package document as a parsed dict. + + Returns: + A new dict with all references resolved. + + Raises: + TypeError: If *content* is not a ``dict``. + """ + if not isinstance(content, dict): + raise TypeError(f"content must be dict, got {type(content).__name__}") + if self._reference_resolver is None: + return content + return self._resolve_value(content, depth=0) + + def _resolve_value(self, value: Any, *, depth: int = 0) -> Any: + """Recursively walk *value* and resolve ``ID:pkg_...`` refs.""" + if depth > self._max_depth: + raise ValueError(f"Maximum nesting depth ({self._max_depth}) exceeded") + if isinstance(value, dict): + return { + key: self._resolve_value(val, depth=depth + 1) + for key, val in value.items() + } + if isinstance(value, list): + return [self._resolve_value(item, depth=depth + 1) for item in value] + if isinstance(value, str) and value.startswith("ID:pkg_"): + return self._reference_resolver(value) + return value