Files
cleveragents-core/features/steps/uko_indexer_protocol_steps.py
T
hamza.khyari 1e606553d4
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 2m32s
CI / integration_tests (pull_request) Successful in 3m18s
CI / docker (pull_request) Successful in 49s
CI / coverage (pull_request) Successful in 5m29s
CI / benchmark-regression (pull_request) Successful in 33m48s
feat(acms): implement Real-time Index Sync / UKOIndexer with pluggable analyzers
Implement the UKOIndexer service that produces UKO triples from resources
using pluggable domain-specific analyzers, wraps each triple with provenance
metadata, and simultaneously indexes into text, vector, and graph backends.

Key design decisions and components:

- UKOIndexer orchestrates the full index lifecycle: add_resource,
  update_resource (remove-then-add), remove_resource, and maintenance
  triggers. Each operation fires lifecycle hooks (on_indexed, on_removed,
  on_error) so callers can observe progress.

- Analyzer selection is pluggable via ContentAnalyzer protocol. The indexer
  accepts a registry mapping resource types to analyzers. PythonAnalyzer
  and MarkdownAnalyzer are provided as built-in implementations.

- LocationContentReader protocol abstracts file I/O with a base_dir
  parameter for path-traversal prevention (post-resolve validation rejects
  paths escaping the base directory and non-regular files).

- UKOTriple model includes a @model_validator ensuring at least one of
  object_uri or object_value is populated, preventing empty triples at
  construction time.

- Triple removal uses scoped deletion via uko:sourceResource predicate to
  avoid shared-subject collision — only triples originating from the
  specific resource are removed, not all triples for a shared subject.

- _resource_subjects.pop is deferred until after all backend removal
  operations succeed, preventing inconsistent state on partial failure.

- analyzer.analyze() is wrapped in try/except so that analyzer errors
  produce an IndexResult with error details rather than propagating
  exceptions to callers.

- All lifecycle hook calls are guarded via _fire_on_indexed,
  _fire_on_removed, and _fire_on_error helpers that catch and log hook
  exceptions without disrupting the indexing pipeline.

- max_triples parameter (default 50,000) bounds analyzer output size to
  prevent runaway resource consumption.

- ResourceFileWatcher monitors filesystem paths via watchdog and triggers
  re-indexing callbacks on file changes with configurable debouncing.
  Emits RESOURCE_MODIFIED domain events via EventBus when file changes
  are detected. Debounce timers coalesce rapid edits into a single
  callback invocation. Thread-safe design with daemon threads for clean
  shutdown.

- SearchResult.__post_init__ validates score is in [0.0, 1.0], correctly
  rejecting NaN values.

- Placeholder embedding uses [1.0] instead of [float(len(content))] to
  avoid leaking content size information.

- isinstance check on graph_backend ensures GraphIndexBackend protocol
  compliance at runtime.

- Test doubles extracted to features/mocks/uko_indexer_mocks.py for reuse
  across BDD steps and Robot helpers.

Spec reference: Architecture > ACMS > Real-time Index Synchronization
(specification.md lines ~43205-43300).

ISSUES CLOSED: #578
2026-03-11 00:40:07 +00:00

283 lines
8.5 KiB
Python

"""Protocol stubs, UKOTriple validation, and direct internal-call steps.
Coverage scenarios that exercise Protocol ``raise NotImplementedError``
stubs, Pydantic validator error paths, and internal helper functions
called directly (bypassing the public API) to ensure slipcover traces
the code.
All steps prefixed with ``idx`` to avoid AmbiguousStep collisions.
"""
from __future__ import annotations
from typing import Any
from behave import then # type: ignore[import-untyped]
from cleveragents.domain.models.acms.analyzers import UKOTriple
from cleveragents.domain.models.acms.provenance import IndexResult
from features.steps.uko_indexer_common import ULID_1, _make_resource
__all__: list[str] = []
# ---------------------------------------------------------------------------
# Protocol stub NotImplementedError steps
# ---------------------------------------------------------------------------
@then(
"idx calling ContentReader.read_content directly should raise NotImplementedError"
)
def step_proto_content_reader(context: Any) -> None:
from cleveragents.application.services.uko_indexer_protocols import (
ContentReader,
)
class _Bare(ContentReader): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
bare.read_content(_make_resource(ULID_1))
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then(
"idx calling IndexLifecycleHook.on_indexed directly"
" should raise NotImplementedError"
)
def step_proto_hook_indexed(context: Any) -> None:
from cleveragents.application.services.uko_indexer_protocols import (
IndexLifecycleHook,
)
class _Bare(IndexLifecycleHook): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
bare.on_indexed(IndexResult(resource_id=ULID_1))
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then(
"idx calling IndexLifecycleHook.on_removed directly"
" should raise NotImplementedError"
)
def step_proto_hook_removed(context: Any) -> None:
from cleveragents.application.services.uko_indexer_protocols import (
IndexLifecycleHook,
)
class _Bare(IndexLifecycleHook): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
bare.on_removed(ULID_1, "p")
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then(
"idx calling IndexLifecycleHook.on_error directly should raise NotImplementedError"
)
def step_proto_hook_error(context: Any) -> None:
from cleveragents.application.services.uko_indexer_protocols import (
IndexLifecycleHook,
)
class _Bare(IndexLifecycleHook): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
bare.on_error(ULID_1, "err")
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then(
"idx calling AnalyzerProtocol.supported_extensions directly"
" should raise NotImplementedError"
)
def step_proto_analyzer_ext(context: Any) -> None:
from cleveragents.domain.models.acms.analyzers import AnalyzerProtocol
class _Bare(AnalyzerProtocol): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
_ = bare.supported_extensions
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then("idx calling AnalyzerProtocol.domain directly should raise NotImplementedError")
def step_proto_analyzer_domain(context: Any) -> None:
from cleveragents.domain.models.acms.analyzers import AnalyzerProtocol
class _Bare(AnalyzerProtocol): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
_ = bare.domain
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
@then("idx calling AnalyzerProtocol.analyze directly should raise NotImplementedError")
def step_proto_analyzer_analyze(context: Any) -> None:
from cleveragents.domain.models.acms.analyzers import AnalyzerProtocol
class _Bare(AnalyzerProtocol): # type: ignore[misc]
pass
bare = _Bare() # type: ignore[abstract]
try:
bare.analyze("content", "uko://r")
assert False, "Expected NotImplementedError" # noqa: B011
except NotImplementedError:
pass
# ---------------------------------------------------------------------------
# UKOTriple validation edge cases
# ---------------------------------------------------------------------------
@then("idx creating UKOTriple with whitespace-only subject_uri should raise ValueError")
def step_triple_ws_subject(context: Any) -> None:
try:
UKOTriple(
subject_uri=" ",
predicate="p",
object_uri="uko://o",
)
assert False, "Expected ValueError" # noqa: B011
except ValueError:
pass
@then("idx creating UKOTriple with whitespace-only predicate should raise ValueError")
def step_triple_ws_predicate(context: Any) -> None:
try:
UKOTriple(
subject_uri="uko://s",
predicate=" ",
object_uri="uko://o",
)
assert False, "Expected ValueError" # noqa: B011
except ValueError:
pass
@then(
"idx creating UKOTriple with empty object_uri and empty object_value"
" should raise ValueError"
)
def step_triple_no_object(context: Any) -> None:
try:
UKOTriple(
subject_uri="uko://s",
predicate="rdf:type",
object_uri="",
object_value="",
)
assert False, "Expected ValueError" # noqa: B011
except ValueError:
pass
# ---------------------------------------------------------------------------
# fire_on_removed direct call
# ---------------------------------------------------------------------------
@then("idx calling fire_on_removed with a raising hook should not raise")
def step_fire_on_removed_direct(context: Any) -> None:
from cleveragents.application.services.uko_indexer_internals import (
fire_on_removed,
)
class _RaisingHook:
def on_indexed(self, result: IndexResult) -> None:
pass
def on_removed(self, resource_id: str, project: str) -> None:
raise RuntimeError("boom")
def on_error(self, resource_id: str, error: str) -> None:
pass
# Should not raise — fire_on_removed swallows the exception
fire_on_removed(
_RaisingHook(), # type: ignore[arg-type]
ULID_1,
"local/test",
)
# ---------------------------------------------------------------------------
# AnalyzerRegistry validation and listing steps
# ---------------------------------------------------------------------------
@then("idx registering a non-protocol object should raise TypeError")
def step_registry_non_protocol(context: Any) -> None:
from cleveragents.domain.models.acms.analyzers import AnalyzerRegistry
registry = AnalyzerRegistry()
try:
registry.register("not_an_analyzer") # type: ignore[arg-type]
assert False, "Expected TypeError" # noqa: B011
except TypeError:
pass
@then("idx registering an analyzer with empty extensions should raise ValueError")
def step_registry_empty_extensions(context: Any) -> None:
from cleveragents.domain.models.acms.analyzers import AnalyzerRegistry
class _NoExtensions:
supported_extensions: frozenset[str] = frozenset()
domain = "empty"
def analyze(self, content: str, resource_uri: str) -> list[UKOTriple]:
return []
registry = AnalyzerRegistry()
try:
registry.register(_NoExtensions()) # type: ignore[arg-type]
assert False, "Expected ValueError" # noqa: B011
except ValueError:
pass
@then("idx looking up empty extension should return None")
def step_registry_empty_ext(context: Any) -> None:
assert context.idx_registry.get_for_extension("") is None
@then('idx list_extensions should include ".py"')
def step_registry_list_ext(context: Any) -> None:
exts = context.idx_registry.list_extensions()
assert ".py" in exts, f".py not in {exts}"
@then("idx list_analyzers should return {n:d} analyzer")
def step_registry_list_analyzers(context: Any, n: int) -> None:
analyzers = context.idx_registry.list_analyzers()
assert len(analyzers) == n, f"Expected {n} analyzers, got {len(analyzers)}"