hamza.khyari
1e606553d4
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 2m32s
CI / integration_tests (pull_request) Successful in 3m18s
CI / docker (pull_request) Successful in 49s
CI / coverage (pull_request) Successful in 5m29s
CI / benchmark-regression (pull_request) Successful in 33m48s
Implement the UKOIndexer service that produces UKO triples from resources using pluggable domain-specific analyzers, wraps each triple with provenance metadata, and simultaneously indexes into text, vector, and graph backends. Key design decisions and components: - UKOIndexer orchestrates the full index lifecycle: add_resource, update_resource (remove-then-add), remove_resource, and maintenance triggers. Each operation fires lifecycle hooks (on_indexed, on_removed, on_error) so callers can observe progress. - Analyzer selection is pluggable via ContentAnalyzer protocol. The indexer accepts a registry mapping resource types to analyzers. PythonAnalyzer and MarkdownAnalyzer are provided as built-in implementations. - LocationContentReader protocol abstracts file I/O with a base_dir parameter for path-traversal prevention (post-resolve validation rejects paths escaping the base directory and non-regular files). - UKOTriple model includes a @model_validator ensuring at least one of object_uri or object_value is populated, preventing empty triples at construction time. - Triple removal uses scoped deletion via uko:sourceResource predicate to avoid shared-subject collision — only triples originating from the specific resource are removed, not all triples for a shared subject. - _resource_subjects.pop is deferred until after all backend removal operations succeed, preventing inconsistent state on partial failure. - analyzer.analyze() is wrapped in try/except so that analyzer errors produce an IndexResult with error details rather than propagating exceptions to callers. - All lifecycle hook calls are guarded via _fire_on_indexed, _fire_on_removed, and _fire_on_error helpers that catch and log hook exceptions without disrupting the indexing pipeline. - max_triples parameter (default 50,000) bounds analyzer output size to prevent runaway resource consumption. - ResourceFileWatcher monitors filesystem paths via watchdog and triggers re-indexing callbacks on file changes with configurable debouncing. Emits RESOURCE_MODIFIED domain events via EventBus when file changes are detected. Debounce timers coalesce rapid edits into a single callback invocation. Thread-safe design with daemon threads for clean shutdown. - SearchResult.__post_init__ validates score is in [0.0, 1.0], correctly rejecting NaN values. - Placeholder embedding uses [1.0] instead of [float(len(content))] to avoid leaking content size information. - isinstance check on graph_backend ensures GraphIndexBackend protocol compliance at runtime. - Test doubles extracted to features/mocks/uko_indexer_mocks.py for reuse across BDD steps and Robot helpers. Spec reference: Architecture > ACMS > Real-time Index Synchronization (specification.md lines ~43205-43300). ISSUES CLOSED: #578
63 lines
1.9 KiB
Python
63 lines
1.9 KiB
Python
"""Shared helpers, constants, and Background steps for UKO Indexer behave tests.
|
|
|
|
All steps prefixed with ``idx`` to avoid AmbiguousStep collisions with
|
|
other feature files.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
from behave import given # type: ignore[import-untyped]
|
|
|
|
from cleveragents.domain.models.acms.analyzers import AnalyzerRegistry
|
|
from cleveragents.domain.models.acms.index_stubs import (
|
|
InMemoryGraphIndexBackend,
|
|
InMemoryTextIndexBackend,
|
|
InMemoryVectorIndexBackend,
|
|
)
|
|
from cleveragents.domain.models.core.resource import Resource
|
|
from features.mocks.uko_indexer_mocks import InMemoryContentReader
|
|
|
|
__all__: list[str] = []
|
|
|
|
ULID_1 = "01HQ8ZDRX50000000000000001"
|
|
DEFAULT_PROJECT = "local/test"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Test helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _make_resource(
|
|
resource_id: str,
|
|
location: str | None = "src/main.py",
|
|
resource_type_name: str = "git-checkout",
|
|
) -> Resource:
|
|
"""Create a minimal Resource for testing."""
|
|
return Resource(
|
|
resource_id=resource_id,
|
|
resource_type_name=resource_type_name,
|
|
classification="physical",
|
|
location=location,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Background steps
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given("idx a clean analyzer registry")
|
|
def step_clean_registry(context: Any) -> None:
|
|
context.idx_registry = AnalyzerRegistry()
|
|
context.idx_content_reader = InMemoryContentReader()
|
|
|
|
|
|
@given("idx in-memory index backends")
|
|
def step_in_memory_backends(context: Any) -> None:
|
|
context.idx_text_backend = InMemoryTextIndexBackend()
|
|
context.idx_vector_backend = InMemoryVectorIndexBackend()
|
|
context.idx_graph_backend = InMemoryGraphIndexBackend()
|