forked from cleveragents/cleveragents-core
hamza.khyari
1e606553d4
Implement the UKOIndexer service that produces UKO triples from resources using pluggable domain-specific analyzers, wraps each triple with provenance metadata, and simultaneously indexes into text, vector, and graph backends. Key design decisions and components: - UKOIndexer orchestrates the full index lifecycle: add_resource, update_resource (remove-then-add), remove_resource, and maintenance triggers. Each operation fires lifecycle hooks (on_indexed, on_removed, on_error) so callers can observe progress. - Analyzer selection is pluggable via ContentAnalyzer protocol. The indexer accepts a registry mapping resource types to analyzers. PythonAnalyzer and MarkdownAnalyzer are provided as built-in implementations. - LocationContentReader protocol abstracts file I/O with a base_dir parameter for path-traversal prevention (post-resolve validation rejects paths escaping the base directory and non-regular files). - UKOTriple model includes a @model_validator ensuring at least one of object_uri or object_value is populated, preventing empty triples at construction time. - Triple removal uses scoped deletion via uko:sourceResource predicate to avoid shared-subject collision — only triples originating from the specific resource are removed, not all triples for a shared subject. - _resource_subjects.pop is deferred until after all backend removal operations succeed, preventing inconsistent state on partial failure. - analyzer.analyze() is wrapped in try/except so that analyzer errors produce an IndexResult with error details rather than propagating exceptions to callers. - All lifecycle hook calls are guarded via _fire_on_indexed, _fire_on_removed, and _fire_on_error helpers that catch and log hook exceptions without disrupting the indexing pipeline. - max_triples parameter (default 50,000) bounds analyzer output size to prevent runaway resource consumption. - ResourceFileWatcher monitors filesystem paths via watchdog and triggers re-indexing callbacks on file changes with configurable debouncing. Emits RESOURCE_MODIFIED domain events via EventBus when file changes are detected. Debounce timers coalesce rapid edits into a single callback invocation. Thread-safe design with daemon threads for clean shutdown. - SearchResult.__post_init__ validates score is in [0.0, 1.0], correctly rejecting NaN values. - Placeholder embedding uses [1.0] instead of [float(len(content))] to avoid leaking content size information. - isinstance check on graph_backend ensures GraphIndexBackend protocol compliance at runtime. - Test doubles extracted to features/mocks/uko_indexer_mocks.py for reuse across BDD steps and Robot helpers. Spec reference: Architecture > ACMS > Real-time Index Synchronization (specification.md lines ~43205-43300). ISSUES CLOSED: #578