hamza.khyari
1e606553d4
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 2m32s
CI / integration_tests (pull_request) Successful in 3m18s
CI / docker (pull_request) Successful in 49s
CI / coverage (pull_request) Successful in 5m29s
CI / benchmark-regression (pull_request) Successful in 33m48s
Implement the UKOIndexer service that produces UKO triples from resources using pluggable domain-specific analyzers, wraps each triple with provenance metadata, and simultaneously indexes into text, vector, and graph backends. Key design decisions and components: - UKOIndexer orchestrates the full index lifecycle: add_resource, update_resource (remove-then-add), remove_resource, and maintenance triggers. Each operation fires lifecycle hooks (on_indexed, on_removed, on_error) so callers can observe progress. - Analyzer selection is pluggable via ContentAnalyzer protocol. The indexer accepts a registry mapping resource types to analyzers. PythonAnalyzer and MarkdownAnalyzer are provided as built-in implementations. - LocationContentReader protocol abstracts file I/O with a base_dir parameter for path-traversal prevention (post-resolve validation rejects paths escaping the base directory and non-regular files). - UKOTriple model includes a @model_validator ensuring at least one of object_uri or object_value is populated, preventing empty triples at construction time. - Triple removal uses scoped deletion via uko:sourceResource predicate to avoid shared-subject collision — only triples originating from the specific resource are removed, not all triples for a shared subject. - _resource_subjects.pop is deferred until after all backend removal operations succeed, preventing inconsistent state on partial failure. - analyzer.analyze() is wrapped in try/except so that analyzer errors produce an IndexResult with error details rather than propagating exceptions to callers. - All lifecycle hook calls are guarded via _fire_on_indexed, _fire_on_removed, and _fire_on_error helpers that catch and log hook exceptions without disrupting the indexing pipeline. - max_triples parameter (default 50,000) bounds analyzer output size to prevent runaway resource consumption. - ResourceFileWatcher monitors filesystem paths via watchdog and triggers re-indexing callbacks on file changes with configurable debouncing. Emits RESOURCE_MODIFIED domain events via EventBus when file changes are detected. Debounce timers coalesce rapid edits into a single callback invocation. Thread-safe design with daemon threads for clean shutdown. - SearchResult.__post_init__ validates score is in [0.0, 1.0], correctly rejecting NaN values. - Placeholder embedding uses [1.0] instead of [float(len(content))] to avoid leaking content size information. - isinstance check on graph_backend ensures GraphIndexBackend protocol compliance at runtime. - Test doubles extracted to features/mocks/uko_indexer_mocks.py for reuse across BDD steps and Robot helpers. Spec reference: Architecture > ACMS > Real-time Index Synchronization (specification.md lines ~43205-43300). ISSUES CLOSED: #578
82 lines
3.7 KiB
Plaintext
82 lines
3.7 KiB
Plaintext
*** Settings ***
|
|
Documentation Smoke tests for UKO Indexer Real-time Index Synchronization
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_uko_indexer.py
|
|
|
|
*** Test Cases ***
|
|
Index Resource Pipeline
|
|
[Documentation] Verify UKOIndexer indexes a resource into graph, text, and vector backends
|
|
${result}= Run Process ${PYTHON} ${HELPER} index-resource cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-index-resource-ok
|
|
|
|
Remove Resource Cleanup
|
|
[Documentation] Verify UKOIndexer removes a resource from all indices
|
|
${result}= Run Process ${PYTHON} ${HELPER} remove-resource cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-remove-resource-ok
|
|
|
|
Reindex Resource Lifecycle
|
|
[Documentation] Verify reindex_resource removes old data then re-indexes
|
|
${result}= Run Process ${PYTHON} ${HELPER} reindex-resource cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-reindex-resource-ok
|
|
|
|
Graceful Degradation Without Optional Backends
|
|
[Documentation] Verify indexing works without text and vector backends
|
|
${result}= Run Process ${PYTHON} ${HELPER} graceful-degradation cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-graceful-degradation-ok
|
|
|
|
Provenance Metadata Attached
|
|
[Documentation] Verify provenance metadata is attached to indexed triples
|
|
${result}= Run Process ${PYTHON} ${HELPER} provenance cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-provenance-ok
|
|
|
|
Index Backend Operations
|
|
[Documentation] Verify in-memory text, vector, and graph backend CRUD operations
|
|
${result}= Run Process ${PYTHON} ${HELPER} index-backends cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-index-backends-ok
|
|
|
|
Input Validation Guards
|
|
[Documentation] Verify validation rejects empty project, empty resource_id, invalid types
|
|
${result}= Run Process ${PYTHON} ${HELPER} validation cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-validation-ok
|
|
|
|
Protocol Compliance
|
|
[Documentation] Verify ContentReader and IndexLifecycleHook protocol compliance
|
|
${result}= Run Process ${PYTHON} ${HELPER} protocol-compliance cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-protocol-compliance-ok
|
|
|
|
File Watching
|
|
[Documentation] Verify ResourceFileWatcher lifecycle, change detection, and EventBus
|
|
${result}= Run Process ${PYTHON} ${HELPER} file-watching cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} uko-indexer-file-watching-ok
|