# UKO Indexer — Real-time Index Synchronization

The UKO Indexer orchestrates the analysis of resources into UKO triples and
simultaneously indexes content into text, vector, and graph backends.  It
implements the index lifecycle (add/change/remove/maintenance) with graceful
degradation when optional backends are unavailable.

Based on `docs/specification.md` > ACMS > Real-time Index
Synchronization and Custom Index Backends.

## Architecture

```
Resource ──► AnalyzerRegistry.get_for_resource()
                 │
                 ▼
            ContentReader.read_content()
                 │
                 ▼
           Analyzer.analyze(content, resource_uri)
                 │
                 ▼
           attach_provenance(triples, resource, timestamp)
                 │
                 ├──► GraphIndexBackend.add_triple()   (required)
                 ├──► TextIndexBackend.index_document() (optional — graceful degradation)
                 └──► VectorIndexBackend.index_embedding() (optional — graceful degradation)
```

## UKOIndexer

Main service class that produces UKO triples from resources and indexes them
into all available backends.

### Constructor

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `analyzer_registry` | `AnalyzerRegistry` | Yes | Registry of domain analyzers |
| `graph_backend` | `GraphIndexBackend` | Yes | Backend for UKO triple storage |
| `text_backend` | `TextIndexBackend \| None` | No | Backend for full-text indexing |
| `vector_backend` | `VectorIndexBackend \| None` | No | Backend for embedding indexing |
| `content_reader` | `ContentReader \| None` | No | Reader for resource content (default: `LocationContentReader`) |
| `lifecycle_hook` | `IndexLifecycleHook \| None` | No | Callback for lifecycle events (default: `DefaultLifecycleHook`) |
| `max_triples` | `int` | No | Maximum triples per resource (default: `50_000`). Must be positive. |

### Methods

| Method | Signature | Returns | Description |
|--------|-----------|---------|-------------|
| `index_resource` | `(resource: Resource, *, project: str)` | `IndexResult` | Full indexing pipeline |
| `remove_resource` | `(resource_id: str, *, project: str)` | `None` | Remove from all indices |
| `reindex_resource` | `(resource: Resource, *, project: str)` | `IndexResult` | Remove + re-index |

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `analyzer_registry` | `AnalyzerRegistry` | The analyzer registry |
| `indexed_resource_count` | `int` | Number of currently indexed resources |
| `has_text_backend` | `bool` | Whether text backend is available |
| `has_vector_backend` | `bool` | Whether vector backend is available |

### Graceful Degradation

If `text_backend` or `vector_backend` is `None`, the corresponding indexing
step is silently skipped.  The `graph_backend` is always required.  Error
counts from individual backend failures are recorded in `IndexResult.errors`
without aborting the entire indexing operation.

## Index Backends (Write-side Protocols)

These are **write-side** protocols for indexing.  They are distinct from the
existing read-side query protocols (`TextBackend`, `VectorBackend`,
`GraphBackend`) in `backends.py`.

### TextIndexBackend

| Method | Signature | Returns |
|--------|-----------|---------|
| `index_document` | `(project: str, doc_id: str, content: str, metadata: dict[str, str])` | `IndexedDocument` |
| `search` | `(project: str, query: str, *, limit: int = 20)` | `list[SearchResult]` |
| `remove_document` | `(project: str, doc_id: str)` | `None` |
| `rebuild_index` | `(project: str)` | `None` |

### VectorIndexBackend

| Method | Signature | Returns |
|--------|-----------|---------|
| `index_embedding` | `(project: str, doc_id: str, embedding: list[float], metadata: dict[str, str])` | `None` |
| `search_similar` | `(project: str, query_embedding: list[float], limit: int = 20, min_relevance: float = 0.0)` | `list[SearchResult]` |
| `remove_embedding` | `(project: str, doc_id: str)` | `None` |

### GraphIndexBackend

| Method | Signature | Returns |
|--------|-----------|---------|
| `add_triple` | `(project: str, subject: str, predicate: str, obj: str)` | `None` |
| `query` | `(project: str, sparql: str)` | `list[dict[str, str]]` |
| `remove_triples` | `(project: str, subject: str \| None, predicate: str \| None, obj: str \| None)` | `None` |

## Result Types

### IndexedDocument

| Field | Type | Description |
|-------|------|-------------|
| `project` | `str` | Namespaced project name |
| `doc_id` | `str` | Document identifier |
| `char_count` | `int` | Character count (non-negative, default 0) |

### SearchResult

| Field | Type | Description |
|-------|------|-------------|
| `doc_id` | `str` | Document identifier (non-empty) |
| `content` | `str` | Matched text snippet |
| `score` | `float` | Relevance score in `[0.0, 1.0]` |
| `metadata` | `dict[str, str]` | Backend-specific metadata |

## Provenance

### ProvenanceMetadata

Tracks the origin and validity of indexed triples.

| Field | Type | Description |
|-------|------|-------------|
| `source_resource` | `str` | Resource ULID that produced the triple |
| `source_path` | `str` | File path of the source resource |
| `source_range` | `str` | Line range (e.g. `"10-25"`), empty if whole file |
| `valid_from` | `datetime` | When the triple was indexed (defaults to `now(UTC)`) |
| `is_current` | `bool` | Whether the triple is still current (default: `True`) |

### ProvenancedTriple

Wrapper combining a `UKOTriple` with its `ProvenanceMetadata`.

| Field | Type | Description |
|-------|------|-------------|
| `triple` | `UKOTriple` | The underlying UKO triple |
| `provenance` | `ProvenanceMetadata` | Origin and validity metadata |

### IndexResult

Summary returned by `index_resource()` and `reindex_resource()`.

| Field | Type | Description |
|-------|------|-------------|
| `resource_id` | `str` | ULID of the indexed resource |
| `triple_count` | `int` | Number of triples produced |
| `text_docs_indexed` | `int` | Text documents indexed (0 or 1) |
| `embeddings_indexed` | `int` | Embeddings indexed (0 or 1) |
| `analyzer_domain` | `str` | Domain of the analyzer used |
| `errors` | `tuple[str, ...]` | Non-fatal error messages |

## Protocols

### ContentReader

Protocol for reading resource content.  Decouples the indexer from the
filesystem.

| Method | Signature | Returns |
|--------|-----------|---------|
| `read_content` | `(resource: Resource)` | `str` |

**Default implementation**: `LocationContentReader` — reads from
`resource.location` via the local filesystem.

### IndexLifecycleHook

Callback protocol for index lifecycle events.

| Method | Signature | Returns |
|--------|-----------|---------|
| `on_indexed` | `(result: IndexResult)` | `None` |
| `on_removed` | `(resource_id: str, project: str)` | `None` |
| `on_error` | `(resource_id: str, error: str)` | `None` |

**Default implementation**: `DefaultLifecycleHook` — logs events via
structlog.

## In-Memory Stubs

For testing, three in-memory stub implementations are provided:

- `InMemoryTextIndexBackend` — dict-based full-text storage with substring search
- `InMemoryVectorIndexBackend` — dict-based embedding storage (no real similarity)
- `InMemoryGraphIndexBackend` — list-based triple storage with pattern matching

## ResourceFileWatcher

Monitors filesystem paths via `watchdog` and triggers re-indexing callbacks
on file changes with configurable debouncing.

### Constructor

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `indexer` | `UKOIndexer` | Yes | Indexer for re-index on change |
| `project` | `str` | Yes | Project namespace |
| `debounce_seconds` | `float` | No | Debounce delay (default: `0.5`) |
| `on_change` | `Callable[[str, str, FileChangeType], None] \| None` | No | Optional change callback |
| `event_bus` | `EventBus \| None` | No | Optional event bus for `RESOURCE_MODIFIED` emission |

### FileChangeType

`StrEnum` with values: `CREATED`, `MODIFIED`, `DELETED`, `MOVED`.

### Methods

| Method | Signature | Returns | Description |
|--------|-----------|---------|-------------|
| `watch` | `(resource_id: str, path: Path)` | `None` | Start watching a file path for a resource |
| `unwatch` | `(resource_id: str)` | `None` | Stop watching a resource |
| `shutdown` | `()` | `None` | Stop all observers and clean up |

## Known Limitations

- **Placeholder vector embedding**: The vector backend receives a
  constant `[1.0]` placeholder instead of a real embedding.  Real
  embedding model integration is tracked as a follow-up to issue #578.

## Spec References

- Real-time Index Synchronization: `specification.md` > ACMS > Real-time Index Synchronization
- Custom Index Backends: `specification.md` > ACMS > Custom Index Backends
- Domain Analyzers: `specification.md` > ACMS > Domain Analyzers
- Config keys: `specification.md` > Configuration Service > Index Backend Keys