cleveragents-core/docs/modules/uko-provenance.md

UKO Provenance Tracking

Package: cleveragents.application.services (UKOIndexer, UKOGraphPersistence) Introduced: v3.8.0 (issue #891)

This module guide covers the provenance tracking and temporal versioning capabilities added to the UKO runtime in v3.8.0. Provenance metadata enables audit trails, point-in-time queries, and historical analysis of the knowledge graph.

For the full UKO runtime API reference, see docs/reference/uko_runtime.md. For the temporal data model, see docs/reference/temporal_data_model.md.

---

Purpose

Every typed triple produced by UKOIndexer.index_graph() now carries three provenance fields:

Field	OWL Property	Type	Description
sourceResource	uko:sourceResource	str (ULID)	The CleverAgents resource that produced this triple
validFrom	uko:validFrom	datetime (UTC)	When this triple version became valid
isCurrent	uko:isCurrent	bool	Whether this is the current (non-superseded) version

These fields are defined in UKO Layer 0 (see docs/reference/uko.md).

---

How Provenance Is Attached

When UKOIndexer.index_graph() runs, it calls attach_provenance() on every triple before storing it in the graph backend:

# Conceptual flow (simplified)
triples = analyzer.analyze(content, resource_uri)
implicit = inference_engine.infer(triples)
all_triples = triples + implicit

for triple in all_triples:
    provenanced = ProvenancedTriple(
        triple=triple,
        provenance=ProvenanceMetadata(
            source_resource=resource.id,
            source_path=resource.location,
            valid_from=datetime.now(UTC),
            is_current=True,
        ),
    )
    graph_backend.add_triple(project, provenanced)

---

Revision Chain

A revision chain tracks the full history of ontology state across indexing runs. When a resource is re-indexed:

1. All existing triples for that resource are marked isCurrent=False and validUntil is set to the re-indexing timestamp.
2. New triples are created with isCurrent=True and a fresh validFrom.
3. A new revision entry is appended to the chain, linked to the previous revision.

This enables point-in-time queries: "what did the knowledge graph look like at timestamp T?"

Revision 1 (2026-03-01)  →  Revision 2 (2026-03-15)  →  Revision 3 (2026-04-01)
   isCurrent=False            isCurrent=False              isCurrent=True
   validUntil=2026-03-15      validUntil=2026-04-01        validUntil=None

---

Querying Provenance

Current Triples Only

from cleveragents.application.services.uko_query_interface import UKOQueryInterface

query = UKOQueryInterface(graph_backend=backend, project="local/my-app")

# Classify a resource using only current triples
result = query.classify_resource("01HXYZ...")
# result.layer, result.primary_type, result.all_types, result.implicit_relations

Historical Triples (Point-in-Time)

from datetime import datetime, timezone
from cleveragents.application.services.uko_persistence import UKOGraphPersistence

persistence = UKOGraphPersistence(
    graph_backend=backend,
    project="local/my-app",
    persist_path=Path("~/.config/cleveragents/uko-graph.json").expanduser(),
)

# Query triples valid at a specific point in time
historical = persistence.query_at_time(
    timestamp=datetime(2026, 3, 15, tzinfo=timezone.utc)
)

Triples by Source Resource

# Get all triples (current and historical) from a specific resource
triples = graph_backend.query(
    project="local/my-app",
    sparql="""
    SELECT ?s ?p ?o WHERE {
        ?s ?p ?o .
        ?s uko:sourceResource "01HXYZ..." .
    }
    """
)

---

Persistence Format

UKOGraphPersistence serialises the full graph (including provenance) to JSON. The format includes all triples with their provenance metadata:

{
    "version": "1",
    "project": "local/my-app",
    "revision": 3,
    "triples": [
        {
            "subject": "uko-py:class/AuthService",
            "predicate": "rdf:type",
            "object": "uko-oo:Class",
            "provenance": {
                "source_resource": "01HXYZ...",
                "source_path": "src/auth/service.py",
                "source_range": "15:1-87:0",
                "valid_from": "2026-04-01T10:00:00Z",
                "valid_until": null,
                "is_current": true
            }
        }
    ]
}

---

BDD Coverage

The provenance tracking feature is covered by 47 BDD scenarios in features/uko_provenance.feature:

• Typed triple provenance fields (sourceResource, validFrom, isCurrent)
• Temporal queries (point-in-time, current-only, historical)
• Revision chain creation and traversal
• Graph persistence with provenance (save/restore round-trip)
• Re-indexing marks previous triples as historical

---

Gotchas

• Naive datetimes are rejected. All validFrom and validUntil values must be timezone-aware UTC datetimes. Passing a naive datetime raises ValueError.
• Re-indexing is not atomic. If index_graph() fails partway through, some triples may be marked historical while their replacements were not yet written. The indexer logs a warning in this case; re-running reindex_resource() will repair the state.
• In-memory backend has no persistence. The InMemoryGraphIndexBackend used in tests does not persist across process restarts. Use UKOGraphPersistence with a file backend in production.

---

Related Documentation

• UKO Runtime Services — full API reference
• UKO Ontology — four-layer ontology and provenance property definitions
• UKO Indexer — ProvenanceMetadata and ProvenancedTriple types
• Temporal Data Model — TemporalNode, RevisionChain, tier storage
• ACMS — how provenance feeds into context assembly