cleveragents-core/docs/modules/context-tier-hydrator.md

Context Tier Hydrator Module

Package: cleveragents.application.services.context_tier_hydrator Introduced: v3.9.0 (fixes #1028)

The context tier hydrator bridges the gap between the resource registry (files on disk) and the ACMS context tier (in-memory fragments). Without this module, the ContextTierService starts empty on every CLI process invocation and the LLM receives zero file context during plan execution.

For the ACMS context tier architecture, see docs/reference/context_tiers.md. For the full ACMS documentation, see docs/reference/acms.md.

---

Purpose

When a plan is executed, the LLM needs file context from the project being worked on. The ContextTierService is an in-memory store that holds TieredFragment objects representing file contents. However, because the CLI is a short-lived process, the tier service is empty at startup.

context_tier_hydrator.py solves this by reading files from linked project resources (via git ls-files for git-checkout resources, or os.walk for other types) and populating the tier service before context assembly begins.

---

Key Functions

hydrate_tiers_from_project()

Reads files from a single resource and stores them as TieredFragment objects in the ContextTierService.

from cleveragents.application.services.context_tier_hydrator import (
    hydrate_tiers_from_project,
)

fragments_stored = hydrate_tiers_from_project(
    tier_service=tier_service,
    project_name="local/my-project",
    resource_id="01ABCDEF...",
    resource_location="/path/to/project",
    resource_type="git-checkout",  # default
)

Parameters:

Parameter	Type	Description
tier_service	ContextTierService	The tier service to populate
project_name	str	Namespaced project name (e.g. local/my-project)
resource_id	str	ULID of the resource
resource_location	str	Filesystem path to the resource root
resource_type	str	Resource type — affects file listing strategy

Returns: Number of fragments stored.

---

hydrate_tiers_for_plan()

Hydrates tiers for all projects linked to a plan. Called automatically by LLMExecuteActor.execute() before context assembly.

from cleveragents.application.services.context_tier_hydrator import (
    hydrate_tiers_for_plan,
)

total_fragments = hydrate_tiers_for_plan(
    tier_service=tier_service,
    project_names=["local/my-project"],
    project_repository=project_repository,
    resource_registry=resource_registry,
)

---

Limits and Filters

The hydrator applies several guards to prevent excessive memory usage:

Limit	Value	Description
Max file size	256 KB	Files larger than this are skipped
Max total bytes	10 MB	Hydration stops when this budget is reached
Binary extensions	.pyc, .so, .png, .pdf, etc.	Binary files are skipped
Skip directories	.git, __pycache__, node_modules, .venv, etc.	These directories are never traversed

---

File Listing Strategy

For git-checkout resources, the hydrator uses git ls-files to enumerate tracked files. This ensures only version-controlled files are indexed and respects .gitignore rules. For other resource types, os.walk is used with the skip-dir filter applied.

---

Fragment Metadata

Each TieredFragment is stored in the HOT tier with the following metadata:

{
    "path": "relative/path/to/file.py",
    "detail_depth": "1",       # string, not int
    "relevance_score": "0.5",  # string, not float
}

Note: detail_depth and relevance_score must be strings. Passing int or float values causes a Pydantic validation error in ContextFragment (fixed in PR #5998).

---

Integration Point

LLMExecuteActor.execute() calls hydrate_tiers_for_plan() before invoking the context assembler. The actor receives tier_service, project_repository, and resource_registry via constructor injection — no get_container() calls are made from within the service.

# Simplified from llm_actors.py
class LLMExecuteActor:
    def __init__(
        self,
        tier_service: ContextTierService,
        project_repository: NamespacedProjectRepository,
        resource_registry: ResourceRegistryService,
        ...
    ):
        self._tier_service = tier_service
        self._project_repository = project_repository
        self._resource_registry = resource_registry

    def execute(self, plan_id: str, ...) -> ExecuteResult:
        # Hydrate before assembly
        hydrate_tiers_for_plan(
            self._tier_service,
            project_names,
            self._project_repository,
            self._resource_registry,
        )
        # Context assembly now has real file fragments
        context = self._assembler.assemble(...)
        ...

---

Logging

The hydrator emits structured log events:

Event	Level	Description
context_hydrator.skip_missing_location	WARNING	Resource location does not exist
context_hydrator.hydrated	INFO	Hydration complete — reports fragment count and bytes
context_hydrator.project_not_found	DEBUG	Project not found in repository
context_hydrator.store_failed	DEBUG	Failed to store a single fragment

---

Related Fixes

Issue	Description
#1028	ACMS indexing pipeline not wired into CLI — ContextTierService started empty