cleveragents-core/docs/modules/context-hydration.md

# ACMS Context Hydration

**Package:** `cleveragents.application.services.context_tier_hydrator`
**Introduced:** Unreleased (issue #1028)

The context hydration module bridges the gap between the resource registry
(files on disk) and the ACMS context tier (in-memory fragments).  Without
it, the `ContextTierService` starts empty on every CLI process invocation
and the LLM receives zero file context during plan execution.

---

## Problem Solved

Before this module, the ACMS indexing pipeline was not wired into the CLI.
On each invocation, `ContextTierService` started with an empty fragment
store, so `LLMExecuteActor.execute()` assembled context with no file
content.  The LLM therefore had no knowledge of the project's source files
when generating plans.

This module fixes that by reading files from all resources linked to the
active project and storing them as `TieredFragment` objects in the tier
service before context assembly begins.

---

## Key Functions

### `hydrate_tiers_from_project`

```python
from cleveragents.application.services.context_tier_hydrator import (
    hydrate_tiers_from_project,
)

count = hydrate_tiers_from_project(
    tier_service=tier_service,
    project_name="local/my-project",
    resource_id="01HZ...",
    resource_location="/path/to/project",
    resource_type="git-checkout",  # default
)
# count — number of TieredFragment objects stored
```

Reads files from a single resource directory and stores them as
`TieredFragment` objects in `ContextTierService`.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `tier_service` | `ContextTierService` | The service to populate |
| `project_name` | `str` | Namespaced project name (e.g. `local/my-project`) |
| `resource_id` | `str` | ULID of the resource |
| `resource_location` | `str` | Filesystem path to the resource root |
| `resource_type` | `str` | Resource type name; affects file listing strategy |

**Returns:** Number of fragments stored.

---

### `hydrate_tiers_for_plan`

```python
from cleveragents.application.services.context_tier_hydrator import (
    hydrate_tiers_for_plan,
)

total = hydrate_tiers_for_plan(
    tier_service=tier_service,
    project_names=["local/my-project"],
    project_repository=project_repo,
    resource_registry=resource_registry,
)
```

Hydrates tiers for all projects linked to a plan.  Iterates over each
project name, resolves linked resources via the project repository and
resource registry, and calls `hydrate_tiers_from_project` for each.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `tier_service` | `ContextTierService` | The service to populate |
| `project_names` | `list[str]` | Namespaced project names |
| `project_repository` | `NamespacedProjectRepository` | Repository for project lookup |
| `resource_registry` | `ResourceRegistryService` | Registry for resource lookup |

**Returns:** Total number of fragments stored across all projects.

---

## File Listing Strategy

The hydrator uses two strategies to enumerate files:

| Strategy | When Used | How |
|----------|-----------|-----|
| `git ls-files` | `resource_type` is `git-checkout` or `git` | Lists tracked and untracked (non-ignored) files via `git ls-files --cached --others --exclude-standard` |
| `os.walk` | All other resource types, or if `git ls-files` fails | Walks the directory tree, skipping hidden directories and binary extensions |

---

## Limits and Exclusions

To avoid overwhelming the context window or indexing irrelevant content,
the hydrator enforces these limits:

| Limit | Value | Description |
|-------|-------|-------------|
| Max file size | 256 KB | Files larger than this are skipped |
| Max total bytes | 10 MB | Indexing stops once this budget is reached |

**Skipped directories** (always excluded):

`.git`, `.hg`, `.svn`, `__pycache__`, `node_modules`, `.venv`, `venv`,
`.nox`, `.tox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`,
`build`, `.eggs`, `.cleveragents`

**Skipped file extensions** (binary files):

`.pyc`, `.pyo`, `.so`, `.o`, `.a`, `.dll`, `.exe`, `.png`, `.jpg`,
`.jpeg`, `.gif`, `.bmp`, `.ico`, `.pdf`, `.zip`, `.tar`, `.gz`, `.bz2`,
`.xz`, `.whl`, `.egg`, `.db`, `.sqlite`, `.sqlite3`

---

## Fragment Metadata

Each stored `TieredFragment` carries:

```python
TieredFragment(
    fragment_id=f"{resource_id}:{rel_path}",
    content=<file content>,
    tier=ContextTier.HOT,
    resource_id=resource_id,
    project_name=project_name,
    token_count=len(content) // 4,  # approximate
    metadata={
        "path": rel_path,
        "detail_depth": "1",
        "relevance_score": "0.5",
    },
)
```

All fragments are stored in the `HOT` tier (always-present context).
`detail_depth` and `relevance_score` are stored as strings to satisfy
`ContextFragment` Pydantic validation requirements.

---

## Integration Point

Hydration runs automatically before context assembly in
`LLMExecuteActor.execute()`.  No manual invocation is required for normal
plan execution.

```python
# Conceptual flow inside LLMExecuteActor.execute()
hydrate_tiers_for_plan(
    tier_service=self._tier_service,
    project_names=plan.project_names,
    project_repository=self._project_repository,
    resource_registry=self._resource_registry,
)
context = self._context_assembler.assemble(plan_id=plan.id)
response = await self._llm.invoke(context)
```

---

## Related

- [ACMS Architecture](../architecture.md#context-management-acms)
- [ADR-014 Context Management](../adr/ADR-014-context-management-acms.md)
- [`cleveragents.acms`](../api/core.md)