docs(spec): document context_tier_hydrator module in ACMS architecture section #6180

Closed
HAL9000 wants to merge 1 commits from spec/document-context-tier-hydrator-6175 into master
+81
View File
@@ -45561,6 +45561,87 @@ When advanced features are unavailable, the system gracefully degrades. The pipe
3. Try `semantic-embedding` (requires vector) -> if unavailable:
4. Fall back to `simple-keyword` (requires only text search / ripgrep)
#### Context Tier Hydration
Before context assembly can occur, the `ContextTierService` must be populated with file fragments from the plan's linked project resources. This is the responsibility of the **context tier hydrator** — a module that bridges the Resource Registry (files on disk) and the ACMS context tier (in-memory fragments).
##### Module: `context_tier_hydrator`
**Location**: `src/cleveragents/application/services/context_tier_hydrator.py`
The hydrator exposes two public functions:
```python
def hydrate_tiers_for_plan(
tier_service: ContextTierService,
project_names: list[str],
project_repository: ProjectRepository,
resource_registry: ResourceRegistry,
) -> int:
"""Hydrate all projects linked to a plan into the tier service.
Returns the total number of fragments stored across all projects.
"""
def hydrate_tiers_from_project(
tier_service: ContextTierService,
project_name: str,
resource_id: str,
resource_location: str,
resource_type: str,
) -> int:
"""Hydrate a single project resource into the tier service.
Returns the number of fragments stored for this resource.
"""
```
##### File Listing Strategy
The hydrator uses a resource-type-aware file listing strategy:
| Resource Type | Strategy | Notes |
|---|---|---|
| `git-checkout` | `git ls-files` | Lists only tracked files; respects `.gitignore` automatically |
| All others | `os.walk` | Recursive filesystem traversal |
##### Size and Budget Limits
The hydrator enforces per-resource budget limits to prevent unbounded memory consumption:
| Limit | Value | Scope |
|---|---|---|
| Maximum file size | 256 KB (262,144 bytes) | Per file — files exceeding this limit are skipped |
| Maximum total size | 10 MB (10,485,760 bytes) | Per project resource — hydration stops when this is reached |
Files in common non-source directories are automatically skipped: `.git/`, `node_modules/`, `__pycache__/`, `.tox/`, `.venv/`, `dist/`, `build/`, `.mypy_cache/`, `.pytest_cache/`. Binary files (detected by null-byte presence in the first 8 KB) are also skipped.
##### Fragment Structure
Each file produces one `TieredFragment` stored in `ContextTierService` with:
| Field | Value | Notes |
|---|---|---|
| `tier` | `ContextTier.HOT` | All hydrated fragments start in the hot tier |
| `content` | File text content | UTF-8 decoded, errors replaced |
| `metadata["path"]` | Relative file path | Relative to the resource root |
| `metadata["detail_depth"]` | `"10"` | Stored as a **string** (not int) |
| `metadata["relevance_score"]` | `"1.0"` | Stored as a **string** (not float) |
!!! warning "String Metadata Values"
`detail_depth` and `relevance_score` are stored as strings, not numeric types. This is intentional — the metadata dict is typed as `dict[str, str]` in `TieredFragment`. Consumers that need numeric values must parse them explicitly.
##### Integration Point
Hydration runs **before context assembly** in `LLMExecuteActor.execute()`. The call sequence is:
1. `LLMExecuteActor.execute()` is invoked with the current plan and project list.
2. `hydrate_tiers_for_plan(tier_service, project_names, ...)` is called to populate `ContextTierService`.
3. The ACMS Context Assembly Pipeline runs, reading from the now-populated `ContextTierService`.
4. The assembled context is injected into the LLM prompt.
Without this hydration step, `ContextTierService` starts empty on every CLI invocation, leaving the LLM with zero file context during plan execution. This was the root cause of bug #1028.
#### ACMS Performance Characteristics
##### Assembly Latency