diff --git a/docs/specification.md b/docs/specification.md index dbe7bd381..9a7036f41 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -45561,6 +45561,87 @@ When advanced features are unavailable, the system gracefully degrades. The pipe 3. Try `semantic-embedding` (requires vector) -> if unavailable: 4. Fall back to `simple-keyword` (requires only text search / ripgrep) +#### Context Tier Hydration + +Before context assembly can occur, the `ContextTierService` must be populated with file fragments from the plan's linked project resources. This is the responsibility of the **context tier hydrator** — a module that bridges the Resource Registry (files on disk) and the ACMS context tier (in-memory fragments). + +##### Module: `context_tier_hydrator` + +**Location**: `src/cleveragents/application/services/context_tier_hydrator.py` + +The hydrator exposes two public functions: + +```python +def hydrate_tiers_for_plan( + tier_service: ContextTierService, + project_names: list[str], + project_repository: ProjectRepository, + resource_registry: ResourceRegistry, +) -> int: + """Hydrate all projects linked to a plan into the tier service. + + Returns the total number of fragments stored across all projects. + """ + +def hydrate_tiers_from_project( + tier_service: ContextTierService, + project_name: str, + resource_id: str, + resource_location: str, + resource_type: str, +) -> int: + """Hydrate a single project resource into the tier service. + + Returns the number of fragments stored for this resource. + """ +``` + +##### File Listing Strategy + +The hydrator uses a resource-type-aware file listing strategy: + +| Resource Type | Strategy | Notes | +|---|---|---| +| `git-checkout` | `git ls-files` | Lists only tracked files; respects `.gitignore` automatically | +| All others | `os.walk` | Recursive filesystem traversal | + +##### Size and Budget Limits + +The hydrator enforces per-resource budget limits to prevent unbounded memory consumption: + +| Limit | Value | Scope | +|---|---|---| +| Maximum file size | 256 KB (262,144 bytes) | Per file — files exceeding this limit are skipped | +| Maximum total size | 10 MB (10,485,760 bytes) | Per project resource — hydration stops when this is reached | + +Files in common non-source directories are automatically skipped: `.git/`, `node_modules/`, `__pycache__/`, `.tox/`, `.venv/`, `dist/`, `build/`, `.mypy_cache/`, `.pytest_cache/`. Binary files (detected by null-byte presence in the first 8 KB) are also skipped. + +##### Fragment Structure + +Each file produces one `TieredFragment` stored in `ContextTierService` with: + +| Field | Value | Notes | +|---|---|---| +| `tier` | `ContextTier.HOT` | All hydrated fragments start in the hot tier | +| `content` | File text content | UTF-8 decoded, errors replaced | +| `metadata["path"]` | Relative file path | Relative to the resource root | +| `metadata["detail_depth"]` | `"10"` | Stored as a **string** (not int) | +| `metadata["relevance_score"]` | `"1.0"` | Stored as a **string** (not float) | + +!!! warning "String Metadata Values" + `detail_depth` and `relevance_score` are stored as strings, not numeric types. This is intentional — the metadata dict is typed as `dict[str, str]` in `TieredFragment`. Consumers that need numeric values must parse them explicitly. + +##### Integration Point + +Hydration runs **before context assembly** in `LLMExecuteActor.execute()`. The call sequence is: + +1. `LLMExecuteActor.execute()` is invoked with the current plan and project list. +2. `hydrate_tiers_for_plan(tier_service, project_names, ...)` is called to populate `ContextTierService`. +3. The ACMS Context Assembly Pipeline runs, reading from the now-populated `ContextTierService`. +4. The assembled context is injected into the LLM prompt. + +Without this hydration step, `ContextTierService` starts empty on every CLI invocation, leaving the LLM with zero file context during plan execution. This was the root cause of bug #1028. + #### ACMS Performance Characteristics ##### Assembly Latency