docs(modules): refine context tier hydrator docs

- document metadata types accurately and wrap long lines

- reflow changelog entries for hydrator and apply-phase fixes

ISSUES CLOSED: #6851

CHANGELOG.md

@@ -14,11 +14,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   Cleanup panel, and `✓ OK Changes applied` footer.  Non-git projects fall
   back to the original flat file copy.
 
-- **Context Hydration Fix** (#4454): Fixed `ContextFragment` metadata types
-  (`detail_depth` and `relevance_score` must be strings, not int/float) that
-  caused Pydantic validation errors during context assembly, resulting in the
-  LLM receiving zero file context.
-
 - **Automation Tracking System**: Replaced shared session state issue tracking with
   individual per-agent tracking issues. Each agent now creates its own `[AUTO-<PREFIX>]`
   titled issues with standardized headers, reporting intervals, and health indicators.
@@ -27,24 +22,24 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   `docs/development/automation-tracking.md`.
 
 - **Automated Health Monitoring and Recovery**: The `system-watchdog` now runs
-  `audit_automation_tracking_health()` every 5 minutes, detecting stalled agents when
-  tracking issues are >20% overdue from their declared reporting interval. On detection,
-  it terminates stalled sessions via the OpenCode Server API, performs root-cause analysis,
-  creates high-priority diagnostic issues, and closes stale tracking issues with recovery
-  notes.
+  `audit_automation_tracking_health()` every 5 minutes. It flags agents whose tracking
+  issues are more than 20% overdue relative to their reporting interval. When triggered,
+  the watchdog terminates stalled sessions through the OpenCode Server API, performs
+  root-cause analysis, files high-priority diagnostic issues, and closes stale tracking
+  issues with recovery notes.
 
-- **Centralized Label Management** (`forgejo-label-manager`): A new specialized subagent
-  centralizes all Forgejo label operations across the agent system. Enforces the
-  organization-level label system, prohibits label creation, and validates label compliance.
-  Agents `backlog-groomer`, `human-liaison`, `project-owner`, `epic-planner`,
-  `new-issue-creator`, and `issue-state-updater` now delegate all label operations to this
-  subagent.
+- **Centralized Label Management** (`forgejo-label-manager`): A new subagent now
+  centralizes Forgejo label operations across every automation agent. It
+  enforces the organization-wide label set, forbids ad-hoc label creation,
+  validates compliance, and serves requests from agents such as
+  `backlog-groomer`, `human-liaison`, and `project-owner`. It also supports
+  `epic-planner`, `new-issue-creator`, and `issue-state-updater`.
 
-- **PR–Issue Label Synchronization**: PRs now inherit `Priority/`, `MoSCoW/`, `Points/`,
-  and `State/` labels from their associated issues at creation time
-  (`pr-api-creator`). The `backlog-groomer` adds a continuous Pass 19 for ongoing
-  PR–issue label synchronization. The `issue-state-updater` syncs PR state labels whenever
-  issue states change.
+- **PR–Issue Label Synchronization**: `pr-api-creator` now copies `Priority/`,
+  `MoSCoW/`, `Points/`, and `State/` labels from linked issues when a PR opens.
+  `backlog-groomer` adds a continuous Pass 19 task to keep labels synchronized,
+  and `issue-state-updater` refreshes PR state labels whenever the issue state
+  changes.
 
 - **Automation Tracking Announcements**: Extended `automation-tracking-manager` with
   announcement issue support (`CREATE_ANNOUNCEMENT_ISSUE`, `CLOSE_ANNOUNCEMENT_ISSUE`,
@@ -103,9 +98,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   ID reconstruction. Added "Decision IDs (for correction)" section with human-readable
   labels for easy reference. Applies to both table and rich/plain text output formats.
 
-- **Automation Tracking Format**: All automation tracking issues now use a standardized
-  header format with mandatory `Reporting Interval: <interval> (Next report expected: <ts>)`
-  declarations, enabling precise staleness detection.
+- **Automation Tracking Format**: Automation tracking issues now use a
+  standardized header with a `Reporting Interval: <interval> (Next report
+  expected: <ts>)` line. The consistent metadata makes overdue detection
+  precise.
 
 - **PR Review Policy**: Reduced PR review requirement from 2 approvals to 1. Self-approval
   is now permitted including for automated bot PRs. Approval can be a formal review OR an
@@ -125,17 +121,32 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   `AUTO-INF-POOL`, `AUTO-ARCH`, `AUTO-EPIC`, `AUTO-EVLV`, `AUTO-GUARD`, `AUTO-SPEC`,
   `AUTO-TIME`, `AUTO-PROJ-OWN`, and `AUTO-PROD-BLDR`.
 
-- **ACMS Context Hydration**: Fixed ACMS indexing pipeline not wired into CLI —
-  `ContextTierService` started empty on every CLI invocation so LLM received zero file
-  context during plan execution. Added `context_tier_hydrator.py` that reads files from
-  linked project resources (via `git ls-files` or `os.walk`) and stores them as
-  `TieredFragment` objects in the tier service. Hydration runs automatically before context
-  assembly in `LLMExecuteActor.execute()`. Respects max file size (256KB), total budget
-  (10MB), binary file exclusion, and `.git`/`node_modules`/`__pycache__` directory
-  skipping. (#1028)
-
 ### Fixed
 
+- **ACMS context tier hydration** (`context_tier_hydrator`):
+  `ContextTierService` used to start empty on each CLI invocation, so the LLM
+  received zero file context. `context_tier_hydrator.py` now reads files from
+  linked resources with `git ls-files` for git checkouts and falls back to
+  `os.walk` for other types. The hydrator stores results as `TieredFragment`
+  entries in the tier service. It runs before context assembly in
+  `LLMExecuteActor.execute()`. Limits remain 256 KB per file and 10 MB overall,
+  and binary extensions plus build directories such as `.git`, `node_modules`,
+  `__pycache__`, and `.venv` are skipped. (#1028, #4219)
+
+- **Apply phase file writing** (`plan apply`): The CLI now copies files from
+  the sandbox directory into the project directory while enforcing path
+  traversal guards. It skips protected directories such as `.cleveragents`,
+  `.git`, `.hg`, and `.svn`, handles per-file failures without discarding the
+  sandbox, and only cleans up on full success. The sandbox root is also wired
+  to `.cleveragents/sandbox/` in `_get_plan_executor()`. (#4222, #4219)
+
+- **`plan use` UNIQUE constraint violation** (`ActionRepository.update`):
+  Replaced the `.clear()` plus `.append()` pattern with explicit `sa_delete()`
+  calls followed by `session.flush()` before re-inserting related rows. Added
+  `session.expire()` so the collections reload and avoid the stale identity-map
+  references that triggered `sqlite3.IntegrityError` when reusing actions.
+  (#4174, #4197)
+
 - **Validation Gate Empty-Run Guard** (#7508): Fixed `ApplyValidationSummary.all_required_passed`
   returning `True` when zero validations were run, silently bypassing the apply gate. The property
   now returns `False` when the validation result set is empty (`is_empty` is `True`), ensuring

docs/modules/context-tier-hydrator.md

@@ -0,0 +1,198 @@
+# Context Tier Hydrator
+
+**Module:** `cleveragents.application.services.context_tier_hydrator`
+
+The context tier hydrator bridges the gap between the resource registry (files
+on disk) and the ACMS context tier (in-memory fragments). Without it, the
+`ContextTierService` starts empty on every CLI process invocation and the LLM
+receives zero file context during plan execution.
+
+This module resolves issue [#1028].
+
+---
+
+## Overview
+
+When a plan is executed, the LLM needs file context from the project's linked
+resources. The hydrator:
+
+1. Iterates over all projects linked to the plan
+2. For each project, finds its linked resources
+3. For each resource, lists files using `git ls-files` (git-checkout) or `os.walk`
+4. Reads each file and stores it as a `TieredFragment` in `ContextTierService`
+5. Respects size and budget limits to avoid overwhelming the LLM context window
+
+Hydration runs automatically before context assembly in
+`LLMExecuteActor.execute()`.
+
+---
+
+## Functions
+
+### `hydrate_tiers_for_plan`
+
+```python
+from cleveragents.application.services.context_tier_hydrator import (
+    hydrate_tiers_for_plan,
+)
+
+total_fragments = hydrate_tiers_for_plan(
+    tier_service=tier_service,
+    project_names=["local/my-project"],
+    project_repository=project_repo,
+    resource_registry=resource_registry,
+)
+```
+
+Hydrates tiers for all projects linked to a plan. This is the primary entry
+point called by `LLMExecuteActor`.
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `tier_service` | `ContextTierService` | The tier service to populate |
+| `project_names` | `list[str]` | Namespaced project names (e.g. `local/my-project`) |
+| `project_repository` | `NamespacedProjectRepository` | Project metadata repository |
+| `resource_registry` | `ResourceRegistryService` | Resolves resource locations |
+
+**Returns:** Total number of `TieredFragment` objects stored across all projects.
+
+---
+
+### `hydrate_tiers_from_project`
+
+```python
+from cleveragents.application.services.context_tier_hydrator import (
+    hydrate_tiers_from_project,
+)
+
+count = hydrate_tiers_from_project(
+    tier_service=tier_service,
+    project_name="local/my-project",
+    resource_id="01ABCDEF...",
+    resource_location="/path/to/project",
+    resource_type="git-checkout",
+)
+```
+
+Reads files from a single resource and stores them as `TieredFragment` objects.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `tier_service` | `ContextTierService` | — | The tier service to populate |
+| `project_name` | `str` | — | Namespaced project name |
+| `resource_id` | `str` | — | ULID of the resource |
+| `resource_location` | `str` | — | Filesystem path to the resource root |
+| `resource_type` | `str` | `"git-checkout"` | Resource type; selects listing mode |
+
+**Returns:** Number of fragments stored.
+
+**Behaviour when `resource_location` is missing or not a directory:** logs a
+warning and returns `0`.
+
+---
+
+## Limits and Filters
+
+| Limit | Value | Description |
+|-------|-------|-------------|
+| Max file size | 256 KB | Files larger than this are skipped |
+| Max total budget | 10 MB | Hydration stops when this is reached |
+
+### Skipped directories
+
+`.git`, `.hg`, `.svn`, `__pycache__`, `node_modules`, `.venv`, `venv`,
+`.nox`, `.tox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`,
+`build`, `.eggs`, `.cleveragents`
+
+### Skipped file extensions (binary)
+
+`.pyc`, `.pyo`, `.so`, `.o`, `.a`, `.dll`, `.exe`, `.png`, `.jpg`, `.jpeg`,
+`.gif`, `.bmp`, `.ico`, `.pdf`, `.zip`, `.tar`, `.gz`, `.bz2`, `.xz`,
+`.whl`, `.egg`, `.db`, `.sqlite`, `.sqlite3`
+
+---
+
+## File Listing Strategy
+
+| Resource type | Strategy |
+|---------------|----------|
+| `git-checkout`, `git` | `git ls-files --cached --others --exclude-standard` |
+| All others | `os.walk` with directory and extension filters |
+
+For git-checkout resources, `git ls-files` is preferred because it respects
+`.gitignore` and only returns tracked/untracked-but-not-ignored files. If the
+`git` command fails (e.g. not a git repo), the hydrator falls back to
+`os.walk`.
+
+---
+
+## Fragment Structure
+
+Each file is stored as a `TieredFragment` with:
+
+```python
+TieredFragment(
+    fragment_id=f"{resource_id}:{rel_path}",
+    content=file_content,
+    tier=ContextTier.HOT,
+    resource_id=resource_id,
+    project_name=project_name,
+    token_count=len(content) // 4,   # approximate
+    metadata={
+        "path": rel_path,
+        "detail_depth": "1",
+        "relevance_score": "0.5",
+    },
+)
+```
+
+All fragments are stored at `ContextTier.HOT` (always included in context
+assembly). Token count is approximated as `len(content) // 4`.
+
+---
+
+## Integration with `LLMExecuteActor`
+
+The hydrator is called inside `LLMExecuteActor.execute()` before context
+assembly:
+
+```python
+# Simplified from llm_actors.py
+from cleveragents.application.services.context_tier_hydrator import (
+    hydrate_tiers_for_plan,
+)
+
+class LLMExecuteActor:
+    def execute(self, plan, ...):
+        # Hydrate context tiers from project resources
+        hydrate_tiers_for_plan(
+            tier_service=self._tier_service,
+            project_names=plan.project_names,
+            project_repository=self._project_repository,
+            resource_registry=self._resource_registry,
+        )
+        # Then assemble context and call LLM
+        context = self._assembler.assemble(plan)
+        ...
+```
+
+Dependencies (`tier_service`, `project_repository`, `resource_registry`) are
+injected at the CLI factory level in `_get_plan_executor()`, not via
+`get_container()` calls inside services.
+
+---
+
+## Logging
+
+The hydrator uses `structlog` with the following event keys:
+
+| Event | Level | Description |
+|-------|-------|-------------|
+| `context_hydrator.skip_missing_location` | WARNING | Missing path or not directory |
+| `context_hydrator.hydrated` | INFO | Hydration complete — reports fragments + bytes |
+| `context_hydrator.project_not_found` | DEBUG | Project not found in repository |
+| `context_hydrator.no_linked_resources` | DEBUG | Project has no linked resources |
+| `context_hydrator.resource_not_found` | DEBUG | Resource not found in registry |
+| `context_hydrator.store_failed` | DEBUG | Fragment store failed (non-fatal) |
+
+[#1028]: https://git.cleverthis.com/cleveragents/cleveragents-core/issues/1028

mkdocs.yml

@@ -27,6 +27,7 @@ nav:
     - Shell Safety: modules/shell-safety.md
     - UKO Provenance Tracking: modules/uko-provenance.md
     - Invariant Reconciliation: modules/invariant-reconciliation.md
+    - Context Tier Hydrator: modules/context-tier-hydrator.md
   - Development:
     - Agent System Specification: development/agent-system-specification.md
     - CI/CD Pipeline: development/ci-cd.md