From 48532de1cdec2e8c3f30b9db8c726fc23240e51c Mon Sep 17 00:00:00 2001 From: CleverThis Date: Thu, 9 Apr 2026 17:27:47 +0000 Subject: [PATCH] docs: add context hydration and git worktree sandbox module docs - Add docs/modules/context-hydration.md: documents the ACMS context hydration pipeline (context_tier_hydrator) that fixes the empty ContextTierService bug (#1028). Covers hydrate_tiers_from_project, hydrate_tiers_for_plan, file listing strategies, limits, and fragment metadata. - Add docs/modules/git-worktree-sandbox.md: documents the GitWorktreeSandbox class that isolates plan apply changes in a dedicated git branch/worktree and merges back on commit (#4454). Covers full lifecycle, error types, branch naming, and fallback for non-git projects. - Update docs/architecture.md: add Git Worktree Sandbox Apply section and extend ACMS section with context hydration note. - Update mkdocs.yml: add both new module pages to the Modules nav. ISSUES CLOSED: #6841 --- docs/architecture.md | 32 +++++ docs/modules/context-hydration.md | 180 +++++++++++++++++++++++++ docs/modules/git-worktree-sandbox.md | 190 +++++++++++++++++++++++++++ mkdocs.yml | 2 + 4 files changed, 404 insertions(+) create mode 100644 docs/modules/context-hydration.md create mode 100644 docs/modules/git-worktree-sandbox.md diff --git a/docs/architecture.md b/docs/architecture.md index e355f0c5e..76ea25633 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -217,6 +217,13 @@ Key UKO capabilities: - **Graph persistence** — `UKOGraphPersistence` serialises/restores state via JSON-file or in-memory backends across application restarts +**Context hydration** bridges the resource registry and the ACMS tier service. +`context_tier_hydrator.hydrate_tiers_for_plan()` reads files from all +resources linked to the active project and stores them as `TieredFragment` +objects in `ContextTierService` before context assembly begins. Without +hydration, the LLM receives zero file context. See +[ACMS Context Hydration](modules/context-hydration.md) for details. + --- ## Invariant Reconciliation @@ -259,6 +266,31 @@ result = actor.run(plan_id="...", project_name="...", action_name="...") --- +## Git Worktree Sandbox Apply + +When `plan apply` commits changes to a git-checkout resource, the changes +are isolated in a dedicated git branch and worktree rather than written +directly to the working tree. + +**Flow:** + +1. `PlanLifecycleService` detects a git-checkout resource and creates a + `GitWorktreeSandbox`. +2. A new branch `cleveragents/plan-` is created from the current HEAD. +3. A temporary worktree is created at a system temp directory. +4. The actor writes changes inside the worktree. +5. On approval: changes are staged, committed in the worktree, then merged + back to the original branch via `git merge`. +6. On rejection: the worktree is discarded and the original branch is + untouched. + +Non-git projects fall back to the original flat file copy strategy. + +See [Git Worktree Sandbox](modules/git-worktree-sandbox.md) for the full +API reference. + +--- + ## A2A Protocol Boundary All CLI, TUI, and server interactions cross the A2A boundary diff --git a/docs/modules/context-hydration.md b/docs/modules/context-hydration.md new file mode 100644 index 000000000..7a9a290df --- /dev/null +++ b/docs/modules/context-hydration.md @@ -0,0 +1,180 @@ +# ACMS Context Hydration + +**Package:** `cleveragents.application.services.context_tier_hydrator` +**Introduced:** Unreleased (issue #1028) + +The context hydration module bridges the gap between the resource registry +(files on disk) and the ACMS context tier (in-memory fragments). Without +it, the `ContextTierService` starts empty on every CLI process invocation +and the LLM receives zero file context during plan execution. + +--- + +## Problem Solved + +Before this module, the ACMS indexing pipeline was not wired into the CLI. +On each invocation, `ContextTierService` started with an empty fragment +store, so `LLMExecuteActor.execute()` assembled context with no file +content. The LLM therefore had no knowledge of the project's source files +when generating plans. + +This module fixes that by reading files from all resources linked to the +active project and storing them as `TieredFragment` objects in the tier +service before context assembly begins. + +--- + +## Key Functions + +### `hydrate_tiers_from_project` + +```python +from cleveragents.application.services.context_tier_hydrator import ( + hydrate_tiers_from_project, +) + +count = hydrate_tiers_from_project( + tier_service=tier_service, + project_name="local/my-project", + resource_id="01HZ...", + resource_location="/path/to/project", + resource_type="git-checkout", # default +) +# count — number of TieredFragment objects stored +``` + +Reads files from a single resource directory and stores them as +`TieredFragment` objects in `ContextTierService`. + +**Parameters:** + +| Parameter | Type | Description | +|-----------|------|-------------| +| `tier_service` | `ContextTierService` | The service to populate | +| `project_name` | `str` | Namespaced project name (e.g. `local/my-project`) | +| `resource_id` | `str` | ULID of the resource | +| `resource_location` | `str` | Filesystem path to the resource root | +| `resource_type` | `str` | Resource type name; affects file listing strategy | + +**Returns:** Number of fragments stored. + +--- + +### `hydrate_tiers_for_plan` + +```python +from cleveragents.application.services.context_tier_hydrator import ( + hydrate_tiers_for_plan, +) + +total = hydrate_tiers_for_plan( + tier_service=tier_service, + project_names=["local/my-project"], + project_repository=project_repo, + resource_registry=resource_registry, +) +``` + +Hydrates tiers for all projects linked to a plan. Iterates over each +project name, resolves linked resources via the project repository and +resource registry, and calls `hydrate_tiers_from_project` for each. + +**Parameters:** + +| Parameter | Type | Description | +|-----------|------|-------------| +| `tier_service` | `ContextTierService` | The service to populate | +| `project_names` | `list[str]` | Namespaced project names | +| `project_repository` | `NamespacedProjectRepository` | Repository for project lookup | +| `resource_registry` | `ResourceRegistryService` | Registry for resource lookup | + +**Returns:** Total number of fragments stored across all projects. + +--- + +## File Listing Strategy + +The hydrator uses two strategies to enumerate files: + +| Strategy | When Used | How | +|----------|-----------|-----| +| `git ls-files` | `resource_type` is `git-checkout` or `git` | Lists tracked and untracked (non-ignored) files via `git ls-files --cached --others --exclude-standard` | +| `os.walk` | All other resource types, or if `git ls-files` fails | Walks the directory tree, skipping hidden directories and binary extensions | + +--- + +## Limits and Exclusions + +To avoid overwhelming the context window or indexing irrelevant content, +the hydrator enforces these limits: + +| Limit | Value | Description | +|-------|-------|-------------| +| Max file size | 256 KB | Files larger than this are skipped | +| Max total bytes | 10 MB | Indexing stops once this budget is reached | + +**Skipped directories** (always excluded): + +`.git`, `.hg`, `.svn`, `__pycache__`, `node_modules`, `.venv`, `venv`, +`.nox`, `.tox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`, +`build`, `.eggs`, `.cleveragents` + +**Skipped file extensions** (binary files): + +`.pyc`, `.pyo`, `.so`, `.o`, `.a`, `.dll`, `.exe`, `.png`, `.jpg`, +`.jpeg`, `.gif`, `.bmp`, `.ico`, `.pdf`, `.zip`, `.tar`, `.gz`, `.bz2`, +`.xz`, `.whl`, `.egg`, `.db`, `.sqlite`, `.sqlite3` + +--- + +## Fragment Metadata + +Each stored `TieredFragment` carries: + +```python +TieredFragment( + fragment_id=f"{resource_id}:{rel_path}", + content=, + tier=ContextTier.HOT, + resource_id=resource_id, + project_name=project_name, + token_count=len(content) // 4, # approximate + metadata={ + "path": rel_path, + "detail_depth": "1", + "relevance_score": "0.5", + }, +) +``` + +All fragments are stored in the `HOT` tier (always-present context). +`detail_depth` and `relevance_score` are stored as strings to satisfy +`ContextFragment` Pydantic validation requirements. + +--- + +## Integration Point + +Hydration runs automatically before context assembly in +`LLMExecuteActor.execute()`. No manual invocation is required for normal +plan execution. + +```python +# Conceptual flow inside LLMExecuteActor.execute() +hydrate_tiers_for_plan( + tier_service=self._tier_service, + project_names=plan.project_names, + project_repository=self._project_repository, + resource_registry=self._resource_registry, +) +context = self._context_assembler.assemble(plan_id=plan.id) +response = await self._llm.invoke(context) +``` + +--- + +## Related + +- [ACMS Architecture](../architecture.md#context-management-acms) +- [ADR-014 Context Management](../adr/ADR-014-context-management-acms.md) +- [`cleveragents.acms`](../api/core.md) diff --git a/docs/modules/git-worktree-sandbox.md b/docs/modules/git-worktree-sandbox.md new file mode 100644 index 000000000..e10c5a208 --- /dev/null +++ b/docs/modules/git-worktree-sandbox.md @@ -0,0 +1,190 @@ +# Git Worktree Sandbox + +**Package:** `cleveragents.infrastructure.sandbox.git_worktree` +**Introduced:** Unreleased (issue #4454) + +The git worktree sandbox isolates plan modifications in a dedicated git +branch and worktree. On commit, changes are merged back to the original +branch via `git merge`. On rollback, the worktree is discarded entirely. +This replaces the previous flat `shutil.copy2` apply strategy for +git-checkout resources. + +--- + +## Purpose + +When a plan applies changes to a git-checkout resource, the changes must +be isolated from the working tree until the user approves them. The git +worktree sandbox achieves this by: + +1. Creating a new branch `cleveragents/plan-` from the current HEAD. +2. Creating a temporary git worktree at a system temp directory. +3. Letting the actor write changes inside the worktree. +4. On commit: staging all changes, committing them in the worktree, then + merging the sandbox branch back into the original branch. +5. On rollback: resetting the worktree (and optionally the original branch) + to the base commit and discarding the worktree. + +Non-git projects fall back to the original flat file copy strategy. + +--- + +## `GitWorktreeSandbox` + +```python +from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox + +sandbox = GitWorktreeSandbox( + resource_id="01HZ...", + original_path="/path/to/repo", +) +``` + +Implements the `Sandbox` protocol. + +**Constructor parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `resource_id` | `str` | — | Identifier of the resource being sandboxed | +| `original_path` | `str` | — | Path to the git repository root | +| `git_timeout` | `int` | `30` | Timeout in seconds for git commands | + +--- + +## Lifecycle + +``` +PENDING ──► create() ──► CREATED ──► get_path() ──► ACTIVE + │ + commit() ─┤─► COMMITTED ──► cleanup() ──► CLEANED_UP + │ + rollback() ─┴─► ROLLED_BACK ──► cleanup() ──► CLEANED_UP +``` + +### `create(plan_id) → SandboxContext` + +Creates the worktree and branch. + +```python +ctx = sandbox.create(plan_id="plan-01HZ...") +# ctx.sandbox_path — path to the worktree +# ctx.metadata["branch"] — e.g. "cleveragents/plan-plan-01HZ..." +# ctx.metadata["base_commit"] — HEAD SHA at creation time +``` + +Raises `SandboxCreationError` if the path is not a git repository root or +if `git worktree add` fails. + +### `get_path(resource_path) → str` + +Translates a resource-relative path to an absolute path inside the +worktree. Rejects `..` path traversal attempts. + +```python +worktree_file = sandbox.get_path("src/main.py") +# → "/tmp/ca-sandbox-plan-01HZ.../src/main.py" +``` + +### `commit(message=None) → CommitResult` + +Stages all changes in the worktree, commits them, then merges the sandbox +branch into the original branch. + +```python +result = sandbox.commit("feat: implement new feature") +# result.success — True on success +# result.commit_ref — SHA of the sandbox commit +# result.changed_files — list of modified file paths +# result.added_files — list of newly added file paths +# result.deleted_files — list of deleted file paths +``` + +If there are no staged changes, returns a `CommitResult` with empty file +lists and no new commit. + +Raises `SandboxCommitError` if the commit or merge fails. + +### `rollback()` + +Discards all worktree changes. + +- From `ACTIVE`: resets the worktree branch to the base commit. +- From `COMMITTED`: also resets the original branch to the pre-merge commit + (fully undoes the merge). + +!!! warning "Multi-worktree safety" + Rolling back from `COMMITTED` executes `git reset --hard` on the + original branch. If other git worktrees or external processes are + tracking the same branch, they will be affected. + +### `cleanup()` + +Removes the worktree directory and deletes the sandbox branch. Idempotent +— safe to call multiple times. + +--- + +## Apply Summary + +When `plan apply` uses the git worktree strategy, the CLI displays a +spec-aligned Apply Summary: + +``` +╭─ Apply Summary ──────────────────────────────────────╮ +│ Plan ID: plan-01HZ... │ +│ Artifacts: 3 files changed │ +│ Insertions: +142 │ +│ Deletions: -17 │ +│ Project: local/my-project │ +│ Timestamp: 2026-04-09 17:30:00 │ +╰──────────────────────────────────────────────────────╯ +╭─ Sandbox Cleanup ────────────────────────────────────╮ +│ ✓ Worktree removed │ +│ ✓ Branch deleted │ +╰──────────────────────────────────────────────────────╯ +✓ OK Changes applied +``` + +--- + +## Error Types + +| Exception | When Raised | +|-----------|-------------| +| `SandboxCreationError` | `git worktree add` fails or path is not a git root | +| `SandboxCommitError` | `git commit` or `git merge` fails | +| `SandboxRollbackError` | `git reset --hard` fails during rollback | +| `SandboxStateError` | Method called in an invalid lifecycle state | + +All exceptions are importable from +`cleveragents.infrastructure.sandbox.protocol`. + +--- + +## Branch Naming + +Branch names are sanitised from the plan ID: + +- Only alphanumeric characters, hyphens, underscores, slashes, and dots + are kept. +- Runs of disallowed characters are collapsed into a single hyphen. +- The final branch name has the form `cleveragents/plan-`. + +--- + +## Fallback for Non-Git Projects + +`PlanLifecycleService` detects whether the resource is a git-checkout type. +If not, it falls back to the original flat file copy strategy +(`shutil.copy2`). The git worktree sandbox is only used for resources +where `resource_type_name` is `git-checkout` or `git`. + +--- + +## Related + +- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) +- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md) +- [ADR-038 Cross-Mechanism Sandbox Coordination](../adr/ADR-038-cross-mechanism-sandbox-coordination.md) +- [`cleveragents.infrastructure.sandbox`](../api/core.md) diff --git a/mkdocs.yml b/mkdocs.yml index 2bb05974c..76940184b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,6 +27,8 @@ nav: - Shell Safety: modules/shell-safety.md - UKO Provenance Tracking: modules/uko-provenance.md - Invariant Reconciliation: modules/invariant-reconciliation.md + - ACMS Context Hydration: modules/context-hydration.md + - Git Worktree Sandbox: modules/git-worktree-sandbox.md - Development: - Agent System Specification: development/agent-system-specification.md - CI/CD Pipeline: development/ci-cd.md -- 2.52.0