docs: add context hydration and git worktree sandbox module docs
CI / helm (pull_request) Successful in 22s
CI / push-validation (pull_request) Successful in 26s
CI / typecheck (pull_request) Successful in 52s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m20s
CI / e2e_tests (pull_request) Successful in 3m12s
CI / build (pull_request) Successful in 3m22s
CI / quality (pull_request) Successful in 3m42s
CI / security (pull_request) Successful in 4m15s
CI / integration_tests (pull_request) Successful in 8m53s
CI / unit_tests (pull_request) Successful in 8m58s
CI / docker (pull_request) Successful in 1m22s
CI / coverage (pull_request) Successful in 11m5s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (push) Waiting to run
CI / push-validation (push) Successful in 18s
CI / helm (push) Successful in 24s
CI / lint (push) Successful in 27s
CI / build (push) Successful in 33s
CI / security (push) Successful in 1m7s
CI / quality (push) Successful in 3m39s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Waiting to run
CI / integration_tests (push) Successful in 4m8s
CI / e2e_tests (push) Successful in 6m15s
CI / unit_tests (push) Successful in 8m10s
CI / docker (push) Successful in 10s
CI / coverage (push) Successful in 10m10s
CI / status-check (push) Successful in 1s
CI / benchmark-regression (pull_request) Successful in 56m36s
CI / helm (pull_request) Successful in 22s
CI / push-validation (pull_request) Successful in 26s
CI / typecheck (pull_request) Successful in 52s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 3m20s
CI / e2e_tests (pull_request) Successful in 3m12s
CI / build (pull_request) Successful in 3m22s
CI / quality (pull_request) Successful in 3m42s
CI / security (pull_request) Successful in 4m15s
CI / integration_tests (pull_request) Successful in 8m53s
CI / unit_tests (pull_request) Successful in 8m58s
CI / docker (pull_request) Successful in 1m22s
CI / coverage (pull_request) Successful in 11m5s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (push) Waiting to run
CI / push-validation (push) Successful in 18s
CI / helm (push) Successful in 24s
CI / lint (push) Successful in 27s
CI / build (push) Successful in 33s
CI / security (push) Successful in 1m7s
CI / quality (push) Successful in 3m39s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Waiting to run
CI / integration_tests (push) Successful in 4m8s
CI / e2e_tests (push) Successful in 6m15s
CI / unit_tests (push) Successful in 8m10s
CI / docker (push) Successful in 10s
CI / coverage (push) Successful in 10m10s
CI / status-check (push) Successful in 1s
CI / benchmark-regression (pull_request) Successful in 56m36s
- Add docs/modules/context-hydration.md: documents the ACMS context hydration pipeline (context_tier_hydrator) that fixes the empty ContextTierService bug (#1028). Covers hydrate_tiers_from_project, hydrate_tiers_for_plan, file listing strategies, limits, and fragment metadata. - Add docs/modules/git-worktree-sandbox.md: documents the GitWorktreeSandbox class that isolates plan apply changes in a dedicated git branch/worktree and merges back on commit (#4454). Covers full lifecycle, error types, branch naming, and fallback for non-git projects. - Update docs/architecture.md: add Git Worktree Sandbox Apply section and extend ACMS section with context hydration note. - Update mkdocs.yml: add both new module pages to the Modules nav. ISSUES CLOSED: #6841
This commit was merged in pull request #6189.
This commit is contained in:
@@ -0,0 +1,180 @@
|
||||
# ACMS Context Hydration
|
||||
|
||||
**Package:** `cleveragents.application.services.context_tier_hydrator`
|
||||
**Introduced:** Unreleased (issue #1028)
|
||||
|
||||
The context hydration module bridges the gap between the resource registry
|
||||
(files on disk) and the ACMS context tier (in-memory fragments). Without
|
||||
it, the `ContextTierService` starts empty on every CLI process invocation
|
||||
and the LLM receives zero file context during plan execution.
|
||||
|
||||
---
|
||||
|
||||
## Problem Solved
|
||||
|
||||
Before this module, the ACMS indexing pipeline was not wired into the CLI.
|
||||
On each invocation, `ContextTierService` started with an empty fragment
|
||||
store, so `LLMExecuteActor.execute()` assembled context with no file
|
||||
content. The LLM therefore had no knowledge of the project's source files
|
||||
when generating plans.
|
||||
|
||||
This module fixes that by reading files from all resources linked to the
|
||||
active project and storing them as `TieredFragment` objects in the tier
|
||||
service before context assembly begins.
|
||||
|
||||
---
|
||||
|
||||
## Key Functions
|
||||
|
||||
### `hydrate_tiers_from_project`
|
||||
|
||||
```python
|
||||
from cleveragents.application.services.context_tier_hydrator import (
|
||||
hydrate_tiers_from_project,
|
||||
)
|
||||
|
||||
count = hydrate_tiers_from_project(
|
||||
tier_service=tier_service,
|
||||
project_name="local/my-project",
|
||||
resource_id="01HZ...",
|
||||
resource_location="/path/to/project",
|
||||
resource_type="git-checkout", # default
|
||||
)
|
||||
# count — number of TieredFragment objects stored
|
||||
```
|
||||
|
||||
Reads files from a single resource directory and stores them as
|
||||
`TieredFragment` objects in `ContextTierService`.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `tier_service` | `ContextTierService` | The service to populate |
|
||||
| `project_name` | `str` | Namespaced project name (e.g. `local/my-project`) |
|
||||
| `resource_id` | `str` | ULID of the resource |
|
||||
| `resource_location` | `str` | Filesystem path to the resource root |
|
||||
| `resource_type` | `str` | Resource type name; affects file listing strategy |
|
||||
|
||||
**Returns:** Number of fragments stored.
|
||||
|
||||
---
|
||||
|
||||
### `hydrate_tiers_for_plan`
|
||||
|
||||
```python
|
||||
from cleveragents.application.services.context_tier_hydrator import (
|
||||
hydrate_tiers_for_plan,
|
||||
)
|
||||
|
||||
total = hydrate_tiers_for_plan(
|
||||
tier_service=tier_service,
|
||||
project_names=["local/my-project"],
|
||||
project_repository=project_repo,
|
||||
resource_registry=resource_registry,
|
||||
)
|
||||
```
|
||||
|
||||
Hydrates tiers for all projects linked to a plan. Iterates over each
|
||||
project name, resolves linked resources via the project repository and
|
||||
resource registry, and calls `hydrate_tiers_from_project` for each.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `tier_service` | `ContextTierService` | The service to populate |
|
||||
| `project_names` | `list[str]` | Namespaced project names |
|
||||
| `project_repository` | `NamespacedProjectRepository` | Repository for project lookup |
|
||||
| `resource_registry` | `ResourceRegistryService` | Registry for resource lookup |
|
||||
|
||||
**Returns:** Total number of fragments stored across all projects.
|
||||
|
||||
---
|
||||
|
||||
## File Listing Strategy
|
||||
|
||||
The hydrator uses two strategies to enumerate files:
|
||||
|
||||
| Strategy | When Used | How |
|
||||
|----------|-----------|-----|
|
||||
| `git ls-files` | `resource_type` is `git-checkout` or `git` | Lists tracked and untracked (non-ignored) files via `git ls-files --cached --others --exclude-standard` |
|
||||
| `os.walk` | All other resource types, or if `git ls-files` fails | Walks the directory tree, skipping hidden directories and binary extensions |
|
||||
|
||||
---
|
||||
|
||||
## Limits and Exclusions
|
||||
|
||||
To avoid overwhelming the context window or indexing irrelevant content,
|
||||
the hydrator enforces these limits:
|
||||
|
||||
| Limit | Value | Description |
|
||||
|-------|-------|-------------|
|
||||
| Max file size | 256 KB | Files larger than this are skipped |
|
||||
| Max total bytes | 10 MB | Indexing stops once this budget is reached |
|
||||
|
||||
**Skipped directories** (always excluded):
|
||||
|
||||
`.git`, `.hg`, `.svn`, `__pycache__`, `node_modules`, `.venv`, `venv`,
|
||||
`.nox`, `.tox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`,
|
||||
`build`, `.eggs`, `.cleveragents`
|
||||
|
||||
**Skipped file extensions** (binary files):
|
||||
|
||||
`.pyc`, `.pyo`, `.so`, `.o`, `.a`, `.dll`, `.exe`, `.png`, `.jpg`,
|
||||
`.jpeg`, `.gif`, `.bmp`, `.ico`, `.pdf`, `.zip`, `.tar`, `.gz`, `.bz2`,
|
||||
`.xz`, `.whl`, `.egg`, `.db`, `.sqlite`, `.sqlite3`
|
||||
|
||||
---
|
||||
|
||||
## Fragment Metadata
|
||||
|
||||
Each stored `TieredFragment` carries:
|
||||
|
||||
```python
|
||||
TieredFragment(
|
||||
fragment_id=f"{resource_id}:{rel_path}",
|
||||
content=<file content>,
|
||||
tier=ContextTier.HOT,
|
||||
resource_id=resource_id,
|
||||
project_name=project_name,
|
||||
token_count=len(content) // 4, # approximate
|
||||
metadata={
|
||||
"path": rel_path,
|
||||
"detail_depth": "1",
|
||||
"relevance_score": "0.5",
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
All fragments are stored in the `HOT` tier (always-present context).
|
||||
`detail_depth` and `relevance_score` are stored as strings to satisfy
|
||||
`ContextFragment` Pydantic validation requirements.
|
||||
|
||||
---
|
||||
|
||||
## Integration Point
|
||||
|
||||
Hydration runs automatically before context assembly in
|
||||
`LLMExecuteActor.execute()`. No manual invocation is required for normal
|
||||
plan execution.
|
||||
|
||||
```python
|
||||
# Conceptual flow inside LLMExecuteActor.execute()
|
||||
hydrate_tiers_for_plan(
|
||||
tier_service=self._tier_service,
|
||||
project_names=plan.project_names,
|
||||
project_repository=self._project_repository,
|
||||
resource_registry=self._resource_registry,
|
||||
)
|
||||
context = self._context_assembler.assemble(plan_id=plan.id)
|
||||
response = await self._llm.invoke(context)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [ACMS Architecture](../architecture.md#context-management-acms)
|
||||
- [ADR-014 Context Management](../adr/ADR-014-context-management-acms.md)
|
||||
- [`cleveragents.acms`](../api/core.md)
|
||||
@@ -0,0 +1,190 @@
|
||||
# Git Worktree Sandbox
|
||||
|
||||
**Package:** `cleveragents.infrastructure.sandbox.git_worktree`
|
||||
**Introduced:** Unreleased (issue #4454)
|
||||
|
||||
The git worktree sandbox isolates plan modifications in a dedicated git
|
||||
branch and worktree. On commit, changes are merged back to the original
|
||||
branch via `git merge`. On rollback, the worktree is discarded entirely.
|
||||
This replaces the previous flat `shutil.copy2` apply strategy for
|
||||
git-checkout resources.
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
When a plan applies changes to a git-checkout resource, the changes must
|
||||
be isolated from the working tree until the user approves them. The git
|
||||
worktree sandbox achieves this by:
|
||||
|
||||
1. Creating a new branch `cleveragents/plan-<plan_id>` from the current HEAD.
|
||||
2. Creating a temporary git worktree at a system temp directory.
|
||||
3. Letting the actor write changes inside the worktree.
|
||||
4. On commit: staging all changes, committing them in the worktree, then
|
||||
merging the sandbox branch back into the original branch.
|
||||
5. On rollback: resetting the worktree (and optionally the original branch)
|
||||
to the base commit and discarding the worktree.
|
||||
|
||||
Non-git projects fall back to the original flat file copy strategy.
|
||||
|
||||
---
|
||||
|
||||
## `GitWorktreeSandbox`
|
||||
|
||||
```python
|
||||
from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox
|
||||
|
||||
sandbox = GitWorktreeSandbox(
|
||||
resource_id="01HZ...",
|
||||
original_path="/path/to/repo",
|
||||
)
|
||||
```
|
||||
|
||||
Implements the `Sandbox` protocol.
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `resource_id` | `str` | — | Identifier of the resource being sandboxed |
|
||||
| `original_path` | `str` | — | Path to the git repository root |
|
||||
| `git_timeout` | `int` | `30` | Timeout in seconds for git commands |
|
||||
|
||||
---
|
||||
|
||||
## Lifecycle
|
||||
|
||||
```
|
||||
PENDING ──► create() ──► CREATED ──► get_path() ──► ACTIVE
|
||||
│
|
||||
commit() ─┤─► COMMITTED ──► cleanup() ──► CLEANED_UP
|
||||
│
|
||||
rollback() ─┴─► ROLLED_BACK ──► cleanup() ──► CLEANED_UP
|
||||
```
|
||||
|
||||
### `create(plan_id) → SandboxContext`
|
||||
|
||||
Creates the worktree and branch.
|
||||
|
||||
```python
|
||||
ctx = sandbox.create(plan_id="plan-01HZ...")
|
||||
# ctx.sandbox_path — path to the worktree
|
||||
# ctx.metadata["branch"] — e.g. "cleveragents/plan-plan-01HZ..."
|
||||
# ctx.metadata["base_commit"] — HEAD SHA at creation time
|
||||
```
|
||||
|
||||
Raises `SandboxCreationError` if the path is not a git repository root or
|
||||
if `git worktree add` fails.
|
||||
|
||||
### `get_path(resource_path) → str`
|
||||
|
||||
Translates a resource-relative path to an absolute path inside the
|
||||
worktree. Rejects `..` path traversal attempts.
|
||||
|
||||
```python
|
||||
worktree_file = sandbox.get_path("src/main.py")
|
||||
# → "/tmp/ca-sandbox-plan-01HZ.../src/main.py"
|
||||
```
|
||||
|
||||
### `commit(message=None) → CommitResult`
|
||||
|
||||
Stages all changes in the worktree, commits them, then merges the sandbox
|
||||
branch into the original branch.
|
||||
|
||||
```python
|
||||
result = sandbox.commit("feat: implement new feature")
|
||||
# result.success — True on success
|
||||
# result.commit_ref — SHA of the sandbox commit
|
||||
# result.changed_files — list of modified file paths
|
||||
# result.added_files — list of newly added file paths
|
||||
# result.deleted_files — list of deleted file paths
|
||||
```
|
||||
|
||||
If there are no staged changes, returns a `CommitResult` with empty file
|
||||
lists and no new commit.
|
||||
|
||||
Raises `SandboxCommitError` if the commit or merge fails.
|
||||
|
||||
### `rollback()`
|
||||
|
||||
Discards all worktree changes.
|
||||
|
||||
- From `ACTIVE`: resets the worktree branch to the base commit.
|
||||
- From `COMMITTED`: also resets the original branch to the pre-merge commit
|
||||
(fully undoes the merge).
|
||||
|
||||
!!! warning "Multi-worktree safety"
|
||||
Rolling back from `COMMITTED` executes `git reset --hard` on the
|
||||
original branch. If other git worktrees or external processes are
|
||||
tracking the same branch, they will be affected.
|
||||
|
||||
### `cleanup()`
|
||||
|
||||
Removes the worktree directory and deletes the sandbox branch. Idempotent
|
||||
— safe to call multiple times.
|
||||
|
||||
---
|
||||
|
||||
## Apply Summary
|
||||
|
||||
When `plan apply` uses the git worktree strategy, the CLI displays a
|
||||
spec-aligned Apply Summary:
|
||||
|
||||
```
|
||||
╭─ Apply Summary ──────────────────────────────────────╮
|
||||
│ Plan ID: plan-01HZ... │
|
||||
│ Artifacts: 3 files changed │
|
||||
│ Insertions: +142 │
|
||||
│ Deletions: -17 │
|
||||
│ Project: local/my-project │
|
||||
│ Timestamp: 2026-04-09 17:30:00 │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
╭─ Sandbox Cleanup ────────────────────────────────────╮
|
||||
│ ✓ Worktree removed │
|
||||
│ ✓ Branch deleted │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
✓ OK Changes applied
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Types
|
||||
|
||||
| Exception | When Raised |
|
||||
|-----------|-------------|
|
||||
| `SandboxCreationError` | `git worktree add` fails or path is not a git root |
|
||||
| `SandboxCommitError` | `git commit` or `git merge` fails |
|
||||
| `SandboxRollbackError` | `git reset --hard` fails during rollback |
|
||||
| `SandboxStateError` | Method called in an invalid lifecycle state |
|
||||
|
||||
All exceptions are importable from
|
||||
`cleveragents.infrastructure.sandbox.protocol`.
|
||||
|
||||
---
|
||||
|
||||
## Branch Naming
|
||||
|
||||
Branch names are sanitised from the plan ID:
|
||||
|
||||
- Only alphanumeric characters, hyphens, underscores, slashes, and dots
|
||||
are kept.
|
||||
- Runs of disallowed characters are collapsed into a single hyphen.
|
||||
- The final branch name has the form `cleveragents/plan-<sanitised_plan_id>`.
|
||||
|
||||
---
|
||||
|
||||
## Fallback for Non-Git Projects
|
||||
|
||||
`PlanLifecycleService` detects whether the resource is a git-checkout type.
|
||||
If not, it falls back to the original flat file copy strategy
|
||||
(`shutil.copy2`). The git worktree sandbox is only used for resources
|
||||
where `resource_type_name` is `git-checkout` or `git`.
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle)
|
||||
- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md)
|
||||
- [ADR-038 Cross-Mechanism Sandbox Coordination](../adr/ADR-038-cross-mechanism-sandbox-coordination.md)
|
||||
- [`cleveragents.infrastructure.sandbox`](../api/core.md)
|
||||
Reference in New Issue
Block a user