Compare commits

...

1 Commits

Author SHA1 Message Date
HAL9000 c2b0e1f67b docs: document git worktree sandbox, ACMS hydrator, and update changelog
CI / quality (pull_request) Successful in 31s
CI / build (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 24s
CI / push-validation (pull_request) Successful in 19s
CI / typecheck (pull_request) Successful in 56s
CI / lint (pull_request) Successful in 3m20s
CI / security (pull_request) Successful in 4m2s
CI / integration_tests (pull_request) Successful in 4m5s
CI / unit_tests (pull_request) Successful in 5m6s
CI / docker (pull_request) Successful in 1m23s
CI / e2e_tests (pull_request) Successful in 6m50s
CI / coverage (pull_request) Successful in 10m32s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m3s
- Add docs/modules/git-worktree-sandbox.md: full module guide for GitWorktreeSandbox (lifecycle, state machine, API, errors, apply summary, fallback)
- Add docs/api/acms.md: context_tier_hydrator API reference (hydrate_tiers_for_plan/from_project, limits, metadata, structured logs)
- Update docs/architecture.md: add Git Worktree Sandbox section and expand ACMS hydrator coverage
- Update mkdocs.yml: surface new docs in Modules and API navigation groups
- Update docs/api/index.md: add ACMS Services entry to API index
- Update CHANGELOG.md: restructure Unreleased entries with Added/Changed/Fixed summary plus existing bug IDs
- Correct Introduced versions and remove duplicate changelog entry for ACMS fix, clarify hydrator parameter types

ISSUES CLOSED: #6837
2026-04-12 17:19:40 +00:00
6 changed files with 518 additions and 5 deletions
+22 -5
View File
@@ -14,11 +14,6 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
Cleanup panel, and `✓ OK Changes applied` footer. Non-git projects fall
back to the original flat file copy.
- **Context Hydration Fix** (#4454): Fixed `ContextFragment` metadata types
(`detail_depth` and `relevance_score` must be strings, not int/float) that
caused Pydantic validation errors during context assembly, resulting in the
LLM receiving zero file context.
- **Automation Tracking System**: Replaced shared session state issue tracking with
individual per-agent tracking issues. Each agent now creates its own `[AUTO-<PREFIX>]`
titled issues with standardized headers, reporting intervals, and health indicators.
@@ -125,6 +120,12 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
`AUTO-INF-POOL`, `AUTO-ARCH`, `AUTO-EPIC`, `AUTO-EVLV`, `AUTO-GUARD`, `AUTO-SPEC`,
`AUTO-TIME`, `AUTO-PROJ-OWN`, and `AUTO-PROD-BLDR`.
- **Implementation Orchestrator**: Updated to use `async-agent-starter` for worker
dispatch, enabling proper async agent lifecycle management.
- **Label Management Hardening**: Label creation is now restricted to the
`forgejo-label-manager` subagent; all other agents must delegate label operations
to prevent unauthorized label proliferation.
### Fixed
- **Validation Gate Empty-Run Guard** (#7508): Fixed `ApplyValidationSummary.all_required_passed`
@@ -169,6 +170,22 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
`pr-merge-pool-supervisor` to the product-builder's supervisor launch list (18 total
supervisors). Updated all numeric references, pre-flight checklists, and validation logic.
- **ACMS Indexing Pipeline** (#1028): `ContextTierService` was never populated on
CLI invocations, causing the LLM to receive zero file context during plan execution.
Added `context_tier_hydrator.py` that reads files from linked project resources
(via `git ls-files` or `os.walk`) and stores them as `TieredFragment` objects in
the tier service. Hydration runs automatically before context assembly in
`LLMExecuteActor.execute()`. Respects max file size (256 KB), total budget (10 MB),
binary file exclusion, and skip-directory list (`.git`, `node_modules`,
`__pycache__`, etc.).
- **Plan Action Arguments** (#4197): Upsert action arguments during `plan use` to
avoid `UNIQUE constraint violation` when the same action is used across multiple
plan invocations.
- **CI Quality Tests**: Resolved lint errors and removed stale `@tdd_expected_fail`
tag that was masking a passing test (#5264).
---
## [3.8.0] — 2026-04-05
+181
View File
@@ -0,0 +1,181 @@
# `cleveragents.application.services` — ACMS Services
This page documents the Advanced Context Management System (ACMS) application
services, including the context tier hydrator that bridges the resource
registry with the in-memory context tier service.
For the ACMS architecture overview, see
[Architecture — Context Management](../architecture.md#context-management-acms).
For the ADR, see
[ADR-014 Context Management (ACMS)](../adr/ADR-014-context-management-acms.md).
---
## `context_tier_hydrator` — ACMS Indexing Pipeline
**Module:** `cleveragents.application.services.context_tier_hydrator`
**Introduced:** v3.4.0 — fixes bug #1028
### Problem Solved
Without this module, `ContextTierService` starts empty on every CLI process
invocation. The LLM receives zero file context during plan execution because
the tier service is never populated from the project's linked resources.
`context_tier_hydrator` bridges the gap: it reads files from the resource
registry and stores them as `TieredFragment` objects in the tier service,
so the LLM always has project context available.
---
### Functions
#### `hydrate_tiers_for_plan`
```python
def hydrate_tiers_for_plan(
tier_service: ContextTierService,
project_names: list[str],
project_repository: Any,
resource_registry: Any,
) -> int:
```
High-level entry point. Hydrates tiers for all projects linked to a plan.
Called automatically by `LLMExecuteActor.execute()` before context assembly.
**Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `tier_service` | `ContextTierService` | The service to populate with fragments |
| `project_names` | `list[str]` | Namespaced project names (e.g. `["local/my-project"]`) |
| `project_repository` | `NamespacedProjectRepository` | Provides project → linked resource lookup |
| `resource_registry` | `ResourceRegistryService` | Provides resource location and type |
**Returns:** Total number of `TieredFragment` objects stored across all projects.
!!! note "Implementation detail"
The production implementation annotates `project_repository` and
`resource_registry` as `Any` to accommodate dependency-injection wiring.
At runtime these parameters are instances of `NamespacedProjectRepository`
and `ResourceRegistryService` respectively. Future refactors will tighten
the function signature once the container exposes stable protocols.
**Example:**
```python
from cleveragents.application.services.context_tier_hydrator import (
hydrate_tiers_for_plan,
)
count = hydrate_tiers_for_plan(
tier_service=container.context_tier_service(),
project_names=["local/my-project"],
project_repository=container.project_repository(),
resource_registry=container.resource_registry(),
)
print(f"Hydrated {count} fragments")
```
---
#### `hydrate_tiers_from_project`
```python
def hydrate_tiers_from_project(
tier_service: ContextTierService,
project_name: str,
resource_id: str,
resource_location: str,
resource_type: str = "git-checkout",
) -> int:
```
Lower-level function. Reads files from a single resource directory and stores
them as `TieredFragment` objects in `ContextTier.HOT`.
**Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `tier_service` | `ContextTierService` | — | Service to populate |
| `project_name` | `str` | — | Namespaced project name |
| `resource_id` | `str` | — | ULID of the resource |
| `resource_location` | `str` | — | Filesystem path to the resource root |
| `resource_type` | `str` | `"git-checkout"` | Affects file listing strategy |
**Returns:** Number of fragments stored.
**File listing strategy:**
- For `git-checkout` / `git` resources: uses `git ls-files --cached --others --exclude-standard`
- For all other resource types: falls back to `os.walk`
---
### Limits and Filters
| Limit | Value | Description |
|-------|-------|-------------|
| Max file size | 256 KB | Files larger than this are skipped |
| Max total bytes | 10 MB | Hydration stops when this budget is reached |
| Binary extensions | `.pyc`, `.so`, `.png`, `.pdf`, `.zip`, etc. | Skipped entirely |
| Skip directories | `.git`, `__pycache__`, `node_modules`, `.venv`, `dist`, `build`, etc. | Not traversed |
---
### Fragment Metadata
Each `TieredFragment` stored by the hydrator has the following metadata:
```python
metadata = {
"path": "src/cleveragents/foo.py", # relative path within resource
"detail_depth": "1", # string (Pydantic requirement)
"relevance_score": "0.5", # string (Pydantic requirement)
}
```
!!! note "String metadata types"
`detail_depth` and `relevance_score` must be strings, not `int`/`float`.
Using numeric types causes a Pydantic validation error during context
assembly (bug #1028).
---
### Automatic Invocation
`hydrate_tiers_for_plan` is called automatically by `LLMExecuteActor.execute()`
before context assembly. No manual invocation is required in normal usage.
```
LLMExecuteActor.execute()
├─ hydrate_tiers_for_plan(...) ← populates ContextTierService
└─ ContextAssembler.assemble() ← now has file context available
```
---
### Structured Log Events
The hydrator emits structured log events at `DEBUG` and `INFO` levels:
| Event | Level | Description |
|-------|-------|-------------|
| `context_hydrator.skip_missing_location` | `WARNING` | Resource location does not exist |
| `context_hydrator.project_not_found` | `DEBUG` | Project not found in repository |
| `context_hydrator.no_linked_resources` | `DEBUG` | Project has no linked resources |
| `context_hydrator.resource_not_found` | `DEBUG` | Resource not found in registry |
| `context_hydrator.store_failed` | `DEBUG` | Failed to store a fragment |
| `context_hydrator.hydrated` | `INFO` | Hydration complete — includes `fragments_stored` and `total_bytes` |
---
### Related Documentation
- [Architecture — Context Management (ACMS)](../architecture.md#context-management-acms)
- [ADR-014 Context Management (ACMS)](../adr/ADR-014-context-management-acms.md)
- [Modules — UKO Provenance Tracking](../modules/uko-provenance.md)
+1
View File
@@ -18,6 +18,7 @@ classes, functions, and exceptions with signatures and usage examples.
| [`cleveragents.config`](config.md) | Application settings, logging, metrics, and security scanning |
| [`cleveragents.providers`](providers.md) | AI provider registry — discovery, selection, LLM factory, and capability metadata |
| [`cleveragents.tui`](tui.md) | Interactive Terminal UI — app, persona system, input routing, slash commands, session export/import |
| [ACMS Services](acms.md) | Advanced Context Management System services — context tier hydrator, fragment indexing pipeline |
> **Note:** Internal modules (prefixed with `_`) and implementation details
> not listed in a module's `__all__` are considered private and may change
+43
View File
@@ -206,6 +206,13 @@ LLM at each phase ([ADR-014](adr/ADR-014-context-management-acms.md)):
The **UKO Runtime** (Universal Knowledge Ontology) provides graph-based
knowledge inference and persistence for Tier 3 context strategies.
The **Context Tier Hydrator** (`context_tier_hydrator.py`) bridges the
resource registry and the in-memory tier service: it reads files from linked
project resources (via `git ls-files` or `os.walk`) and stores them as
`TieredFragment` objects before context assembly. Without it, the tier service
starts empty on every CLI invocation and the LLM receives no file context.
See [`docs/api/acms.md`](api/acms.md) for the full API reference.
Key UKO capabilities:
- **Typed triples with provenance** — every triple carries `sourceResource`,
@@ -259,6 +266,42 @@ result = actor.run(plan_id="...", project_name="...", action_name="...")
---
## Git Worktree Sandbox
The **Git Worktree Sandbox** (`cleveragents.infrastructure.sandbox.git_worktree`)
provides git-native isolation for the Execute and Apply phases of the plan
lifecycle when the project resource is a `git-checkout`.
Instead of copying files to a temporary directory, it creates a real git
worktree on a dedicated branch (`cleveragents/plan-<plan_id>`). Changes are
committed in the worktree and merged back to the original branch on Apply, or
discarded entirely on rollback.
```
Execute phase:
GitWorktreeSandbox.create(plan_id)
→ creates branch cleveragents/plan-<id>
→ creates worktree at /tmp/ca-sandbox-<id>-<random>/
LLM writes files into worktree
GitWorktreeSandbox.commit("...")
→ git add -A && git commit (in worktree)
→ git merge cleveragents/plan-<id> (in original repo)
Apply phase:
Displays Apply Summary (plan ID, artifacts, insertions/deletions)
Displays Sandbox Cleanup panel
GitWorktreeSandbox.cleanup()
→ git worktree remove --force
→ git branch -D cleveragents/plan-<id>
```
Non-git resources fall back to the original flat-copy strategy.
For full API documentation, see
[Modules — Git Worktree Sandbox](modules/git-worktree-sandbox.md).
---
## A2A Protocol Boundary
All CLI, TUI, and server interactions cross the A2A boundary
+269
View File
@@ -0,0 +1,269 @@
# Git Worktree Sandbox Module
**Package:** `cleveragents.infrastructure.sandbox.git_worktree`
**Introduced:** v3.5.0
The Git Worktree Sandbox provides isolated, git-native sandboxing for
`git-checkout` resources. Instead of copying files to a temporary directory,
it creates a real git worktree on a dedicated branch. Changes are committed
in the worktree and merged back to the original branch on `commit`, or
discarded entirely on `rollback`.
For the sandbox protocol interface, see
[`docs/development/custom_sandbox_strategy.md`](../development/custom_sandbox_strategy.md).
For the sandbox ADR, see
[`ADR-015 Sandbox & Checkpoint`](../adr/ADR-015-sandbox-and-checkpoint.md).
---
## Purpose
The previous flat-copy sandbox (`shutil.copy2`) had no awareness of git
history, making it impossible to produce meaningful diffs or leverage git's
merge machinery. The git worktree sandbox:
- Preserves full git history for every plan's changes
- Enables `git merge` for conflict detection and resolution
- Produces accurate insertions/deletions counts for the Apply Summary panel
- Falls back gracefully to flat-copy for non-git resources
---
## Lifecycle
```
GitWorktreeSandbox(resource_id, original_path)
sandbox.create(plan_id)
│ Creates branch cleveragents/plan-<plan_id>
│ Creates worktree at /tmp/ca-sandbox-<plan_id>-<random>/
sandbox.get_path("src/foo.py")
│ Returns /tmp/ca-sandbox-.../src/foo.py
│ Actor writes changes here
sandbox.commit("feat: ...")
│ git add -A (in worktree)
│ git commit (in worktree)
│ git merge cleveragents/plan-<id> (in original repo)
sandbox.cleanup()
│ git worktree remove --force <path>
│ git branch -D cleveragents/plan-<id>
│ git worktree prune
CLEANED_UP
```
**Rollback path:**
```
sandbox.rollback()
│ If COMMITTED: git reset --hard <pre_merge_commit> (original repo)
│ git reset --hard <base_commit> (worktree)
│ git clean -fd (worktree)
ROLLED_BACK
```
---
## Status State Machine
| Status | Description |
|--------|-------------|
| `PENDING` | Sandbox created but `create()` not yet called |
| `CREATED` | Worktree and branch exist; no writes yet |
| `ACTIVE` | `get_path()` called; actor is writing |
| `COMMITTED` | Changes committed and merged to original branch |
| `ROLLED_BACK` | Changes discarded; worktree reset to base commit |
| `ERRORED` | A git command failed; sandbox is unusable |
| `CLEANED_UP` | Worktree and branch removed |
---
## Key Class: `GitWorktreeSandbox`
```python
from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox
sandbox = GitWorktreeSandbox(
resource_id="res-01JXYZ",
original_path="/path/to/project",
git_timeout=30, # seconds; default
)
```
### Constructor Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `resource_id` | `str` | — | Identifier of the resource being sandboxed |
| `original_path` | `str` | — | Absolute path to the git repository root |
| `git_timeout` | `int` | `30` | Timeout in seconds for each git command |
### Methods
#### `create(plan_id: str) → SandboxContext`
Creates the worktree and branch. Must be called before any other method.
- Branch name: `cleveragents/plan-<sanitised_plan_id>`
- Worktree path: `/tmp/ca-sandbox-<plan_id>-<random>/`
- `SandboxContext.metadata` includes `strategy`, `branch`, `original_branch`,
`base_commit`, and `worktree_path`
**Raises:** `SandboxCreationError` if git commands fail; `SandboxStateError`
if not in `PENDING` status.
#### `get_path(resource_path: str) → str`
Translates a resource-relative path to its worktree equivalent.
```python
worktree_file = sandbox.get_path("src/cleveragents/foo.py")
# → /tmp/ca-sandbox-.../src/cleveragents/foo.py
```
**Raises:** `ValueError` on path traversal (`..` components); `SandboxStateError`
if sandbox is not usable.
#### `commit(message: str | None = None) → CommitResult`
Stages all changes, commits in the worktree, then merges back to the original
branch. Returns a `CommitResult` with `changed_files`, `added_files`,
`deleted_files`, and `commit_ref`.
If there are no staged changes, returns a no-op `CommitResult` with the base
commit ref and empty file lists.
**Raises:** `SandboxCommitError` on git failure; `SandboxStateError` if not
in `CREATED` or `ACTIVE` status.
#### `rollback() → None`
Discards all changes:
- From `ACTIVE`: resets the worktree branch to `base_commit`
- From `COMMITTED`: also resets the original branch to `pre_merge_commit`
(undoes the merge)
!!! warning "Multi-worktree safety"
Rolling back from `COMMITTED` executes `git reset --hard` on the original
branch. Other git worktrees or external processes tracking the same branch
will be affected.
**Raises:** `SandboxRollbackError` on git failure.
#### `cleanup() → None`
Removes the worktree directory and deletes the sandbox branch. Idempotent —
safe to call multiple times. Falls back to `shutil.rmtree` if
`git worktree remove` fails.
---
## CommitResult
```python
@dataclass
class CommitResult:
sandbox_id: str
success: bool
commit_ref: str | None # SHA of the commit in the worktree
changed_files: list[str] # Modified files (M status)
added_files: list[str] # New files (A status)
deleted_files: list[str] # Deleted files (D status)
error: str | None
timestamp: datetime
```
---
## Error Types
| Exception | When raised |
|-----------|-------------|
| `SandboxCreationError` | `git worktree add` or `git rev-parse` fails |
| `SandboxCommitError` | `git commit` or `git merge` fails |
| `SandboxRollbackError` | `git reset --hard` or `git clean` fails |
| `SandboxStateError` | Method called in wrong lifecycle state |
All exceptions are subclasses of `SandboxError` from
`cleveragents.infrastructure.sandbox.protocol`.
---
## Non-Git Fallback
For resources that are not git repositories, `plan apply` falls back to the
original flat-copy strategy (`shutil.copy2`). The `GitWorktreeSandbox` itself
raises `SandboxCreationError` if the `original_path` is not the root of a git
repository (verified via `git rev-parse --show-toplevel`).
---
## Usage Example
```python
from cleveragents.infrastructure.sandbox.git_worktree import GitWorktreeSandbox
sandbox = GitWorktreeSandbox(
resource_id="res-01JXYZ",
original_path="/home/user/my-project",
)
ctx = sandbox.create(plan_id="plan-01JABC")
print(f"Worktree at: {ctx.sandbox_path}")
print(f"Branch: {ctx.metadata['branch']}")
# Actor writes files into the worktree
import pathlib
target = pathlib.Path(sandbox.get_path("src/new_feature.py"))
target.write_text("# new feature\n")
# Commit and merge back
result = sandbox.commit("feat: add new_feature module")
print(f"Commit: {result.commit_ref}")
print(f"Added: {result.added_files}")
# Always clean up
sandbox.cleanup()
```
---
## Apply Summary Output
When `plan apply` uses the git worktree sandbox, the CLI renders a
spec-aligned Apply Summary panel:
```
╭─ Apply Summary ──────────────────────────────────────────────╮
│ Plan ID plan-01JABC │
│ Project local/my-project │
│ Timestamp 2026-04-09 14:33:34 UTC │
│ │
│ Artifacts │
│ src/new_feature.py │
│ │
│ +1 insertion 0 deletions │
╰──────────────────────────────────────────────────────────────╯
╭─ Sandbox Cleanup ────────────────────────────────────────────╮
│ Worktree removed /tmp/ca-sandbox-plan-01JABC-xxxxx/ │
│ Branch deleted cleveragents/plan-01JABC │
╰──────────────────────────────────────────────────────────────╯
✓ OK Changes applied
```
---
## Related Documentation
- [ADR-015 Sandbox & Checkpoint](../adr/ADR-015-sandbox-and-checkpoint.md) — design rationale
- [Custom Sandbox Strategy](../development/custom_sandbox_strategy.md) — implementing your own sandbox
- [Architecture — Plan Lifecycle](../architecture.md#plan-lifecycle) — where apply fits
- [Shell Safety Module](shell-safety.md) — another infrastructure safety module
+2
View File
@@ -23,10 +23,12 @@ nav:
- Configuration: api/config.md
- AI Providers: api/providers.md
- TUI: api/tui.md
- ACMS Services: api/acms.md
- Modules:
- Shell Safety: modules/shell-safety.md
- UKO Provenance Tracking: modules/uko-provenance.md
- Invariant Reconciliation: modules/invariant-reconciliation.md
- Git Worktree Sandbox: modules/git-worktree-sandbox.md
- Development:
- Agent System Specification: development/agent-system-specification.md
- CI/CD Pipeline: development/ci-cd.md