diff --git a/docs/modules/depth-reduction-compressor.md b/docs/modules/depth-reduction-compressor.md new file mode 100644 index 000000000..7469c241e --- /dev/null +++ b/docs/modules/depth-reduction-compressor.md @@ -0,0 +1,183 @@ +# Depth Reduction Compressor + +**Package:** `cleveragents.application.services.depth_reduction_compressor` +**Introduced:** v3.8.0 (issue #919) + +The `DepthReductionCompressor` is the production ACMS skeleton compression +stage. It compresses inherited parent context fragments by re-rendering them +at lower UKO detail depths (overview levels 0–1), reducing token usage while +preserving structural information for child plans. + +--- + +## Purpose + +When a parent plan spawns a child plan, the parent's accumulated context +fragments are passed as inherited context. Without compression, large parent +contexts can exhaust the child plan's token budget before any new context is +added. + +The `DepthReductionCompressor` solves this by re-rendering inherited fragments +at shallower detail depths (overview levels 0–1 in the UKO detail-level map), +producing a compact structural summary rather than full source content. + +--- + +## How It Works + +The compressor uses the **UKO detail-level map chain** to determine the +appropriate overview depth for each fragment: + +1. For each input fragment, look up the UKO node's detail-level map. +2. Re-render the fragment at depth 0 (module listing) or depth 1 (type + signatures), whichever fits within the skeleton budget. +3. Return the re-rendered fragments as the compressed skeleton. + +This is distinct from the `SkeletonCompressorService` (which drops +lower-relevance fragments entirely) — `DepthReductionCompressor` keeps all +fragments but reduces their verbosity. + +--- + +## Key Classes + +### `DepthReductionCompressor` + +```python +from cleveragents.application.services.depth_reduction_compressor import ( + DepthReductionCompressor, +) + +compressor = DepthReductionCompressor( + uko_detail_map=container.uko_detail_map(), +) +``` + +**Constructor parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `uko_detail_map` | `UKODetailLevelMap` | required | UKO detail-level map for depth resolution | +| `target_depths` | `tuple[int, ...]` | `(1, 0)` | Depth levels to try, in order (highest first) | + +**Methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `compress(fragments, skeleton_budget)` | `tuple[ContextFragment, ...]` | Re-render fragments at overview depths to fit within `skeleton_budget` tokens | + +--- + +### `DepthReductionResult` + +Frozen dataclass returned by `compress()`: + +| Field | Type | Description | +|-------|------|-------------| +| `fragments` | `tuple[ContextFragment, ...]` | Re-rendered fragments | +| `original_tokens` | `int` | Total tokens before compression | +| `compressed_tokens` | `int` | Total tokens after compression | +| `depth_reductions` | `dict[str, int]` | Fragment ID → new depth for each reduced fragment | + +--- + +## Usage Example + +```python +from cleveragents.application.services.depth_reduction_compressor import ( + DepthReductionCompressor, +) +from cleveragents.domain.models.core.context_fragment import ( + ContextBudget, + ContextFragment, + FragmentProvenance, +) + +# Create compressor (typically obtained from DI container) +compressor = DepthReductionCompressor(uko_detail_map=uko_detail_map) + +# Compress parent context for child plan inheritance +result = compressor.compress( + fragments=parent_context.fragments, + skeleton_budget=512, # tokens available for inherited context +) + +print(f"Compressed {result.original_tokens} → {result.compressed_tokens} tokens") +print(f"Depth reductions: {result.depth_reductions}") + +# Use compressed fragments as child plan's inherited context +child_context = ContextPayload( + plan_id=child_plan_id, + fragments=result.fragments, + ... +) +``` + +--- + +## Pipeline Integration + +The `DepthReductionCompressor` is registered as the **builtin +`SkeletonCompressor`** in the ACMS pipeline (Phase 3 — Context +Finalization). It replaces the previous identity-compression stub. + +``` +ACMSPipeline + └── Phase 3: Context Finalization + ├── PreambleGenerator (no-op stub) + └── SkeletonCompressor ← DepthReductionCompressor (v3.8.0+) +``` + +The pipeline invokes `compress()` automatically when assembling context +for child plans. You do not need to call it directly in most cases. + +--- + +## DI Container Registration + +The compressor is registered as a singleton in the DI container: + +```python +from cleveragents.application.container import Container + +container = Container() +compressor = container.skeleton_compressor() +# Returns the configured DepthReductionCompressor instance +``` + +--- + +## Relationship to `SkeletonCompressorService` + +| | `SkeletonCompressorService` | `DepthReductionCompressor` | +|---|---|---| +| **Strategy** | Drop low-relevance fragments | Re-render all fragments at lower depth | +| **Output** | Subset of original fragments | All fragments, re-rendered | +| **Token reduction** | By fragment count | By content verbosity | +| **Use case** | Skeleton ratio budget enforcement | Child plan context inheritance | +| **Introduced** | v3.7.0 | v3.8.0 | + +Both components are part of the ACMS pipeline's Phase 3 skeleton compression +stage. `SkeletonCompressorService` applies the `skeleton_ratio` budget +policy; `DepthReductionCompressor` handles the actual re-rendering. + +--- + +## BDD Coverage + +The depth reduction compressor is covered by BDD scenarios in +`features/acms_skeleton_compression.feature`: + +- Compressor output at depth 0 and depth 1 +- Default pipeline wiring (compressor is the configured builtin) +- Token budget enforcement (compressed output fits within `skeleton_budget`) +- Depth reduction metadata (`depth_reductions` dict) + +--- + +## Related Documentation + +- [Skeleton Compressor](../reference/skeleton_compressor.md) — `SkeletonCompressorService` and skeleton ratio +- [ACMS Pipeline](../reference/acms.md) — 10-component pipeline architecture +- [ACMS Strategy Coordinator & Fusion Engine](../reference/acms_fusion.md) — Phase 1 and Phase 2 +- [UKO Runtime](../reference/uko_runtime.md) — UKO detail-level map diff --git a/docs/reference/devcontainer_resources.md b/docs/reference/devcontainer_resources.md index 12cf50b57..1f87225cb 100644 --- a/docs/reference/devcontainer_resources.md +++ b/docs/reference/devcontainer_resources.md @@ -73,17 +73,49 @@ root): 1. `.devcontainer/devcontainer.json` 2. `.devcontainer.json` (root-level) +3. `.devcontainer//devcontainer.json` (named configurations — **added in v3.8.0**) + +### Named Configurations (v3.8.0+) + +`discover_devcontainers()` now scans one subdirectory level inside +`.devcontainer/` for named configurations, in addition to the two +fixed root-level paths above. This enables correct auto-discovery of +all devcontainer instances in monorepo projects. + +For example, a project with the following structure: + +``` +.devcontainer/ + api/ + devcontainer.json ← named config "api" + frontend/ + devcontainer.json ← named config "frontend" + devcontainer.json ← root-level (unnamed) config +``` + +will produce **three** `DevcontainerDiscoveryResult` entries: + +| `config_name` | Path | +|---------------|------| +| `None` | `.devcontainer/devcontainer.json` | +| `"api"` | `.devcontainer/api/devcontainer.json` | +| `"frontend"` | `.devcontainer/frontend/devcontainer.json` | + +Root-level configs retain `config_name=None` for full backward +compatibility. Named configs set `config_name` to the subdirectory +name (e.g. `"api"`, `"frontend"`). ### Discovery Process (Planned) 1. **Trigger**: Linking a `git-checkout` or `fs-directory` resource. 2. **Scan**: The discovery module checks for configuration files at the - well-known paths listed above. + well-known paths listed above, including named subdirectory configs. 3. **Validate**: Each discovered file is parsed as JSON. Invalid files are skipped with a warning. 4. **Register**: For each valid configuration: - A `devcontainer-instance` child resource is created under the - parent resource. + parent resource. Named configs produce a distinct resource per + `config_name`. - A `devcontainer-file` child resource is created under the devcontainer instance, pointing to the JSON file. @@ -298,10 +330,13 @@ when the resource is in a non-running state. It handles both The `cleveragents.resource.handlers.discovery` module provides: - `discover_devcontainers(location, type)`: Scans a path for - devcontainer configs. + devcontainer configs, including named subdirectory configurations + (`.devcontainer//devcontainer.json`) added in v3.8.0. - `is_trigger_type(type)`: Checks if a resource type triggers auto-discovery. - `DevcontainerDiscoveryResult`: Data class holding discovery results. + The `config_name` field is `None` for root-level configs and set to + the subdirectory name (e.g. `"api"`) for named configurations. ## See Also diff --git a/docs/reference/skeleton_compressor.md b/docs/reference/skeleton_compressor.md index 55120d4ea..9f190e9c3 100644 --- a/docs/reference/skeleton_compressor.md +++ b/docs/reference/skeleton_compressor.md @@ -11,6 +11,14 @@ The compressor lives in `cleveragents.application.services.skeleton_compressor.SkeletonCompressorService` and is registered in the DI container as `skeleton_compressor_service`. +> **v3.8.0+:** The ACMS pipeline's Phase 3 `SkeletonCompressor` slot is now +> filled by `DepthReductionCompressor`, which re-renders fragments at lower +> UKO detail depths rather than dropping them. `SkeletonCompressorService` +> (this document) handles the `skeleton_ratio` budget policy and is a +> separate component. See +> [Depth Reduction Compressor](../modules/depth-reduction-compressor.md) +> for the pipeline-integrated compressor. + ## Skeleton Ratio | Ratio | Meaning | Behaviour | @@ -107,3 +115,9 @@ compressed fragments as the child's inherited context budget. The `--skeleton-ratio` flag on `agents project context set` sets the ratio for a project's context policy. The `plan status` command displays the compression summary when `skeleton_metadata` is present. + +## See Also + +- [Depth Reduction Compressor](../modules/depth-reduction-compressor.md) — + production ACMS pipeline compressor that re-renders fragments at lower + UKO detail depths (v3.8.0+) diff --git a/mkdocs.yml b/mkdocs.yml index 83244ee3c..c9a25599e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -22,6 +22,10 @@ nav: - Resources: api/resource.md - Configuration: api/config.md - TUI: api/tui.md + - Module Guides: + - Shell Safety: modules/shell-safety.md + - UKO Provenance Tracking: modules/uko-provenance.md + - Depth Reduction Compressor: modules/depth-reduction-compressor.md - Development: - CI/CD Pipeline: development/ci-cd.md - Quality Automation: development/quality-automation.md