forked from HAL9000/cleveragents-core
freemo
711e867112
Implemented SkeletonCompressorService for ACMS context inheritance, producing compressed context representations for propagation from parent plans to child plans. Key design decisions and implementation details: - SkeletonMetadata (frozen Pydantic model): records ratio, original_tokens, compressed_tokens, and source_decision_ids for full auditability of each compression pass. Persisted on Plan.skeleton_metadata. - SkeletonCompressorService: stateless service accepting a list of ContextFragment objects and a skeleton_ratio in [0.0, 1.0]. Fragments are sorted by relevance descending with a stable secondary sort on fragment_id to guarantee deterministic output. Token budget is original_tokens*(1-ratio); fragments are greedily selected until budget is exhausted. - Ratio semantics: 0.0 = no compression (pass-through), 1.0 = maximum compression (single top fragment only), None = default 0.3. - Integration: Plan model gains optional skeleton_metadata field exposed in as_cli_dict() under the 'skeleton' key. Service registered in DI container as skeleton_compressor_service (Singleton, stateless). - Tests: 22 BDD scenarios (features/skeleton_compressor.feature) covering ratio validation, stable ordering, metadata correctness, edge cases, and plan model integration. 6 Robot Framework smoke tests. ASV benchmark suites at 10/100/1000 fragment scales. - Documentation: docs/reference/skeleton_compressor.md with ratio table, algorithm description, metadata schema, and multi-decision plan example. ISSUES CLOSED: #194
110 lines
3.6 KiB
Markdown
110 lines
3.6 KiB
Markdown
# Skeleton Compressor
|
|
|
|
## Overview
|
|
|
|
The **Skeleton Compressor** produces a compressed representation of a
|
|
plan's accumulated context for propagation to child plans as inherited
|
|
context. Compression is governed by the `skeleton_ratio` budget
|
|
parameter set on a project's context policy.
|
|
|
|
The compressor lives in
|
|
`cleveragents.application.services.skeleton_compressor.SkeletonCompressorService`
|
|
and is registered in the DI container as `skeleton_compressor_service`.
|
|
|
|
## Skeleton Ratio
|
|
|
|
| Ratio | Meaning | Behaviour |
|
|
|------:|:--------|:----------|
|
|
| `0.0` | No compression | All fragments pass through unchanged. |
|
|
| `0.3` | Default | ~70 % of tokens retained (top-relevance first). |
|
|
| `0.5` | Moderate | ~50 % of tokens retained. |
|
|
| `0.8` | Heavy | ~20 % of tokens retained. |
|
|
| `1.0` | Maximum | Only the single highest-relevance fragment is kept. |
|
|
|
|
The ratio is validated to the closed interval `[0.0, 1.0]`. Values
|
|
outside this range raise `ValueError`.
|
|
|
|
### Default Handling
|
|
|
|
When a plan or project context policy does not set `skeleton_ratio`,
|
|
the service applies the constant `DEFAULT_SKELETON_RATIO = 0.3`.
|
|
|
|
## Fragment Ordering
|
|
|
|
Fragments are sorted by **relevance descending** with a stable
|
|
secondary sort on **fragment_id ascending**. This guarantees
|
|
deterministic output: identical inputs always produce identical
|
|
compressed payloads regardless of the order in which fragments
|
|
arrive.
|
|
|
|
## Compression Algorithm
|
|
|
|
1. Validate all inputs (ratio, fragment fields).
|
|
2. Compute `original_tokens` — sum of `token_count` across all
|
|
fragments.
|
|
3. Sort fragments by `(-relevance, fragment_id)`.
|
|
4. Compute a token budget: `budget = original_tokens * (1 - ratio)`.
|
|
5. Iterate sorted fragments, accumulating tokens until the budget is
|
|
exhausted.
|
|
6. Return the kept fragments and a `SkeletonMetadata` record.
|
|
|
|
## Metadata
|
|
|
|
Every compression pass produces a frozen `SkeletonMetadata`:
|
|
|
|
| Field | Type | Description |
|
|
|:------|:-----|:------------|
|
|
| `ratio` | `float` | The ratio applied. |
|
|
| `original_tokens` | `int` | Tokens before compression. |
|
|
| `compressed_tokens` | `int` | Tokens after compression. |
|
|
| `source_decision_ids` | `tuple[str, ...]` | Decision ULIDs included. |
|
|
|
|
The metadata is persisted on the `Plan` model via the
|
|
`skeleton_metadata` field, making compression auditable.
|
|
|
|
### Compression Summary
|
|
|
|
The metadata doubles as a compression summary: compare
|
|
`original_tokens` to `compressed_tokens` to see how much context was
|
|
removed. The summary is included in `plan status` CLI output under
|
|
the `skeleton` key.
|
|
|
|
## Example: Multi-Decision Plan
|
|
|
|
Consider a plan with three decision context fragments:
|
|
|
|
```text
|
|
Fragment A (relevance=0.9, tokens=400, decision=01HX...)
|
|
Fragment B (relevance=0.6, tokens=300, decision=01HY...)
|
|
Fragment C (relevance=0.3, tokens=300, decision=01HZ...)
|
|
```
|
|
|
|
With `skeleton_ratio = 0.5` the token budget is
|
|
`1000 * (1 - 0.5) = 500` tokens:
|
|
|
|
- Fragment A (400 tokens, cumulative 400) — kept.
|
|
- Fragment B (300 tokens, cumulative 700) — exceeds budget; skipped.
|
|
|
|
**Result:**
|
|
|
|
```text
|
|
Compressed fragments: [A]
|
|
original_tokens: 1000
|
|
compressed_tokens: 400
|
|
source_decision_ids: (01HX...)
|
|
```
|
|
|
|
## Integration
|
|
|
|
The skeleton output feeds into the **subplan context inheritance**
|
|
pipeline. When a parent plan spawns a child, the strategy coordinator
|
|
calls the compressor on the parent's accumulated context, stores the
|
|
resulting `SkeletonMetadata` on the child plan, and passes the
|
|
compressed fragments as the child's inherited context budget.
|
|
|
|
## CLI
|
|
|
|
The `--skeleton-ratio` flag on `agents project context set` sets the
|
|
ratio for a project's context policy. The `plan status` command
|
|
displays the compression summary when `skeleton_metadata` is present.
|