docs: add ACMS v1 context management CLI reference and API documentation (v3.4.0) [AUTO-DOCS-1]
CI / lint (pull_request) Successful in 42s
CI / typecheck (pull_request) Successful in 50s
CI / security (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 34s
CI / build (pull_request) Successful in 35s
CI / push-validation (pull_request) Successful in 30s
CI / helm (pull_request) Successful in 39s
CI / integration_tests (pull_request) Successful in 4m0s
CI / e2e_tests (pull_request) Successful in 4m29s
CI / unit_tests (pull_request) Successful in 5m53s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 13m38s
CI / status-check (pull_request) Successful in 1s
CI / lint (pull_request) Successful in 42s
CI / typecheck (pull_request) Successful in 50s
CI / security (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 34s
CI / build (pull_request) Successful in 35s
CI / push-validation (pull_request) Successful in 30s
CI / helm (pull_request) Successful in 39s
CI / integration_tests (pull_request) Successful in 4m0s
CI / e2e_tests (pull_request) Successful in 4m29s
CI / unit_tests (pull_request) Successful in 5m53s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 13m38s
CI / status-check (pull_request) Successful in 1s
Add docs/api/context-management.md documenting the ACMS v1 context management system introduced in v3.4.0. Covers agents context list/add/show/clear CLI commands, context policy configuration (YAML format, view-specific settings), budget enforcement (max_file_size, max_total_size), hot/warm/cold storage tier lifecycle management, context analysis and summarization pipeline, Python API (ContextTierService, ProjectContextPolicy, hydration functions), and example workflows for large projects (10,000+ files). Update docs/api/index.md to add a Feature Guides section linking to the new context management guide. Update CHANGELOG.md with Documentation entries under [Unreleased].
This commit is contained in:
@@ -5,6 +5,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Documentation
|
||||
|
||||
- **ACMS v1 Context Management CLI Reference** [AUTO-DOCS-1]: Add
|
||||
`docs/api/context-management.md` documenting the v3.4.0 ACMS v1 context
|
||||
management system, including `agents context list/add/show/clear` CLI commands,
|
||||
context policy configuration (YAML format, view-specific settings), budget
|
||||
enforcement (`max_file_size`, `max_total_size`), hot/warm/cold storage tier
|
||||
lifecycle management, context analysis and summarization pipeline, Python API
|
||||
(`ContextTierService`, `ProjectContextPolicy`, hydration functions), and example
|
||||
workflows for large projects (10,000+ files). Updated `docs/api/index.md` to
|
||||
link to the new guide.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Automation Profile Silent Fallback** (#8232): `_resolve_profile_for_plan` in
|
||||
|
||||
@@ -0,0 +1,934 @@
|
||||
# ACMS v1 Context Management
|
||||
|
||||
**Introduced:** v3.4.0
|
||||
**Module:** `cleveragents.application.services.context_tiers`, `cleveragents.domain.models.core.context_policy`
|
||||
**ADR:** [ADR-014 Context Management (ACMS)](../adr/ADR-014-context-management-acms.md)
|
||||
|
||||
The **Adaptive Context Management System (ACMS) v1** provides CLI commands and a Python API
|
||||
for managing context entries in large projects. ACMS solves the fundamental challenge of
|
||||
finite LLM context windows by selectively indexing, retrieving, and assembling the most
|
||||
relevant project fragments within a configurable token budget.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
ACMS v1 introduces four top-level CLI commands under `agents context` for managing context
|
||||
entries associated with a project:
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| [`agents context list`](#agents-context-list) | List context entries for a project |
|
||||
| [`agents context add`](#agents-context-add) | Add context entries to a project |
|
||||
| [`agents context show`](#agents-context-show) | Show details of a context entry |
|
||||
| [`agents context clear`](#agents-context-clear) | Clear context entries from a project |
|
||||
|
||||
In addition, ACMS v1 provides:
|
||||
|
||||
- **Context policies** — view-specific, phase-aware configuration (see [`agents project context`](../reference/project_context_cli.md))
|
||||
- **Budget enforcement** — `max_file_size` and `max_total_size` constraints
|
||||
- **Hot/warm/cold storage tiers** — lifecycle management for context fragments
|
||||
- **Context analysis** — produces meaningful summaries for LLM calls
|
||||
|
||||
> **Related commands:** For project-level context policy configuration (include/exclude
|
||||
> patterns, ACMS pipeline settings), see the
|
||||
> [`agents project context`](../reference/project_context_cli.md) reference.
|
||||
|
||||
---
|
||||
|
||||
## Context Assembly Pipeline
|
||||
|
||||
ACMS assembles context through a **10-component pipeline** that transforms raw retrieval
|
||||
results into a budget-constrained, coherently ordered context window:
|
||||
|
||||
```
|
||||
ContextRequest
|
||||
│
|
||||
▼
|
||||
1. StrategySelector — selects retrieval strategies with confidence scores
|
||||
│
|
||||
▼
|
||||
2. BudgetAllocator — distributes token budget across strategies
|
||||
│
|
||||
▼
|
||||
3. StrategyExecutor — runs strategies in parallel (timeout: 30s, circuit breaker: 3 failures)
|
||||
│
|
||||
▼
|
||||
4. FragmentDeduplicator — removes duplicates via content-hash, UKO-identity, or semantic similarity
|
||||
│
|
||||
▼
|
||||
5. DetailDepthResolver — resolves depth conflicts for the same UKO node
|
||||
│
|
||||
▼
|
||||
6. FragmentScorer — computes composite relevance scores
|
||||
│ (relevance: 0.4, hierarchy: 0.3, strategy quality: 0.2, recency: 0.1)
|
||||
▼
|
||||
7. BudgetPacker — greedy knapsack packing with depth fallback [9, 4, 2, 0]
|
||||
│
|
||||
▼
|
||||
8. FragmentOrderer — orders fragments for optimal coherence
|
||||
│
|
||||
▼
|
||||
9. PreambleGenerator — generates provenance summaries (max: 200 tokens)
|
||||
│
|
||||
▼
|
||||
10. SkeletonCompressor — compresses parent context for child plan inheritance
|
||||
│
|
||||
▼
|
||||
AssembledContext (delivered to LLM)
|
||||
```
|
||||
|
||||
The budget is calculated as:
|
||||
|
||||
```
|
||||
budget = model_context_window − response_reserve (4096) − tool_definitions − skeleton_allocation
|
||||
```
|
||||
|
||||
Context assembly is skipped when the computed budget falls below `context.budget.min-useful-budget`
|
||||
(default: 500 tokens). Budget refresh is triggered when available budget changes by more than
|
||||
`context.budget.refresh-threshold` (default: 30%).
|
||||
|
||||
---
|
||||
|
||||
## `agents context list`
|
||||
|
||||
List context entries for a project.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
agents context list PROJECT [OPTIONS]
|
||||
```
|
||||
|
||||
### Arguments
|
||||
|
||||
| Argument | Description |
|
||||
|-----------|--------------------------------------------------------------|
|
||||
| `PROJECT` | Project namespaced name (e.g. `local/my-app`) |
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|------------------|-------|------------------------------------------------------|
|
||||
| `--tier`, `-t` | `str` | Filter by storage tier: `hot`, `warm`, `cold`, or `all` (default: `all`) |
|
||||
| `--limit`, `-n` | `int` | Maximum number of entries to return |
|
||||
| `--format`, `-f` | `str` | Output format: `json`, `yaml`, `plain`, `table`, `rich` |
|
||||
|
||||
### Output
|
||||
|
||||
The `list` command displays a table of context entries with:
|
||||
|
||||
- **Fragment ID** — stable unique identifier
|
||||
- **Path** — relative file path within the resource
|
||||
- **Tier** — current storage tier (`hot`, `warm`, or `cold`)
|
||||
- **Tokens** — estimated token count
|
||||
- **Language** — detected programming language
|
||||
- **Last Accessed** — timestamp of last access
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# List all context entries for a project
|
||||
agents context list local/my-app
|
||||
|
||||
# List only hot-tier entries
|
||||
agents context list local/my-app --tier hot
|
||||
|
||||
# List with a limit, in JSON format
|
||||
agents context list local/my-app --limit 50 --format json
|
||||
|
||||
# List warm and cold entries for a large project
|
||||
agents context list local/large-project --tier warm
|
||||
agents context list local/large-project --tier cold
|
||||
```
|
||||
|
||||
### JSON Output Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"command": "",
|
||||
"status": "ok",
|
||||
"exit_code": 0,
|
||||
"data": {
|
||||
"project": "local/my-app",
|
||||
"total_entries": 42,
|
||||
"entries": [
|
||||
{
|
||||
"fragment_id": "01HZ...:<path>",
|
||||
"path": "src/main.py",
|
||||
"tier": "hot",
|
||||
"token_count": 128,
|
||||
"language": "python",
|
||||
"last_accessed": "2026-04-14T09:00:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"timing": { "duration_ms": 12 },
|
||||
"messages": [{ "level": "ok", "text": "42 entries listed" }]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `agents context add`
|
||||
|
||||
Add one or more context entries to a project. Files are read from the linked resource,
|
||||
stored as `TieredFragment` objects in the `ContextTierService`, and placed in the **hot**
|
||||
tier by default.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
agents context add PROJECT PATH [PATH ...] [OPTIONS]
|
||||
```
|
||||
|
||||
### Arguments
|
||||
|
||||
| Argument | Description |
|
||||
|------------|--------------------------------------------------------------|
|
||||
| `PROJECT` | Project namespaced name (e.g. `local/my-app`) |
|
||||
| `PATH` | One or more file paths or glob patterns to add (repeatable) |
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|---------------------|--------|----------------------------------------------------------|
|
||||
| `--tier`, `-t` | `str` | Initial storage tier: `hot`, `warm`, or `cold` (default: `hot`) |
|
||||
| `--recursive`, `-r` | `flag` | Recursively add all files in a directory |
|
||||
| `--format`, `-f` | `str` | Output format: `json`, `yaml`, `plain`, `table`, `rich` |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Add a single file to hot context
|
||||
agents context add local/my-app src/main.py
|
||||
|
||||
# Add multiple files
|
||||
agents context add local/my-app src/auth.py src/models.py
|
||||
|
||||
# Add all Python files recursively
|
||||
agents context add local/my-app "src/**/*.py" --recursive
|
||||
|
||||
# Add a file directly to warm tier
|
||||
agents context add local/my-app docs/architecture.md --tier warm
|
||||
|
||||
# Add a directory of configuration files
|
||||
agents context add local/my-app config/ --recursive
|
||||
```
|
||||
|
||||
### Budget Enforcement
|
||||
|
||||
When adding context entries, ACMS enforces the project's budget constraints:
|
||||
|
||||
- **`max_file_size`** — files exceeding this limit (bytes) are skipped with a warning.
|
||||
Configure via `agents project context set --max-file-size <bytes>`.
|
||||
- **`max_total_size`** — once cumulative context size reaches this limit, additional files
|
||||
are skipped. Configure via `agents project context set --max-total-size <bytes>`.
|
||||
|
||||
Files that exceed limits are reported in the command output but do not cause the command
|
||||
to fail. The command exits with code 0 if at least one file was added successfully.
|
||||
|
||||
### Output
|
||||
|
||||
```
|
||||
╭──────────────────────────────────────────────────────╮
|
||||
│ Context Added — local/my-app │
|
||||
│ Files added: 3 │
|
||||
│ Files skipped: 1 (exceeded max_file_size) │
|
||||
│ Tokens added: 1,024 │
|
||||
│ Tier: hot │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
✓ OK Context updated
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `agents context show`
|
||||
|
||||
Show the details and content of a specific context entry.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
agents context show PROJECT FRAGMENT_ID [OPTIONS]
|
||||
```
|
||||
|
||||
### Arguments
|
||||
|
||||
| Argument | Description |
|
||||
|---------------|--------------------------------------------------------------|
|
||||
| `PROJECT` | Project namespaced name (e.g. `local/my-app`) |
|
||||
| `FRAGMENT_ID` | Fragment ID as shown by `agents context list` |
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|------------------|-------|------------------------------------------------------|
|
||||
| `--format`, `-f` | `str` | Output format: `json`, `yaml`, `plain`, `table`, `rich` |
|
||||
|
||||
### Output
|
||||
|
||||
The `show` command displays:
|
||||
|
||||
- **Fragment metadata** — ID, path, tier, token count, language, timestamps
|
||||
- **Content** — the full text content of the fragment as stored in the context tier
|
||||
- **Relevance score** — the fragment's composite relevance score (0.0–1.0)
|
||||
- **Detail depth** — the UKO detail depth level (0–9)
|
||||
- **Access statistics** — access count and last accessed timestamp
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Show a context entry by fragment ID
|
||||
agents context show local/my-app "01HZ...:src/main.py"
|
||||
|
||||
# Show in JSON format for scripting
|
||||
agents context show local/my-app "01HZ...:src/main.py" --format json
|
||||
```
|
||||
|
||||
### Output Format
|
||||
|
||||
```
|
||||
╭──────────────────────────────────────────────────────────────╮
|
||||
│ Fragment: 01HZ...:src/main.py │
|
||||
│ Project: local/my-app │
|
||||
│ Tier: hot │
|
||||
│ Tokens: 128 │
|
||||
│ Language: python │
|
||||
│ Depth: 1 │
|
||||
│ Score: 0.75 │
|
||||
│ Accessed: 2026-04-14T09:00:00Z (42 times) │
|
||||
╰──────────────────────────────────────────────────────────────╯
|
||||
╭──────────────────────────────────────────────────────────────╮
|
||||
│ Content │
|
||||
│ ─────────────────────────────────────────────────────────── │
|
||||
│ # src/main.py │
|
||||
│ def main(): │
|
||||
│ print("Hello from CleverAgents!") │
|
||||
╰──────────────────────────────────────────────────────────────╯
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `agents context clear`
|
||||
|
||||
Clear (remove) context entries from a project. Entries can be cleared by tier, by path
|
||||
pattern, or all at once.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
agents context clear PROJECT [OPTIONS]
|
||||
```
|
||||
|
||||
### Arguments
|
||||
|
||||
| Argument | Description |
|
||||
|-----------|--------------------------------------------------------------|
|
||||
| `PROJECT` | Project namespaced name (e.g. `local/my-app`) |
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|---------------------|--------|----------------------------------------------------------|
|
||||
| `--tier`, `-t` | `str` | Clear only entries in this tier: `hot`, `warm`, or `cold` |
|
||||
| `--path`, `-p` | `str` | Clear entries matching this path pattern (glob) |
|
||||
| `--all` | `flag` | Clear all context entries for the project |
|
||||
| `--yes`, `-y` | `flag` | Skip confirmation prompt (for scripting) |
|
||||
| `--format`, `-f` | `str` | Output format: `json`, `yaml`, `plain`, `table`, `rich` |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Clear all context entries (with confirmation prompt)
|
||||
agents context clear local/my-app --all
|
||||
|
||||
# Clear without confirmation (for CI/scripting)
|
||||
agents context clear local/my-app --all --yes
|
||||
|
||||
# Clear only cold-tier entries
|
||||
agents context clear local/my-app --tier cold --yes
|
||||
|
||||
# Clear entries matching a path pattern
|
||||
agents context clear local/my-app --path "tests/**" --yes
|
||||
|
||||
# Clear warm and cold tiers (run twice)
|
||||
agents context clear local/my-app --tier warm --yes
|
||||
agents context clear local/my-app --tier cold --yes
|
||||
```
|
||||
|
||||
### Output
|
||||
|
||||
```
|
||||
╭──────────────────────────────────────────────────────╮
|
||||
│ Context Cleared — local/my-app │
|
||||
│ Entries removed: 127 │
|
||||
│ Tiers affected: hot, warm, cold │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
✓ OK Context cleared
|
||||
```
|
||||
|
||||
> **Note:** Clearing context entries does not affect the project's ACMS configuration or
|
||||
> context policy. The next plan execution will re-hydrate the context tiers from the
|
||||
> project's linked resources automatically.
|
||||
|
||||
---
|
||||
|
||||
## Context Policy Configuration
|
||||
|
||||
Context policies control what resources and files are visible during each ACMS phase.
|
||||
Policies use **view inheritance** so each phase can selectively override or extend its parent.
|
||||
|
||||
### Inheritance Chain
|
||||
|
||||
```
|
||||
default → strategize → execute → apply
|
||||
```
|
||||
|
||||
| Phase | Inherits from |
|
||||
|--------------|---------------|
|
||||
| `default` | *(none)* |
|
||||
| `strategize` | `default` |
|
||||
| `execute` | `strategize` |
|
||||
| `apply` | `execute` |
|
||||
|
||||
An empty policy defaults to including everything. Exclusions always take precedence over
|
||||
inclusions.
|
||||
|
||||
### YAML Configuration Format
|
||||
|
||||
Context policies can be expressed in YAML for documentation and version control purposes.
|
||||
The following structure mirrors the `ProjectContextPolicy` domain model:
|
||||
|
||||
```yaml
|
||||
# context-policy.yaml
|
||||
default_view:
|
||||
include_resources:
|
||||
- "db-*"
|
||||
- "api-*"
|
||||
exclude_resources:
|
||||
- "db-test"
|
||||
include_paths:
|
||||
- "src/**/*.py"
|
||||
- "src/**/*.ts"
|
||||
exclude_paths:
|
||||
- "**/*.pyc"
|
||||
- "**/__pycache__/**"
|
||||
max_file_size: 1048576 # 1 MB per file
|
||||
max_total_size: 52428800 # 50 MB total
|
||||
|
||||
strategize_view:
|
||||
include_resources:
|
||||
- "db-*"
|
||||
- "api-*"
|
||||
- "cache-*"
|
||||
max_file_size: 2097152 # 2 MB per file (override for strategize phase)
|
||||
|
||||
# execute_view and apply_view inherit from strategize_view if not set
|
||||
```
|
||||
|
||||
### Applying a Policy via CLI
|
||||
|
||||
```bash
|
||||
# Set default view with resource and path filters
|
||||
agents project context set local/my-app \
|
||||
--view default \
|
||||
--include-resource "db-*" \
|
||||
--include-resource "api-*" \
|
||||
--exclude-resource "db-test" \
|
||||
--include-path "src/**/*.py" \
|
||||
--exclude-path "**/*.pyc" \
|
||||
--max-file-size 1048576 \
|
||||
--max-total-size 52428800
|
||||
|
||||
# Override strategize view with additional resources
|
||||
agents project context set local/my-app \
|
||||
--view strategize \
|
||||
--include-resource "db-*" \
|
||||
--include-resource "api-*" \
|
||||
--include-resource "cache-*" \
|
||||
--max-file-size 2097152
|
||||
|
||||
# Clear execute view so it inherits from strategize
|
||||
agents project context set local/my-app --view execute --clear
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Budget Enforcement
|
||||
|
||||
ACMS enforces two budget constraints at the `ContextView` level:
|
||||
|
||||
### `max_file_size`
|
||||
|
||||
Maximum size in bytes for a single file to be included in context. Files exceeding this
|
||||
limit are skipped during indexing and hydration.
|
||||
|
||||
| Setting | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `max_file_size` | `None` (no limit) | Per-file size cap in bytes |
|
||||
|
||||
**Recommended values for large projects:**
|
||||
|
||||
| Project Scale | Recommended `max_file_size` |
|
||||
|---------------|----------------------------|
|
||||
| Small (< 1K files) | `None` (no limit) |
|
||||
| Medium (1K–10K files) | `1048576` (1 MB) |
|
||||
| Large (10K+ files) | `262144` (256 KB) |
|
||||
|
||||
### `max_total_size`
|
||||
|
||||
Maximum cumulative size in bytes across all context fragments. Indexing stops once this
|
||||
budget is reached.
|
||||
|
||||
| Setting | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `max_total_size` | `None` (no limit) | Total context size cap in bytes |
|
||||
|
||||
**Recommended values for large projects:**
|
||||
|
||||
| Project Scale | Recommended `max_total_size` |
|
||||
|---------------|------------------------------|
|
||||
| Small (< 1K files) | `None` (no limit) |
|
||||
| Medium (1K–10K files) | `10485760` (10 MB) |
|
||||
| Large (10K+ files) | `52428800` (50 MB) |
|
||||
|
||||
### Validation Rules
|
||||
|
||||
- Both `max_file_size` and `max_total_size` must be positive integers or `None`.
|
||||
- Zero and negative values are rejected with a `ValidationError`.
|
||||
- `max_total_size` must be greater than or equal to `max_file_size` when both are set.
|
||||
|
||||
```python
|
||||
from cleveragents.domain.models.core.context_policy import ContextView
|
||||
|
||||
# Valid
|
||||
view = ContextView(max_file_size=1_048_576, max_total_size=52_428_800)
|
||||
|
||||
# Invalid — raises ValidationError
|
||||
view = ContextView(max_file_size=0) # zero not allowed
|
||||
view = ContextView(max_file_size=-1) # negative not allowed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hot/Warm/Cold Storage Tiers
|
||||
|
||||
ACMS manages context fragments across three storage tiers with different latency,
|
||||
capacity, and retention characteristics.
|
||||
|
||||
### Tier Overview
|
||||
|
||||
| Tier | Backend | Latency | Capacity | Actor Visibility |
|
||||
|--------|-------------|---------|------------------|-----------------|
|
||||
| **Hot** | In-memory | Low | Token-budget | All actors |
|
||||
| **Warm** | SQLite stub | Medium | Decision-count | Strategist, Executor |
|
||||
| **Cold** | File stub | High | Decision-count | Strategist only |
|
||||
|
||||
### Actor Role Visibility
|
||||
|
||||
| Actor Role | Sees Hot | Sees Warm | Sees Cold |
|
||||
|--------------|----------|-----------|-----------|
|
||||
| `strategist` | ✓ | ✓ | ✓ |
|
||||
| `executor` | ✓ | ✓ | ✗ |
|
||||
| `reviewer` | ✓ | ✗ | ✗ |
|
||||
|
||||
### Tier Lifecycle
|
||||
|
||||
Fragments move between tiers based on access patterns and LRU eviction:
|
||||
|
||||
```
|
||||
promote()
|
||||
Cold ──────────────────────────────► Warm ──────────────────────────────► Hot
|
||||
◄────────────────────────────── ◄──────────────────────────────
|
||||
demote() demote()
|
||||
(+ summarization)
|
||||
```
|
||||
|
||||
- **`promote(fragment_id)`** — moves a fragment from cold→warm or warm→hot
|
||||
- **`demote(fragment_id)`** — moves a fragment from hot→warm or warm→cold (with optional
|
||||
summarization on demotion to cold)
|
||||
- **`evict_lru(tier, count)`** — evicts the least-recently-used fragments from a tier
|
||||
|
||||
### Tier Configuration
|
||||
|
||||
Tier capacity is controlled by `TierBudget`:
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `max_tokens_hot` | 8,000 | Maximum tokens in the hot tier |
|
||||
| `max_decisions_warm` | 500 | Maximum fragment count in the warm tier |
|
||||
| `max_decisions_cold` | 5,000 | Maximum fragment count in the cold tier |
|
||||
|
||||
Override via environment variables:
|
||||
|
||||
```bash
|
||||
export CLEVERAGENTS_CONTEXT_MAX_TOKENS_HOT=16000
|
||||
export CLEVERAGENTS_CONTEXT_MAX_DECISIONS_WARM=1000
|
||||
export CLEVERAGENTS_CONTEXT_MAX_DECISIONS_COLD=10000
|
||||
```
|
||||
|
||||
Or via `agents project context set`:
|
||||
|
||||
```bash
|
||||
agents project context set local/my-app \
|
||||
--hot-max-tokens 16000 \
|
||||
--warm-max-decisions 1000 \
|
||||
--cold-max-decisions 10000
|
||||
```
|
||||
|
||||
### Retention Periods
|
||||
|
||||
| Tier | Retention | Config Key |
|
||||
|------|-----------|------------|
|
||||
| Hot | Session lifetime | N/A (in-memory) |
|
||||
| Warm | 24 hours (default) | `context.tiers.warm.retention-hours` |
|
||||
| Cold | 90 days (default) | `context.tiers.cold.retention-days` |
|
||||
|
||||
---
|
||||
|
||||
## Context Analysis and Summarization
|
||||
|
||||
ACMS produces **context analysis summaries** that are prepended to assembled context
|
||||
before LLM calls. These summaries help the LLM understand the provenance and structure
|
||||
of the context it receives.
|
||||
|
||||
### Preamble Generation
|
||||
|
||||
The `PreambleGenerator` (pipeline component 9) generates a provenance summary for each
|
||||
assembled context. The preamble includes:
|
||||
|
||||
- Total fragment count and token usage
|
||||
- Strategies used and their contribution percentages
|
||||
- Top-level resource and file paths included
|
||||
- Budget utilization percentage
|
||||
|
||||
**Example preamble:**
|
||||
|
||||
```
|
||||
Context assembled from 23 fragments (4,218 / 8,000 tokens, 52.7% budget used).
|
||||
Strategies: breadth-depth-navigator (52%), simple-keyword (31%), semantic-embedding (17%).
|
||||
Resources: local/my-app (git-checkout). Top paths: src/auth.py, src/models.py, src/api/routes.py.
|
||||
```
|
||||
|
||||
### Summarization on Demotion
|
||||
|
||||
When a fragment is demoted from warm to cold tier, ACMS can optionally summarize its
|
||||
content to reduce storage requirements. Summarization is controlled by:
|
||||
|
||||
```bash
|
||||
# Enable summarization (default: enabled)
|
||||
agents project context set local/my-app --summarize
|
||||
|
||||
# Disable summarization
|
||||
agents project context set local/my-app --no-summarize
|
||||
|
||||
# Set maximum tokens for summaries
|
||||
agents project context set local/my-app --summary-max-tokens 200
|
||||
```
|
||||
|
||||
### Context Analysis Output
|
||||
|
||||
Use `agents project context simulate` to preview what context would be assembled for a
|
||||
project without executing a plan:
|
||||
|
||||
```bash
|
||||
# Simulate context assembly with default settings
|
||||
agents project context simulate local/my-app
|
||||
|
||||
# Simulate with a custom token budget
|
||||
agents project context simulate local/my-app --budget 4000
|
||||
|
||||
# Simulate for a specific phase
|
||||
agents project context simulate local/my-app --view strategize
|
||||
```
|
||||
|
||||
The simulate output includes a **Summary** panel showing token usage, strategies, and
|
||||
fragment count — the same information that appears in the preamble delivered to the LLM.
|
||||
|
||||
---
|
||||
|
||||
## Python API
|
||||
|
||||
### ContextTierService
|
||||
|
||||
The primary service for managing context fragments programmatically.
|
||||
|
||||
```python
|
||||
from cleveragents.application.services.context_tiers import (
|
||||
ContextTierService,
|
||||
ContextTier,
|
||||
ActorRole,
|
||||
TieredFragment,
|
||||
TierBudget,
|
||||
)
|
||||
from cleveragents.application.container import get_container
|
||||
|
||||
# Get the singleton service from the DI container
|
||||
container = get_container()
|
||||
svc = container.context_tier_service()
|
||||
|
||||
# Store a fragment in the hot tier
|
||||
fragment = TieredFragment(
|
||||
fragment_id="my-resource-id:src/main.py",
|
||||
content="# src/main.py\ndef main():\n pass\n",
|
||||
tier=ContextTier.HOT,
|
||||
resource_id="01HZ...",
|
||||
project_name="local/my-app",
|
||||
token_count=12,
|
||||
)
|
||||
svc.store(fragment)
|
||||
|
||||
# Retrieve a fragment by ID
|
||||
frag = svc.get("my-resource-id:src/main.py")
|
||||
|
||||
# Promote from warm to hot
|
||||
svc.promote("my-resource-id:src/main.py")
|
||||
|
||||
# Demote from hot to warm (with optional summarization)
|
||||
svc.demote("my-resource-id:src/main.py")
|
||||
|
||||
# Evict 5 least-recently-used fragments from hot tier
|
||||
svc.evict_lru(ContextTier.HOT, count=5)
|
||||
|
||||
# Get all fragments visible to a strategist for a project
|
||||
fragments = svc.get_for_actor(
|
||||
ActorRole.STRATEGIST,
|
||||
project_names=["local/my-app"],
|
||||
)
|
||||
|
||||
# Get a project-scoped view
|
||||
view = svc.get_scoped_view(["local/my-app"])
|
||||
|
||||
# Get tier metrics
|
||||
metrics = svc.get_metrics()
|
||||
print(f"Hot hits: {metrics.hot_hit_count}, misses: {metrics.hot_miss_count}")
|
||||
```
|
||||
|
||||
### ProjectContextPolicy
|
||||
|
||||
Configure view-specific context policies programmatically.
|
||||
|
||||
```python
|
||||
from cleveragents.domain.models.core.context_policy import (
|
||||
ContextView,
|
||||
ProjectContextPolicy,
|
||||
)
|
||||
|
||||
# Create a policy with phase-specific views
|
||||
policy = ProjectContextPolicy(
|
||||
default_view=ContextView(
|
||||
include_resources=["db-*", "api-*"],
|
||||
exclude_paths=["**/*.pyc", "**/__pycache__/**"],
|
||||
max_file_size=1_048_576, # 1 MB
|
||||
max_total_size=52_428_800, # 50 MB
|
||||
),
|
||||
strategize_view=ContextView(
|
||||
include_resources=["db-*", "api-*", "cache-*"],
|
||||
max_file_size=2_097_152, # 2 MB for strategize phase
|
||||
),
|
||||
# execute_view and apply_view inherit from strategize_view
|
||||
)
|
||||
|
||||
# Resolve the effective view for a phase
|
||||
execute_view = policy.resolve_view("execute")
|
||||
assert execute_view.include_resources == ["db-*", "api-*", "cache-*"]
|
||||
|
||||
# Serialize to JSON
|
||||
import json
|
||||
data = json.loads(policy.model_dump_json())
|
||||
|
||||
# Deserialize from JSON
|
||||
restored = ProjectContextPolicy.model_validate(data)
|
||||
```
|
||||
|
||||
### Context Hydration
|
||||
|
||||
Hydrate context tiers from project resources before plan execution.
|
||||
|
||||
```python
|
||||
from cleveragents.application.services.context_tier_hydrator import (
|
||||
hydrate_tiers_from_project,
|
||||
hydrate_tiers_for_plan,
|
||||
)
|
||||
|
||||
# Hydrate from a single resource
|
||||
count = hydrate_tiers_from_project(
|
||||
tier_service=svc,
|
||||
project_name="local/my-app",
|
||||
resource_id="01HZ...",
|
||||
resource_location="/path/to/project",
|
||||
resource_type="git-checkout",
|
||||
)
|
||||
print(f"Hydrated {count} fragments")
|
||||
|
||||
# Hydrate for all projects linked to a plan
|
||||
total = hydrate_tiers_for_plan(
|
||||
tier_service=svc,
|
||||
project_names=["local/my-app"],
|
||||
project_repository=project_repo,
|
||||
resource_registry=resource_registry,
|
||||
)
|
||||
print(f"Total fragments hydrated: {total}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Example Workflows for Large Projects (10,000+ Files)
|
||||
|
||||
### Workflow 1: Initial Setup for a Large Monorepo
|
||||
|
||||
```bash
|
||||
# 1. Create the project
|
||||
agents project create local/my-monorepo \
|
||||
--description "Large monorepo with 10K+ files"
|
||||
|
||||
# 2. Link the git repository resource
|
||||
agents project link-resource local/my-monorepo \
|
||||
--resource-type git-checkout \
|
||||
--location /path/to/monorepo
|
||||
|
||||
# 3. Configure budget constraints for scale
|
||||
agents project context set local/my-monorepo \
|
||||
--view default \
|
||||
--max-file-size 262144 \
|
||||
--max-total-size 52428800 \
|
||||
--include-path "src/**/*.py" \
|
||||
--include-path "src/**/*.ts" \
|
||||
--exclude-path "**/node_modules/**" \
|
||||
--exclude-path "**/__pycache__/**" \
|
||||
--exclude-path "**/dist/**"
|
||||
|
||||
# 4. Configure ACMS pipeline for large-scale retrieval
|
||||
agents project context set local/my-monorepo \
|
||||
--hot-max-tokens 16000 \
|
||||
--warm-max-decisions 1000 \
|
||||
--strategy breadth-depth-navigator \
|
||||
--strategy semantic-embedding \
|
||||
--default-breadth 3 \
|
||||
--default-depth 4
|
||||
|
||||
# 5. Verify the configuration
|
||||
agents project context show local/my-monorepo
|
||||
|
||||
# 6. Simulate context assembly to check budget utilization
|
||||
agents project context simulate local/my-monorepo --budget 16000
|
||||
```
|
||||
|
||||
### Workflow 2: Phase-Specific Context for a Microservices Project
|
||||
|
||||
```bash
|
||||
# Default view: only shared libraries
|
||||
agents project context set local/microservices \
|
||||
--view default \
|
||||
--include-resource "shared-lib-*" \
|
||||
--max-file-size 524288
|
||||
|
||||
# Strategize view: add all service interfaces
|
||||
agents project context set local/microservices \
|
||||
--view strategize \
|
||||
--include-resource "shared-lib-*" \
|
||||
--include-resource "service-api-*" \
|
||||
--max-file-size 1048576
|
||||
|
||||
# Execute view: add implementation files for the target service
|
||||
agents project context set local/microservices \
|
||||
--view execute \
|
||||
--include-resource "shared-lib-*" \
|
||||
--include-resource "service-api-*" \
|
||||
--include-resource "auth-service" \
|
||||
--max-file-size 2097152
|
||||
|
||||
# Apply view inherits from execute (no override needed)
|
||||
|
||||
# Inspect the resolved views
|
||||
agents project context show local/microservices --view default
|
||||
agents project context show local/microservices --view strategize
|
||||
agents project context show local/microservices --view execute
|
||||
```
|
||||
|
||||
### Workflow 3: Managing Context Lifecycle for Long-Running Projects
|
||||
|
||||
```bash
|
||||
# List current hot-tier entries
|
||||
agents context list local/my-app --tier hot
|
||||
|
||||
# Add critical files to ensure they stay in hot tier
|
||||
agents context add local/my-app \
|
||||
src/core/engine.py \
|
||||
src/core/config.py \
|
||||
src/api/routes.py \
|
||||
--tier hot
|
||||
|
||||
# Check what's in warm tier (recently accessed but not hot)
|
||||
agents context list local/my-app --tier warm --format json
|
||||
|
||||
# Show details of a specific fragment
|
||||
agents context show local/my-app "01HZ...:src/core/engine.py"
|
||||
|
||||
# Clear stale cold-tier entries to free storage
|
||||
agents context clear local/my-app --tier cold --yes
|
||||
|
||||
# Inspect tier metrics after cleanup
|
||||
agents project context inspect local/my-app
|
||||
```
|
||||
|
||||
### Workflow 4: Scripting Context Management in CI/CD
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# ci-context-setup.sh — Configure context for CI plan execution
|
||||
|
||||
PROJECT="local/ci-project"
|
||||
MAX_FILE_SIZE=262144 # 256 KB
|
||||
MAX_TOTAL_SIZE=52428800 # 50 MB
|
||||
|
||||
# Configure context policy
|
||||
agents project context set "$PROJECT" \
|
||||
--view default \
|
||||
--max-file-size "$MAX_FILE_SIZE" \
|
||||
--max-total-size "$MAX_TOTAL_SIZE" \
|
||||
--include-path "src/**" \
|
||||
--exclude-path "**/test_*" \
|
||||
--exclude-path "**/__pycache__/**"
|
||||
|
||||
# Verify configuration (JSON for parsing)
|
||||
CONFIG=$(agents project context show "$PROJECT" --format json)
|
||||
echo "Context policy configured:"
|
||||
echo "$CONFIG" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
policy = data['data']['policy']
|
||||
print(f\" max_file_size: {policy['default_view']['max_file_size']}\")
|
||||
print(f\" max_total_size: {policy['default_view']['max_total_size']}\")
|
||||
"
|
||||
|
||||
# Clear stale context from previous runs
|
||||
agents context clear "$PROJECT" --all --yes
|
||||
|
||||
# Simulate to verify budget utilization
|
||||
SIMULATION=$(agents project context simulate "$PROJECT" --format json)
|
||||
BUDGET_PCT=$(echo "$SIMULATION" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
summary = data['data']['summary']
|
||||
print(f\"{summary['budget_used_pct']:.1f}\")
|
||||
")
|
||||
echo "Estimated budget utilization: ${BUDGET_PCT}%"
|
||||
|
||||
if (( $(echo "$BUDGET_PCT > 90" | python3 -c "import sys; print(int(eval(sys.stdin.read())))") )); then
|
||||
echo "WARNING: Budget utilization > 90%. Consider increasing hot-max-tokens."
|
||||
fi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Project Context CLI Reference](../reference/project_context_cli.md) — `agents project context set/show/inspect/simulate`
|
||||
- [Project Context Policy](../reference/project_context_policy.md) — `ProjectContextPolicy` domain model
|
||||
- [Context Tiers](../reference/context_tiers.md) — `ContextTierService`, tier models, and configuration
|
||||
- [Context Strategy Registry](../reference/context_strategies.md) — retrieval strategies and configuration
|
||||
- [Context Hydration](../modules/context-hydration.md) — `hydrate_tiers_from_project` and `hydrate_tiers_for_plan`
|
||||
- [Repository Indexing Service](../reference/context_indexing.md) — file-level index for 10K+ file projects
|
||||
- [ADR-014 Context Management (ACMS)](../adr/ADR-014-context-management-acms.md) — architectural decision record
|
||||
@@ -19,6 +19,12 @@ classes, functions, and exceptions with signatures and usage examples.
|
||||
| [`cleveragents.providers`](providers.md) | AI provider registry — discovery, selection, LLM factory, and capability metadata |
|
||||
| [`cleveragents.tui`](tui.md) | Interactive Terminal UI — app, persona system, input routing, slash commands, session export/import |
|
||||
|
||||
## Feature Guides
|
||||
|
||||
| Guide | Description |
|
||||
|-------|-------------|
|
||||
| [ACMS v1 Context Management](context-management.md) | `agents context list/add/show/clear` CLI reference, context policies, budget enforcement, hot/warm/cold storage tiers, and Python API (v3.4.0+) |
|
||||
|
||||
> **Note:** Internal modules (prefixed with `_`) and implementation details
|
||||
> not listed in a module's `__all__` are considered private and may change
|
||||
> without notice.
|
||||
|
||||
Reference in New Issue
Block a user