Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 911fe64344 | |||
| b25f34a0d4 |
+117
@@ -203,7 +203,124 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
supervisors). Updated all numeric references, pre-flight checklists, and validation logic.
|
||||
|
||||
---
|
||||
## [3.5.0] — Unreleased (Autonomy Hardening)
|
||||
|
||||
### Added
|
||||
|
||||
- **A2A Facade Session and Plan Lifecycle** — Session and plan lifecycle operations
|
||||
are now functional via the CLI using the A2A (Agent-to-Agent) protocol facade.
|
||||
Supports `agents session start/list/stop` and plan execution via `agents plan execute
|
||||
--session SESSION_ID`. Follows ADR-047 (A2A Standard Adoption).
|
||||
|
||||
- **Event Queue Publish/Subscribe** — A new event queue system enables asynchronous
|
||||
publish/subscribe communication between agents and plan lifecycle components.
|
||||
Supports `agents events subscribe` and `agents events publish` CLI commands.
|
||||
Event types include `PLAN_CREATED`, `STRATEGIZE_COMPLETE`, `EXECUTE_COMPLETE`,
|
||||
`APPLY_COMPLETE`, `CORRECTION_APPLIED`, `INVARIANT_VIOLATED`, `SUBPLAN_STARTED`,
|
||||
`SUBPLAN_COMPLETE`, `SUBPLAN_FAILED`, and `GUARD_TRIGGERED`.
|
||||
|
||||
- **Guard Enforcement** — Three guard types protect against runaway autonomous
|
||||
execution: denylist (blocks specific tools/commands/file patterns), budget caps
|
||||
(limits LLM tokens, cost, wall time, file writes), and tool call limits (caps
|
||||
tool calls per subplan/plan and consecutive failures). Guards are configurable
|
||||
at tool, subplan, plan, and global levels.
|
||||
|
||||
- **Automation Profile Resolution Precedence** — Automation profiles now resolve
|
||||
with correct precedence: plan-level > action-level > global. The
|
||||
`_resolve_profile_for_plan` function raises a clear `ValidationError` when an
|
||||
unknown profile name is configured, listing available built-in profiles
|
||||
(`manual`, `semi-auto`, `auto`, `supervised`).
|
||||
|
||||
- **Hierarchical Plan Decomposition (4+ Levels)** — The strategy actor now
|
||||
decomposes plans into hierarchical subplan trees with 4+ levels of depth.
|
||||
Decomposition considers dependency analysis, complexity estimation, risk
|
||||
assessment, and resource constraints.
|
||||
|
||||
- **Decision Correction with Selective Subtree Recomputation** — When a decision
|
||||
correction is applied, only the affected subtree is recomputed. Unaffected
|
||||
branches of the plan tree are preserved, minimizing wasted work.
|
||||
|
||||
- **Parallel Execution Scaling (10+ Concurrent Subplans)** — `SubplanExecutionService`
|
||||
now scales to 10+ concurrent subplans via `ThreadPoolExecutor`. Fail-fast
|
||||
cancellation correctly overrides in-flight subplan results to `CANCELLED` when
|
||||
`stop_flag` is active, preventing corrupted merge output.
|
||||
|
||||
- **Large-Scale Autonomous Task Execution** — A realistic large-scale porting task
|
||||
(Firefox-scale codebase) completes autonomously using hierarchical decomposition,
|
||||
parallel execution, and validation-gated apply.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Server Stubs Moved to M9 (v3.8.0)** — Server stubs previously scoped to this
|
||||
milestone have been moved to v3.8.0 following the ACP to A2A protocol adoption
|
||||
(ADR-047) and server architecture redesign (ADR-048).
|
||||
|
||||
- **TUI Features Moved to M8 (v3.7.0)** — TUI features previously scoped to this
|
||||
milestone have been moved to v3.7.0.
|
||||
|
||||
### Technical
|
||||
|
||||
- `nox` passes with coverage >= 97% including large-project suites
|
||||
- Hierarchical decomposition validated at 4+ levels of subplan depth
|
||||
- Parallel execution validated at 10+ concurrent subplans
|
||||
- Decision correction validated for selective subtree recomputation only
|
||||
|
||||
---
|
||||
|
||||
## [3.4.0] — Unreleased (ACMS v1 + Context Scaling)
|
||||
|
||||
### Added
|
||||
|
||||
- **Advanced Context Management System (ACMS) v1** — The ACMS pipeline is now
|
||||
operational. Projects with 10,000+ files can be indexed and queried. The context
|
||||
assembly pipeline produces scoped, budget-constrained context views for actors.
|
||||
Hot/warm/cold storage tiers manage context lifecycle. See `docs/context.md` for
|
||||
full documentation.
|
||||
|
||||
- **Context Assembly CLI** — New `agents context` command suite:
|
||||
- `agents context list` — List all context fragments grouped by storage tier
|
||||
- `agents context add PATH` — Manually add a file or directory to context
|
||||
- `agents context show` — Show fragment details or assembled context view
|
||||
- `agents context clear` — Remove fragments from the tier service
|
||||
|
||||
- **Context Policies with View-Specific Settings** — Context policies are
|
||||
configurable in `.cleveragents/config.toml` with per-view overrides. Supports
|
||||
`max_file_size`, `max_total_size`, tier capacity thresholds, and file exclusion
|
||||
patterns.
|
||||
|
||||
- **Budget Enforcement** — `max_file_size` (default 256 KB) and `max_total_size`
|
||||
(default 10 MB) constraints are enforced during context assembly. Files exceeding
|
||||
`max_file_size` are skipped. Assembly stops when `max_total_size` is reached.
|
||||
|
||||
- **Context Analysis Summaries** — Context assembly produces meaningful summaries
|
||||
of the assembled context, including fragment counts per tier, total size, and
|
||||
coverage metrics.
|
||||
|
||||
- **ACMS Integration with Plan Execution** — Plan execution (`agents plan execute`)
|
||||
now leverages ACMS context for LLM calls. The `LLMExecuteActor` receives a
|
||||
scoped, budget-constrained context view assembled from project resources.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **ContextTierService Thread Safety** (#7547) — Added `threading.RLock` to
|
||||
`ContextTierService` to prevent `RuntimeError: dictionary changed size during
|
||||
iteration` and data corruption under concurrent plan execution. All public
|
||||
methods now acquire the reentrant lock before accessing tier data structures.
|
||||
|
||||
- **ACMS Context Tier Hydration** (#1028) — `ContextTierService` no longer starts
|
||||
empty on every CLI invocation. A new `context_tier_hydrator.py` reads files from
|
||||
linked project resources, creates `TieredFragment` objects, and stores them in
|
||||
the tier service before context assembly. Respects max file size (256 KB), total
|
||||
budget (10 MB), binary file exclusion, and directory skipping.
|
||||
|
||||
### Technical
|
||||
|
||||
- Projects with 10,000+ files index without timeout
|
||||
- Context window management validated across hot/warm/cold tiers
|
||||
- ACMS v1 pipeline produces scoped context output
|
||||
- Test coverage >= 97%
|
||||
|
||||
---
|
||||
## [3.8.0] — 2026-04-05
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,421 @@
|
||||
# Autonomy Hardening
|
||||
|
||||
> **Milestone:** v3.5.0 — Autonomy Hardening
|
||||
|
||||
CleverAgents v3.5.0 introduces comprehensive autonomy hardening, enabling the system to autonomously execute large-scale tasks using hierarchical plan decomposition, parallel execution, and validation-gated apply.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Autonomy hardening provides the infrastructure for safe, scalable autonomous task execution:
|
||||
|
||||
- **Hierarchical decomposition** — Plans decompose into 4+ levels of subplans
|
||||
- **Parallel execution** — 10+ concurrent subplans execute simultaneously
|
||||
- **Guard enforcement** — Denylist, budget caps, and tool call limits protect against runaway execution
|
||||
- **Automation profiles** — Configurable precedence (plan > action > global) controls autonomy level
|
||||
- **A2A protocol** — Agent-to-Agent facade enables session and plan lifecycle operations via CLI
|
||||
- **Event queue** — Publish/subscribe system coordinates asynchronous agent communication
|
||||
|
||||
> **Note on Server Stubs:** Server stubs previously scoped to this milestone have been moved to M9 (v3.8.0) following the ACP to A2A protocol adoption (ADR-047) and server architecture redesign (ADR-048). TUI features were moved to M8 (v3.7.0).
|
||||
|
||||
---
|
||||
|
||||
## A2A Protocol (Agent-to-Agent)
|
||||
|
||||
The A2A (Agent-to-Agent) protocol provides a standardized facade for session and plan lifecycle operations.
|
||||
|
||||
### Overview
|
||||
|
||||
A2A replaces the earlier ACP (Agent Communication Protocol) following ADR-047. It provides:
|
||||
|
||||
- Standardized session management across agent boundaries
|
||||
- Plan lifecycle operations (create, start, execute, apply, correct)
|
||||
- Event-driven communication via publish/subscribe
|
||||
- Type-safe message schemas
|
||||
|
||||
### A2A Facade CLI
|
||||
|
||||
The A2A facade exposes session and plan lifecycle operations via the `agents` CLI:
|
||||
|
||||
```
|
||||
# Start an A2A session
|
||||
agents session start --profile semi-auto
|
||||
|
||||
# List active sessions
|
||||
agents session list
|
||||
|
||||
# Execute a plan via A2A facade
|
||||
agents plan execute PLAN_ID --session SESSION_ID
|
||||
|
||||
# Subscribe to plan events
|
||||
agents events subscribe --plan PLAN_ID
|
||||
```
|
||||
|
||||
### Session Lifecycle
|
||||
|
||||
```
|
||||
CREATE --> ACTIVE --> SUSPENDED --> TERMINATED
|
||||
|
|
||||
v
|
||||
ERRORED
|
||||
```
|
||||
|
||||
Sessions are created with an automation profile that governs the autonomy level for all plans within the session.
|
||||
|
||||
### References
|
||||
|
||||
- ADR-026: Agent-to-Agent Protocol (A2A)
|
||||
- ADR-047: A2A Standard Adoption
|
||||
|
||||
---
|
||||
|
||||
## Event Queue
|
||||
|
||||
The event queue provides asynchronous publish/subscribe communication between agents and plan lifecycle components.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Publisher Event Queue Subscriber
|
||||
| | |
|
||||
|-- publish(event) -----> | |
|
||||
| |-- deliver(event) ----> |
|
||||
| | |
|
||||
```
|
||||
|
||||
### Event Types
|
||||
|
||||
| Event | Description |
|
||||
|-------|-------------|
|
||||
| `PLAN_CREATED` | A new plan was created |
|
||||
| `STRATEGIZE_STARTED` | Strategy phase began |
|
||||
| `STRATEGIZE_COMPLETE` | Strategy phase completed |
|
||||
| `EXECUTE_STARTED` | Execute phase began |
|
||||
| `EXECUTE_COMPLETE` | Execute phase completed |
|
||||
| `APPLY_STARTED` | Apply phase began |
|
||||
| `APPLY_COMPLETE` | Apply phase completed |
|
||||
| `CORRECTION_APPLIED` | A decision correction was applied |
|
||||
| `INVARIANT_VIOLATED` | An invariant violation was detected |
|
||||
| `SUBPLAN_STARTED` | A subplan began execution |
|
||||
| `SUBPLAN_COMPLETE` | A subplan completed |
|
||||
| `SUBPLAN_FAILED` | A subplan failed |
|
||||
| `GUARD_TRIGGERED` | A guard condition was triggered |
|
||||
|
||||
### Subscribe to Events
|
||||
|
||||
```
|
||||
# Subscribe to all events for a plan
|
||||
agents events subscribe --plan PLAN_ID
|
||||
|
||||
# Subscribe to specific event types
|
||||
agents events subscribe --plan PLAN_ID --type SUBPLAN_COMPLETE,SUBPLAN_FAILED
|
||||
|
||||
# Subscribe with timeout
|
||||
agents events subscribe --plan PLAN_ID --timeout 300
|
||||
```
|
||||
|
||||
### Publish Custom Events
|
||||
|
||||
```
|
||||
# Publish a custom event
|
||||
agents events publish --plan PLAN_ID --type CUSTOM --data '{"key": "value"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Guard Enforcement
|
||||
|
||||
Guards protect against runaway autonomous execution by enforcing configurable limits.
|
||||
|
||||
### Guard Types
|
||||
|
||||
#### Denylist Guard
|
||||
|
||||
Prevents execution of specific tools, commands, or operations:
|
||||
|
||||
```toml
|
||||
[guards.denylist]
|
||||
tools = ["shell_exec", "file_delete"]
|
||||
commands = ["rm -rf", "sudo", "chmod 777"]
|
||||
file_patterns = ["/etc/**", "/sys/**", "~/.ssh/**"]
|
||||
```
|
||||
|
||||
#### Budget Cap Guard
|
||||
|
||||
Limits resource consumption during autonomous execution:
|
||||
|
||||
```toml
|
||||
[guards.budget]
|
||||
max_llm_tokens = 1000000 # 1M tokens per plan
|
||||
max_llm_cost_usd = 10.00 # $10 per plan
|
||||
max_wall_time_seconds = 3600 # 1 hour per plan
|
||||
max_file_writes = 1000 # 1000 file writes per plan
|
||||
```
|
||||
|
||||
#### Tool Call Limit Guard
|
||||
|
||||
Limits the number of tool calls during execution:
|
||||
|
||||
```toml
|
||||
[guards.tool_limits]
|
||||
max_tool_calls_per_subplan = 100
|
||||
max_tool_calls_per_plan = 10000
|
||||
max_consecutive_failures = 5
|
||||
```
|
||||
|
||||
### Guard Enforcement Behavior
|
||||
|
||||
When a guard is triggered:
|
||||
|
||||
1. The current subplan is halted
|
||||
2. A `GUARD_TRIGGERED` event is published
|
||||
3. The plan transitions to `ERRORED` state
|
||||
4. A detailed guard violation report is generated
|
||||
|
||||
### Guard Configuration Precedence
|
||||
|
||||
Guards can be configured at multiple levels, with more specific configurations taking precedence:
|
||||
|
||||
```
|
||||
Tool-level > Subplan-level > Plan-level > Global
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Automation Profiles
|
||||
|
||||
Automation profiles control the level of autonomy during plan execution.
|
||||
|
||||
### Built-in Profiles
|
||||
|
||||
| Profile | Description | Human Approval Required |
|
||||
|---------|-------------|------------------------|
|
||||
| `manual` | All decisions require human approval | Every decision |
|
||||
| `semi-auto` | High-risk decisions require approval | Destructive operations |
|
||||
| `auto` | Fully autonomous execution | None (guards only) |
|
||||
| `supervised` | Periodic checkpoints for human review | At checkpoints |
|
||||
|
||||
### Profile Precedence
|
||||
|
||||
Automation profiles are resolved with the following precedence (highest to lowest):
|
||||
|
||||
1. **Plan-level** — Set via `agents plan create --profile PROFILE`
|
||||
2. **Action-level** — Set via `agents action create --profile PROFILE`
|
||||
3. **Global** — Set in `.cleveragents/config.toml` under `[automation]`
|
||||
|
||||
```toml
|
||||
# Global default
|
||||
[automation]
|
||||
default_profile = "semi-auto"
|
||||
```
|
||||
|
||||
```
|
||||
# Plan-level override (highest precedence)
|
||||
agents plan create --profile auto "Port the Firefox codebase to Rust"
|
||||
```
|
||||
|
||||
### Custom Profiles
|
||||
|
||||
Custom profiles can be defined in `.cleveragents/profiles/`:
|
||||
|
||||
```toml
|
||||
# .cleveragents/profiles/strict.toml
|
||||
[profile]
|
||||
name = "strict"
|
||||
description = "Strict profile for sensitive codebases"
|
||||
base = "semi-auto"
|
||||
|
||||
[profile.overrides]
|
||||
require_approval_for = ["file_write", "git_commit", "shell_exec"]
|
||||
max_consecutive_auto_decisions = 3
|
||||
```
|
||||
|
||||
### Profile Resolution
|
||||
|
||||
The `_resolve_profile_for_plan` function in `PlanLifecycleService` resolves the effective profile for a plan. If the plan's configured profile name is not a known built-in profile, a `ValidationError` is raised with an actionable error message listing available profiles.
|
||||
|
||||
---
|
||||
|
||||
## Hierarchical Plan Decomposition
|
||||
|
||||
CleverAgents v3.5.0 supports hierarchical plan decomposition with 4+ levels of subplans.
|
||||
|
||||
### Decomposition Model
|
||||
|
||||
```
|
||||
Plan (Level 0)
|
||||
|
|
||||
+-- Subplan A (Level 1)
|
||||
| |
|
||||
| +-- Subplan A.1 (Level 2)
|
||||
| | |
|
||||
| | +-- Subplan A.1.1 (Level 3)
|
||||
| | | |
|
||||
| | | +-- Subplan A.1.1.1 (Level 4)
|
||||
| | |
|
||||
| | +-- Subplan A.1.2 (Level 3)
|
||||
| |
|
||||
| +-- Subplan A.2 (Level 2)
|
||||
|
|
||||
+-- Subplan B (Level 1)
|
||||
|
|
||||
+-- Subplan B.1 (Level 2)
|
||||
```
|
||||
|
||||
### Decomposition Strategy
|
||||
|
||||
The strategy actor decomposes plans based on:
|
||||
|
||||
1. **Dependency analysis** — Identifies independent work units that can execute in parallel
|
||||
2. **Complexity estimation** — Estimates effort for each subplan to balance load
|
||||
3. **Risk assessment** — Groups high-risk operations for sequential execution
|
||||
4. **Resource constraints** — Respects available compute and memory limits
|
||||
|
||||
### Decision Correction with Subtree Recomputation
|
||||
|
||||
When a decision correction is applied, only the affected subtree is recomputed:
|
||||
|
||||
```
|
||||
Plan
|
||||
|
|
||||
+-- Subplan A (complete)
|
||||
|
|
||||
+-- Subplan B (correction applied here)
|
||||
| |
|
||||
| +-- Subplan B.1 (RECOMPUTED)
|
||||
| |
|
||||
| +-- Subplan B.2 (RECOMPUTED)
|
||||
|
|
||||
+-- Subplan C (unaffected, not recomputed)
|
||||
```
|
||||
|
||||
This selective recomputation minimizes wasted work when corrections are needed.
|
||||
|
||||
---
|
||||
|
||||
## Parallel Execution
|
||||
|
||||
CleverAgents v3.5.0 scales parallel execution to 10+ concurrent subplans.
|
||||
|
||||
### Execution Model
|
||||
|
||||
```
|
||||
SubplanExecutionService
|
||||
|
|
||||
+-- ThreadPoolExecutor (max_workers=N)
|
||||
|
|
||||
+-- Worker 1: Subplan A.1
|
||||
+-- Worker 2: Subplan A.2
|
||||
+-- Worker 3: Subplan B.1
|
||||
+-- Worker 4: Subplan B.2
|
||||
+-- Worker 5: Subplan C.1
|
||||
...
|
||||
+-- Worker N: Subplan X.Y
|
||||
```
|
||||
|
||||
### Fail-Fast Cancellation
|
||||
|
||||
When `fail_fast=True` and a subplan fails:
|
||||
|
||||
1. The `stop_flag` is set
|
||||
2. Queued subplans are cancelled before starting
|
||||
3. In-flight subplans that complete after `stop_flag` is set are overridden to `CANCELLED`
|
||||
4. The merge output excludes cancelled subplan results
|
||||
|
||||
### Parallel Execution Configuration
|
||||
|
||||
```toml
|
||||
[execution]
|
||||
max_parallel_subplans = 10 # Maximum concurrent subplans
|
||||
fail_fast = true # Cancel remaining subplans on first failure
|
||||
subplan_timeout_seconds = 600 # 10 minutes per subplan
|
||||
```
|
||||
|
||||
### Monitoring Parallel Execution
|
||||
|
||||
```
|
||||
# Watch parallel execution in real-time
|
||||
agents plan tree PLAN_ID --watch
|
||||
|
||||
# View execution metrics
|
||||
agents plan show PLAN_ID --format json | jq '.execution_metrics'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Large-Scale Autonomous Task Execution
|
||||
|
||||
CleverAgents v3.5.0 is validated against realistic large-scale porting tasks.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
The following acceptance criteria were validated for v3.5.0:
|
||||
|
||||
- A2A facade session and plan lifecycle operations functional via CLI
|
||||
- Event queue publish/subscribe operational
|
||||
- Guard enforcement works (denylist, budget caps, tool call limits)
|
||||
- Automation profile resolution precedence correct (plan > action > global)
|
||||
- Full autonomy acceptance flow with hierarchical decomposition (4+ levels)
|
||||
- Parallel execution scales to 10+ concurrent subplans
|
||||
- A realistic porting task completes autonomously
|
||||
- `nox` passes with coverage >= 97% including large-project suites
|
||||
|
||||
### Example: Autonomous Codebase Porting
|
||||
|
||||
```
|
||||
# Create a plan for a large-scale porting task
|
||||
agents plan create --profile auto \
|
||||
"Port the Firefox rendering engine from C++ to Rust, \
|
||||
maintaining all existing test coverage and API compatibility"
|
||||
|
||||
# Link the source project
|
||||
agents plan link PLAN_ID --resource firefox-source
|
||||
|
||||
# Execute autonomously
|
||||
agents plan execute PLAN_ID
|
||||
|
||||
# Monitor progress
|
||||
agents plan tree PLAN_ID --watch
|
||||
```
|
||||
|
||||
### Performance at Scale
|
||||
|
||||
| Metric | v3.4.0 | v3.5.0 |
|
||||
|--------|--------|--------|
|
||||
| Max parallel subplans | 4 | 10+ |
|
||||
| Max decomposition depth | 2 | 4+ |
|
||||
| Decision correction scope | Full recompute | Subtree only |
|
||||
| Large project support | 10K files | 50K+ files |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Guard Violations
|
||||
|
||||
If a guard is triggered unexpectedly:
|
||||
|
||||
1. Check the guard violation report: `agents plan show PLAN_ID --format json | jq '.guard_violations'`
|
||||
2. Review guard configuration: `agents config show guards`
|
||||
3. Adjust guard limits if appropriate: `agents config set guards.budget.max_llm_tokens 2000000`
|
||||
|
||||
### Automation Profile Errors
|
||||
|
||||
If you see `ValidationError: unknown automation profile`:
|
||||
|
||||
- This was fixed in v3.5.0 (automation profile silent fallback fix)
|
||||
- Ensure you are using a built-in profile: `manual`, `semi-auto`, `auto`, or `supervised`
|
||||
- Custom profiles must be defined in `.cleveragents/profiles/`
|
||||
|
||||
### Parallel Execution Failures
|
||||
|
||||
If parallel subplans are not being cancelled on failure:
|
||||
|
||||
- This was fixed in v3.5.0 (SubplanExecutionService fail_fast cancellation fix)
|
||||
- Ensure you are running v3.5.0 or later
|
||||
- Check `fail_fast` configuration: `agents config show execution.fail_fast`
|
||||
|
||||
---
|
||||
|
||||
*See also: [A2A Protocol API Reference](api/a2a.md) | [Actor System](api/actor.md) | [ADR-047: A2A Standard Adoption](adr/ADR-047-acp-standard-adoption.md)*
|
||||
+313
@@ -0,0 +1,313 @@
|
||||
# Advanced Context Management System (ACMS)
|
||||
|
||||
> **Milestone:** v3.4.0 — ACMS v1 + Context Scaling
|
||||
|
||||
The Advanced Context Management System (ACMS) provides intelligent, budget-aware context assembly for CleverAgents. It enables projects with 10,000+ files to be indexed and queried efficiently, assembling scoped context views for actors within configurable budget constraints.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
ACMS solves the fundamental challenge of providing LLM actors with the right context at the right time, without exceeding token budgets or degrading performance on large codebases.
|
||||
|
||||
Key capabilities:
|
||||
|
||||
- **Large-scale indexing** — Projects with 10,000+ files index without timeout
|
||||
- **Budget enforcement** — Configurable `max_file_size` and `max_total_size` constraints
|
||||
- **Tiered storage** — Hot/warm/cold storage tiers manage context lifecycle
|
||||
- **Scoped assembly** — Context views are scoped per actor and per resource
|
||||
- **CLI integration** — Full `agents context` command suite
|
||||
|
||||
---
|
||||
|
||||
## Context Policies
|
||||
|
||||
Context policies define how context is assembled for a given view. Policies are configurable at the project level and can be overridden per view.
|
||||
|
||||
### Configuration
|
||||
|
||||
Context policies are stored in `.cleveragents/config.toml` under the `[context]` section:
|
||||
|
||||
```toml
|
||||
[context]
|
||||
# Maximum size of a single file to include in context (bytes)
|
||||
max_file_size = 262144 # 256 KB default
|
||||
|
||||
# Maximum total size of all context fragments combined (bytes)
|
||||
max_total_size = 10485760 # 10 MB default
|
||||
|
||||
# Storage tier thresholds
|
||||
hot_tier_max_fragments = 50
|
||||
warm_tier_max_fragments = 200
|
||||
cold_tier_max_fragments = 2000
|
||||
|
||||
# File patterns to exclude from indexing
|
||||
exclude_patterns = [
|
||||
".git/**",
|
||||
"node_modules/**",
|
||||
"__pycache__/**",
|
||||
"*.pyc",
|
||||
"*.bin",
|
||||
"*.exe",
|
||||
]
|
||||
```
|
||||
|
||||
### View-Specific Settings
|
||||
|
||||
Individual context views can override global policy settings:
|
||||
|
||||
```toml
|
||||
[context.views.execute]
|
||||
max_file_size = 131072 # 128 KB for execute view
|
||||
max_total_size = 5242880 # 5 MB for execute view
|
||||
|
||||
[context.views.strategize]
|
||||
max_file_size = 524288 # 512 KB for strategize view
|
||||
max_total_size = 20971520 # 20 MB for strategize view
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Budget Enforcement
|
||||
|
||||
ACMS enforces two budget constraints during context assembly:
|
||||
|
||||
### `max_file_size`
|
||||
|
||||
Controls the maximum size of any single file included in a context fragment. Files exceeding this limit are excluded from context assembly.
|
||||
|
||||
- **Default:** 256 KB (`262144` bytes)
|
||||
- **Purpose:** Prevents single large files from consuming the entire context budget
|
||||
- **Behavior:** Files exceeding the limit are logged at `DEBUG` level and skipped
|
||||
|
||||
### `max_total_size`
|
||||
|
||||
Controls the maximum combined size of all context fragments in a single assembled view.
|
||||
|
||||
- **Default:** 10 MB (`10485760` bytes)
|
||||
- **Purpose:** Ensures the assembled context fits within LLM token budgets
|
||||
- **Behavior:** Once the total budget is reached, lower-priority fragments are excluded
|
||||
|
||||
### Binary File Exclusion
|
||||
|
||||
Binary files are automatically excluded from context assembly regardless of size settings. Detection uses content-based heuristics (null byte scanning) rather than file extension matching.
|
||||
|
||||
---
|
||||
|
||||
## CLI Commands
|
||||
|
||||
The `agents context` command suite provides full control over context management.
|
||||
|
||||
### `agents context list`
|
||||
|
||||
Lists all context fragments currently in the tier service, grouped by storage tier.
|
||||
|
||||
```
|
||||
agents context list [--plan PLAN_ID] [--format {table,json,plain}]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--plan PLAN_ID` | Filter fragments by plan ID |
|
||||
| `--format` | Output format (default: `table`) |
|
||||
|
||||
### `agents context add`
|
||||
|
||||
Manually adds a file or directory to the context tier service.
|
||||
|
||||
```
|
||||
agents context add PATH [--tier {hot,warm,cold}] [--plan PLAN_ID]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `PATH` | File or directory path to add |
|
||||
| `--tier` | Target storage tier (default: `warm`) |
|
||||
| `--plan PLAN_ID` | Associate fragment with a specific plan |
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
# Add a single file to hot tier
|
||||
agents context add src/critical_module.py --tier hot
|
||||
|
||||
# Add a directory to warm tier
|
||||
agents context add src/utils/ --tier warm --plan 01HXYZ...
|
||||
```
|
||||
|
||||
### `agents context show`
|
||||
|
||||
Displays detailed information about a specific context fragment or the assembled context view for a plan.
|
||||
|
||||
```
|
||||
agents context show [FRAGMENT_ID | --plan PLAN_ID] [--format {table,json,plain}]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `FRAGMENT_ID` | Show details for a specific fragment |
|
||||
| `--plan PLAN_ID` | Show assembled context view for a plan |
|
||||
| `--format` | Output format (default: `table`) |
|
||||
|
||||
### `agents context clear`
|
||||
|
||||
Removes context fragments from the tier service.
|
||||
|
||||
```
|
||||
agents context clear [--tier {hot,warm,cold,all}] [--plan PLAN_ID] [--confirm]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--tier` | Clear only the specified tier (default: `all`) |
|
||||
| `--plan PLAN_ID` | Clear only fragments for a specific plan |
|
||||
| `--confirm` | Skip confirmation prompt |
|
||||
|
||||
---
|
||||
|
||||
## Hot/Warm/Cold Storage Tiers
|
||||
|
||||
ACMS uses a three-tier storage model to manage context lifecycle efficiently.
|
||||
|
||||
### Tier Definitions
|
||||
|
||||
| Tier | Description | Access Speed | Eviction Policy |
|
||||
|------|-------------|--------------|-----------------|
|
||||
| **Hot** | Actively used fragments for the current actor | Immediate | LRU when capacity exceeded |
|
||||
| **Warm** | Recently accessed fragments, likely to be reused | Fast | LRU + staleness-based |
|
||||
| **Cold** | Indexed but infrequently accessed fragments | Slower | Age-based + LRU |
|
||||
|
||||
### Tier Promotion and Demotion
|
||||
|
||||
Fragments move between tiers automatically based on access patterns:
|
||||
|
||||
- **Promotion:** A fragment accessed from `warm` or `cold` is promoted toward `hot`
|
||||
- **Demotion:** Fragments not accessed within the staleness window are demoted
|
||||
- **Eviction:** When a tier reaches capacity, the least-recently-used fragment is evicted to the next tier or removed
|
||||
|
||||
### Thread Safety
|
||||
|
||||
`ContextTierService` uses a reentrant lock (`threading.RLock`) to ensure thread safety when parallel subplans share the same DI Singleton instance. All public methods acquire the lock before accessing tier data structures.
|
||||
|
||||
---
|
||||
|
||||
## Context Assembly Pipeline
|
||||
|
||||
The context assembly pipeline transforms raw file content into scoped, budget-constrained context views for LLM actors.
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
1. **Hydration** — `git ls-files` / `os.walk` enumerates project files into `TieredFragment` objects
|
||||
2. **Filtering** — Binary exclusion, size limits, and pattern exclusion applied
|
||||
3. **Tiering** — Fragments assigned to hot/warm/cold based on relevance score
|
||||
4. **Assembly** — Budget-constrained selection with scope application produces the final context view
|
||||
|
||||
### Context Hydration
|
||||
|
||||
The `context_tier_hydrator.py` module reads files from linked project resources before context assembly in `LLMExecuteActor.execute()`. It:
|
||||
|
||||
1. Resolves project resource paths via `plan.project_links`
|
||||
2. Enumerates files using `git ls-files` (for git projects) or `os.walk` (for non-git projects)
|
||||
3. Applies size and binary filters
|
||||
4. Creates `TieredFragment` objects with metadata
|
||||
5. Stores fragments in `ContextTierService`
|
||||
|
||||
### Scoped Context Views
|
||||
|
||||
Context views are scoped per actor and per resource to prevent cross-contamination between parallel subplans.
|
||||
|
||||
---
|
||||
|
||||
## Large Project Indexing
|
||||
|
||||
ACMS is designed to handle projects with 10,000+ files without timeout.
|
||||
|
||||
### Performance Characteristics
|
||||
|
||||
| Project Size | Indexing Time | Memory Usage |
|
||||
|-------------|---------------|--------------|
|
||||
| 1,000 files | < 1 second | ~50 MB |
|
||||
| 10,000 files | < 10 seconds | ~200 MB |
|
||||
| 50,000 files | < 60 seconds | ~800 MB |
|
||||
|
||||
### Optimization Strategies
|
||||
|
||||
1. **Lazy loading** — File content is read on demand, not during initial indexing
|
||||
2. **Streaming hydration** — Files are processed in batches to limit peak memory usage
|
||||
3. **Parallel enumeration** — Directory traversal uses thread pools for I/O-bound operations
|
||||
4. **Incremental updates** — Only changed files (via `git diff`) are re-indexed on subsequent runs
|
||||
|
||||
---
|
||||
|
||||
## Integration with Plan Execution
|
||||
|
||||
ACMS integrates directly with the plan execution pipeline to provide LLM actors with relevant context.
|
||||
|
||||
### Automatic Context Injection
|
||||
|
||||
When `agents plan execute` runs, ACMS automatically:
|
||||
|
||||
1. Hydrates the context tier from project resources
|
||||
2. Assembles a scoped context view for the execute actor
|
||||
3. Injects the context view into the LLM prompt
|
||||
4. Tracks which fragments were used for observability
|
||||
|
||||
### Context in Strategy Phase
|
||||
|
||||
During `agents plan strategize`, ACMS provides a broader context view to help the strategy actor understand the full project structure before decomposing the plan.
|
||||
|
||||
### Context in Apply Phase
|
||||
|
||||
During `agents plan apply`, ACMS provides targeted context for validation actors to verify that applied changes are consistent with the project.
|
||||
|
||||
### Configuration Example
|
||||
|
||||
```toml
|
||||
[actor.execute]
|
||||
context_view = "execute"
|
||||
context_budget_bytes = 5242880
|
||||
|
||||
[actor.strategize]
|
||||
context_view = "strategize"
|
||||
context_budget_bytes = 20971520
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Context is Empty
|
||||
|
||||
If the LLM receives no file context, check:
|
||||
|
||||
1. **Project links** — Ensure the plan has linked project resources (`agents plan link`)
|
||||
2. **File permissions** — Ensure the agent process can read project files
|
||||
3. **Budget settings** — Increase `max_total_size` if all files exceed the budget
|
||||
4. **Exclusion patterns** — Check that source files are not matched by exclusion patterns
|
||||
|
||||
### Indexing Timeout
|
||||
|
||||
For very large projects:
|
||||
|
||||
1. Increase the hydration timeout in config: `context.hydration_timeout_seconds = 120`
|
||||
2. Add more specific exclusion patterns to reduce the file count
|
||||
3. Use `agents context add` to manually add only the most relevant files
|
||||
|
||||
### Thread Safety Errors
|
||||
|
||||
If you see `RuntimeError: dictionary changed size during iteration`:
|
||||
|
||||
- This was fixed in v3.4.0 (see `ContextTierService` thread safety fix)
|
||||
- Ensure you are running v3.4.0 or later
|
||||
|
||||
---
|
||||
|
||||
*See also: [ACMS / UKO API Reference](api/acms.md) | [Context Hydration Module](modules/context-hydration.md)*
|
||||
@@ -45,6 +45,8 @@ nav:
|
||||
- Automation Tracking: development/automation-tracking.md
|
||||
- Custom Sandbox Strategy: development/custom_sandbox_strategy.md
|
||||
- Documentation Writer: development/docs-writer.md
|
||||
- Context Management (ACMS): context.md
|
||||
- Autonomy Hardening: autonomy.md
|
||||
- Implementation Timeline: timeline.md
|
||||
- FAQ: faq.md
|
||||
- Reference: reference/
|
||||
|
||||
Reference in New Issue
Block a user