cleveragents-core/docs/adr/ADR-014-context-management-acms.md

adr_number, title, status_history, tier, authors, superseded_by, related_adrs, acceptance

adr_number	title	status_history	tier	authors	superseded_by	related_adrs	acceptance
14	Context Management (ACMS)	2026-02-16 Proposed Jeffrey Phillips Freeman 2026-02-16 Accepted Jeffrey Phillips Freeman	3	Jeffrey Phillips Freeman	null	number title relationship 6 Plan Lifecycle Context is assembled and refreshed at each plan phase transition number title relationship 8 Resource System Resources are indexed by the Unified Knowledge Ontology for context retrieval number title relationship 9 Project Model Projects configure context retrieval strategies and index scopes number title relationship 10 Actor and Agent Architecture Actors consume assembled context to inform their decisions	votes_for votes_against abstentions voter comment Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> A pipeline-based context assembly system keeps retrieval strategies composable and tunable per project

Context

LLM context windows are finite and expensive. Plans operating on large codebases cannot load everything into context — they must selectively retrieve the most relevant information. The system needs a context management mechanism that can index project resources into a queryable knowledge base, retrieve relevant fragments using multiple strategies (keyword, semantic, graph), assemble those fragments within a token budget, and present them coherently to the actor.

A naive approach (load everything, or load files by name) fails at scale. The system needs a principled architecture for context retrieval, scoring, deduplication, budget packing, and tiered retention.

Decision Drivers

• LLM context windows are finite and expensive; naive "load everything" fails at scale
• Multiple retrieval strategies (keyword, semantic, graph) capture complementary facets of relevance
• Context assembly must respect a hard token budget derived from the model's context window
• Fragments need progressive detail depth so budget-constrained assemblies degrade gracefully rather than omitting content entirely
• Context must be refreshed at each plan phase transition with up-to-date retrieval results
• The pipeline must be pluggable so projects can customize retrieval strategies and scoring weights

Decision

CleverAgents implements the Adaptive Context Management System (ACMS), built on two foundations: the Unified Knowledge Ontology (UKO) for indexing and the Context Retrieval Pipeline (CRP) for assembly. The ACMS uses a 10-component pipeline that transforms raw retrieval results into a budget-constrained, coherently ordered context window.

Design

Unified Knowledge Ontology (UKO)

UKO is the indexing layer that transforms project resources into queryable knowledge nodes. Language-specific analyzers (Python, TypeScript, Rust, Java, Markdown, JSON Schema, and custom) extract structural elements (functions, classes, modules, types) into a hierarchy of UKO nodes with multiple detail depths.

Detail depths enable progressive disclosure — a function can be represented as just its signature (depth 0), signature + docstring (depth 2), signature + body skeleton (depth 4), or full source (depth 9). When context budget is tight, fragments are downgraded to shallower depths.

Context Tiers

Hot context: The current working set loaded into the LLM context window. Budget controlled by context.hot.max-tokens (default: 16,000).

Warm context: Recent decisions and fragments available for retrieval but not loaded by default. Retained for context.tiers.warm.retention-hours (default: 24 hours). Max decisions: context.warm.max-decisions (default: 100).

Cold context: Historical decisions and fragments archived for audit and correction. Retained for context.tiers.cold.retention-days (default: 90 days). Max decisions: context.cold.max-decisions (default: 500).

Context Retrieval Strategies

Multiple strategies run in parallel and their results are fused:

• simple-keyword: Full-text search via Tantivy or SQLite FTS5.
• semantic-embedding: Vector similarity search via FAISS or Qdrant.
• breadth-depth-navigator: Graph traversal over structural relationships (max hops: context.strategies.breadth-depth-navigator.max-hops, default: 4).
• ARCE (Autonomous Reasoning Context Extraction): LLM-driven iterative search-refine loop (max rounds: context.strategies.arce.max-rounds, default: 3).
• Custom strategies: Registered via context.strategies.custom.* with module, class, max_quality, and optional config.

Enabled strategies are configured via context.strategies.enabled (default: ["simple-keyword", "semantic-embedding", "breadth-depth-navigator"]).

10-Component Context Assembly Pipeline

1. StrategySelector: Decides which strategies to invoke and with what confidence.
2. BudgetAllocator: Distributes the token budget across selected strategies.
3. StrategyExecutor: Runs strategies in parallel with timeouts (default: 30s), circuit breakers (threshold: 3 failures), and configurable max workers (default: 4).
4. FragmentDeduplicator: Removes duplicate fragments via content-hash, UKO-identity, or semantic similarity.
5. DetailDepthResolver: Resolves conflicts when the same UKO node appears at different detail depths (keeps the most appropriate depth for the budget).
6. FragmentScorer: Computes composite relevance scores using weighted factors — relevance (0.4), hierarchy position (0.3), strategy quality (0.2), recency (0.1).
7. BudgetPacker: Fits scored fragments into the token budget using a greedy knapsack algorithm with depth fallback (steps: [9, 4, 2, 0]). Minimum fragment size: 10 tokens.
8. FragmentOrderer: Orders packed fragments for optimal coherence in the context window.
9. PreambleGenerator: Generates provenance summaries prepended to assembled context (max tokens: 200).
10. SkeletonCompressor: Compresses parent plan context into skeleton form for child plan inheritance (budget ratio: context.budget.skeleton-ratio, default: 0.15).

Each component is pluggable via context.pipeline.* configuration keys specifying "module:ClassName".

Context Budget Calculation

The context budget is derived from the actor's model context window:

budget = model_context_window - response_reserve (4096) - tool_definitions - skeleton_allocation

Budget refresh is triggered when the available budget changes by more than context.budget.refresh-threshold (default: 0.30). If the computed budget falls below context.budget.min-useful-budget (default: 500 tokens), context assembly is skipped.

Child Plan Context Inheritance

Child plans inherit a compressed "skeleton" of their parent's context. The SkeletonCompressor reduces parent context to the most essential structural information within the skeleton budget ratio (default: 15% of the child's budget).

Constraints

• Context assembly must operate within the actor's model context window. The token budget is a hard ceiling, not a guideline.
• Retrieval results below context.query.min-relevance (default: 0.3) are discarded even if the budget is not filled.
• Files exceeding context.file.max-size (default: 1 MB) are summarized or excluded.
• Total file size across all context fragments must not exceed context.file.max-total-size (default: 50 MB).
• Strategy executor timeout (default: 30s per strategy) and circuit breaker (default: 3 failures) prevent slow or failing strategies from blocking assembly.
• All 10 pipeline components must execute for every context assembly. Components may produce empty output but must not be skipped.

Consequences

Positive

• Multiple retrieval strategies (keyword, semantic, graph, LLM-driven) capture different facets of relevance.
• The 10-component pipeline provides fine-grained control over every aspect of context assembly.
• Depth fallback enables graceful degradation when the budget is tight — fragments are shown at shallower depth rather than omitted entirely.
• Pluggable components enable customization without modifying the core pipeline.
• Tiered retention (hot/warm/cold) balances immediate utility with historical availability.

Negative

• The 10-component pipeline is complex and adds latency to every plan phase.
• Multiple configurable components create a large configuration surface area.
• Strategy quality depends on index quality — stale or incomplete indexes produce poor context.

Risks

• Token budget estimation errors could cause context to exceed the model's actual window, producing truncation or errors.
• The greedy knapsack packing is not optimal — it may exclude high-value fragments that would fit with different packing.
• Circuit breakers on strategies could permanently disable a strategy if transient failures trigger the threshold.

Alternatives Considered

Simple file-based context (load files by name) — Insufficient at scale. Does not handle large codebases, provides no relevance ranking, and wastes context window on irrelevant content.

Single retrieval strategy (e.g., semantic search only) — Missing the complementary strengths of keyword search (exact matches, short identifiers) and graph traversal (structural relationships, call chains).

Compliance

• Budget constraint tests: Tests verify that assembled context never exceeds the declared token budget.
• Pipeline component tests: Each of the 10 components has unit tests verifying correct behavior for edge cases (empty input, budget exhaustion, duplicate fragments).
• Strategy integration tests: Tests verify that each retrieval strategy returns correctly structured results.
• Depth fallback tests: Tests verify that fragments are downgraded to shallower depths when budget is insufficient for full depth.
• Configuration tests: Tests verify that custom pipeline components and strategy registrations are loaded and invoked correctly.