spec(acms): add comprehensive ACMS v1 specification section (v3.4.0)
CI / push-validation (push) Successful in 30s
CI / helm (push) Successful in 33s
CI / lint (push) Successful in 1m8s
CI / build (push) Successful in 1m0s
CI / quality (push) Successful in 1m39s
CI / typecheck (push) Successful in 1m44s
CI / security (push) Successful in 1m47s
CI / benchmark-publish (push) Failing after 41s
CI / e2e_tests (push) Successful in 4m4s
CI / integration_tests (push) Successful in 4m17s
CI / unit_tests (push) Successful in 4m59s
CI / docker (push) Successful in 1m30s
CI / coverage (push) Successful in 10m50s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 1m0s
CI / quality (pull_request) Successful in 1m16s
CI / build (pull_request) Successful in 50s
CI / typecheck (pull_request) Successful in 1m36s
CI / security (pull_request) Successful in 1m41s
CI / push-validation (pull_request) Successful in 36s
CI / helm (pull_request) Successful in 38s
CI / integration_tests (pull_request) Successful in 3m54s
CI / e2e_tests (pull_request) Successful in 4m12s
CI / unit_tests (pull_request) Successful in 4m58s
CI / docker (pull_request) Successful in 1m26s
CI / coverage (pull_request) Successful in 12m34s
CI / status-check (pull_request) Successful in 3s
CI / push-validation (push) Successful in 30s
CI / helm (push) Successful in 33s
CI / lint (push) Successful in 1m8s
CI / build (push) Successful in 1m0s
CI / quality (push) Successful in 1m39s
CI / typecheck (push) Successful in 1m44s
CI / security (push) Successful in 1m47s
CI / benchmark-publish (push) Failing after 41s
CI / e2e_tests (push) Successful in 4m4s
CI / integration_tests (push) Successful in 4m17s
CI / unit_tests (push) Successful in 4m59s
CI / docker (push) Successful in 1m30s
CI / coverage (push) Successful in 10m50s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 1m0s
CI / quality (pull_request) Successful in 1m16s
CI / build (pull_request) Successful in 50s
CI / typecheck (pull_request) Successful in 1m36s
CI / security (pull_request) Successful in 1m41s
CI / push-validation (pull_request) Successful in 36s
CI / helm (pull_request) Successful in 38s
CI / integration_tests (pull_request) Successful in 3m54s
CI / e2e_tests (pull_request) Successful in 4m12s
CI / unit_tests (pull_request) Successful in 4m58s
CI / docker (pull_request) Successful in 1m26s
CI / coverage (pull_request) Successful in 12m34s
CI / status-check (pull_request) Successful in 3s
Adds a comprehensive ACMS v1 (Advanced Context Management System) specification section covering all aspects of the context management architecture for milestone v3.4.0 — ACMS v1 + Context Scaling. Sections included: - Overview and design goals - Architecture: UKO (Unified Knowledge Ontology), CRP (Context Request Protocol), Context Pipeline, Storage Tiers (hot/warm/cold) - Module Definitions across all 4 architecture layers (Domain, Application, Infrastructure, Presentation) - Data Models: ContextFragment, ContextBudget, ProjectContextPolicy, ContextView, AssembledContext/ContextPayload, IndexMetadata - Key Interfaces: ACMSPipeline, ContextBudgetEnforcer, UKOStore, ContextTierService, RepoIndexingService - 10-Component Pipeline Architecture (3 phases: Strategy Orchestration, Fragment Fusion, Context Finalization) - Context Strategies: 6 built-in strategies with fallback degradation path - CLI Commands: agents context list/add/show/clear and agents project context set/show/inspect/simulate - Integration Points: Plan Lifecycle, Resource System, Actor/Agent, Configuration, DI Container - Database Schema: repo_indexes and indexed_files tables - v1 Known Limitations with planned conformance path - Acceptance Criteria for milestone v3.4.0 Replaces stuck [AUTO-ARCH-19] worker from Cycle 26. [AUTO-ARCH-20] Cycle 27 # Conflicts: # docs/specification.md
This commit was merged in pull request #9856.
This commit is contained in:
+698
-148
@@ -89,7 +89,7 @@ The following standards are integrated into the architecture:
|
||||
: A persisted choice point in a plan's decision tree, created during Strategize or Execute. Records the question, chosen option, alternatives, confidence score, rationale, context snapshot, and downstream dependencies. Types: `prompt_definition`, `invariant_enforced`, `strategy_choice`, `subplan_spawn`, `subplan_parallel_spawn`, among others. Supports targeted correction with selective subtree recomputation.
|
||||
|
||||
Invariant
|
||||
: A natural-language constraint on plan execution scoped to global, project, action, or plan level. The runtime precedence chain is four-tier: ==plan > action > project > global==. Exception: global invariants marked `non_overridable` always win regardless of scope. Reconciled by the Invariant Reconciliation Actor at each phase boundary (before Strategize, Execute, and Apply, and after Apply); recorded as `invariant_enforced` decisions that propagate to child plans.
|
||||
: A natural-language constraint on plan execution scoped to global, project, action, or plan level. The runtime precedence chain is four-tier: ==plan > action > project > global==. Exception: global invariants marked `non_overridable` always win regardless of scope. Reconciled by the Invariant Reconciliation Actor at the start of Strategize; recorded as `invariant_enforced` decisions that propagate to child plans.
|
||||
|
||||
Automation Profile
|
||||
: A named set of confidence thresholds (each `0.0`–`1.0`) gating which plan operations proceed automatically versus requiring human approval. `0.0` = always automatic; `1.0` = always manual. Eight built-in profiles (`manual` through `full-auto`). Custom profiles namespaced as `[[server:]namespace/]name`. Each profile composes a **Safety Profile** that controls hard safety constraints (sandbox, checkpoint, unsafe-tool gating, skill restrictions, cost/retry limits).
|
||||
@@ -18393,17 +18393,6 @@ List invariants at a given scope. Use `--effective` with `--plan` to show the fi
|
||||
!!! adr "Architecture Decision"
|
||||
The plan lifecycle, phase transitions, and plan hierarchy are defined in [ADR-006: Plan Lifecycle](adr/ADR-006-plan-lifecycle.md).
|
||||
|
||||
!!! note "Multi-Phase Invariant Enforcement"
|
||||
|
||||
Invariant reconciliation is enforced at **every phase boundary**, not just at the start of Strategize. The Invariant Reconciliation Actor runs:
|
||||
|
||||
- **Before Strategize** -- establishes the effective invariant view for strategy generation
|
||||
- **Before Execute** -- re-validates invariants against the finalized strategy decisions
|
||||
- **Before Apply** -- confirms invariants still hold before committing sandbox changes
|
||||
- **After Apply** -- records final invariant state in the decision tree
|
||||
|
||||
Each enforcement pass records `invariant_enforced` decisions that propagate to child plans, ensuring invariant constraints remain consistent throughout the full plan lifecycle.
|
||||
|
||||
A **plan** is the fundamental unit of orchestration and traceability.
|
||||
|
||||
#### Plan Lifecycle Phases
|
||||
@@ -45743,45 +45732,6 @@ When advanced features are unavailable, the system gracefully degrades. The pipe
|
||||
3. Try `semantic-embedding` (requires vector) -> if unavailable:
|
||||
4. Fall back to `simple-keyword` (requires only text search / ripgrep)
|
||||
|
||||
#### ACMS Thread Safety
|
||||
|
||||
The ACMS context management system is designed for concurrent access by multiple actors and plan phases. Thread safety is enforced at multiple levels using Python's `threading.RLock` (reentrant lock), which allows the same thread to acquire the lock multiple times without deadlocking -- essential for recursive context assembly operations.
|
||||
|
||||
**Concurrency Contract:**
|
||||
|
||||
| Component | Thread Safety Guarantee |
|
||||
| :-------- | :---------------------- |
|
||||
| `ContextAssemblyPipeline` | Fully thread-safe; each `assemble()` call acquires a per-pipeline RLock for the duration of the assembly session |
|
||||
| `HotContextStore` | Thread-safe reads and writes via RLock; concurrent actors receive isolated views |
|
||||
| `WarmContextStore` | Thread-safe; RLock guards decision index updates and eviction |
|
||||
| `ColdContextStore` | Thread-safe; RLock guards archive compaction and retrieval |
|
||||
| `StrategyExecutor` | Strategies execute in parallel (thread pool); results are merged under RLock |
|
||||
| `FusionCoordinator` | Thread-safe; RLock guards fragment deduplication and ranking |
|
||||
| `PerActorContextView` | Each actor view is isolated; no cross-actor locking required |
|
||||
|
||||
**RLock Usage Pattern:**
|
||||
|
||||
```python
|
||||
class ContextAssemblyPipeline:
|
||||
def __init__(self):
|
||||
self._lock = threading.RLock()
|
||||
|
||||
def assemble(self, request: ContextRequest) -> AssembledContext:
|
||||
with self._lock:
|
||||
# Strategy execution runs in thread pool (releases GIL)
|
||||
fragments = self._strategy_executor.run_parallel(request)
|
||||
# Fusion and finalization under lock
|
||||
fused = self._fusion_coordinator.fuse(fragments)
|
||||
return self._finalizer.finalize(fused, request)
|
||||
```
|
||||
|
||||
**Key Invariants:**
|
||||
|
||||
- No actor can observe a partially-assembled context; assembly is atomic from the caller's perspective.
|
||||
- Hot context writes from one actor never corrupt another actor's view; per-actor isolation is enforced at the store level.
|
||||
- Strategy execution (the most expensive phase) runs concurrently in a thread pool; only the merge/finalization step requires the pipeline lock.
|
||||
- RLock re-entrancy supports recursive plan hierarchies where a parent plan's context assembly triggers child plan context requests within the same thread.
|
||||
|
||||
#### ACMS Performance Characteristics
|
||||
|
||||
##### Assembly Latency
|
||||
@@ -47372,150 +47322,750 @@ These architectural invariants must be maintained across all milestones:
|
||||
9. **File size limit**: No source file exceeds 500 lines. Split into modules if approaching limit.
|
||||
10. **Atomic commits**: One logical change per commit. No mixed concerns.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## M7: Advanced Concepts and Deferred Features (v3.6.0)
|
||||
|
||||
!!! info "Milestone Overview"
|
||||
Milestone 7 (v3.6.0) introduces advanced platform capabilities that build on the stable foundation established in M1-M6. These features address deferred complexity, close architectural gaps identified during earlier milestones, and prepare the platform for enterprise-scale deployments.
|
||||
## ACMS v1 — Advanced Context Management System (Milestone v3.4.0)
|
||||
|
||||
### Overview
|
||||
|
||||
v3.6.0 delivers four major capability clusters:
|
||||
The **Advanced Context Management System (ACMS) v1** is the context intelligence layer of
|
||||
CleverAgents. It provides a principled, scalable mechanism for indexing project knowledge,
|
||||
assembling budget-constrained context views, and delivering scoped context payloads to actors
|
||||
for LLM calls. ACMS v1 is the primary deliverable of milestone **v3.4.0 — ACMS v1 + Context
|
||||
Scaling**.
|
||||
|
||||
1. **Advanced Invariant Lifecycle** - multi-phase enforcement, versioned invariant snapshots, and conflict audit trails
|
||||
2. **ACMS Observability and Tuning** - context assembly telemetry, per-strategy latency budgets, and adaptive tier promotion
|
||||
3. **Plan Hierarchy Enhancements** - cross-plan invariant propagation, sibling plan coordination, and plan-level resource locking
|
||||
4. **CLI Communication Pattern Migration** - formal migration path from direct imports to A2A for all CLI commands
|
||||
ACMS is built on two foundational subsystems:
|
||||
|
||||
- **UKO (Unified Knowledge Ontology)** — a layered RDF/Turtle ontology that indexes project
|
||||
files, symbols, relationships, and semantic concepts into a queryable knowledge graph.
|
||||
- **CRP (Context Request Protocol)** — a structured protocol through which actors declare
|
||||
what information they need, at what level of detail, and with what scope, triggering a
|
||||
context assembly cycle in the ACMS pipeline.
|
||||
|
||||
Together, UKO and CRP power a **10-component context assembly pipeline** that transforms raw
|
||||
retrieval results into a budget-constrained, coherently ordered context window for each actor.
|
||||
|
||||
**Design goals for ACMS v1:**
|
||||
|
||||
- Support projects with 10,000+ files without timeout or memory exhaustion.
|
||||
- Respect hard token budget ceilings derived from the actor's model context window.
|
||||
- Enable multiple complementary retrieval strategies (keyword, semantic, graph, LLM-driven).
|
||||
- Provide tiered context lifecycle management (hot/warm/cold) with per-actor visibility.
|
||||
- Expose a clean CLI interface for context inspection, simulation, and policy management.
|
||||
|
||||
---
|
||||
|
||||
### Architecture
|
||||
|
||||
#### UKO — Unified Knowledge Ontology
|
||||
|
||||
UKO is the indexing layer. It transforms project resources into a queryable hierarchy of
|
||||
knowledge nodes, each with multiple detail depths enabling progressive disclosure.
|
||||
|
||||
**Layer structure:**
|
||||
|
||||
| Layer | Prefix(es) | IRI Namespace | Purpose |
|
||||
|-------|------------|---------------|---------|
|
||||
| 0 | `uko:` | `https://cleveragents.ai/ontology/uko#` | Universal foundation — base information unit types and core relationships |
|
||||
| 1 | `uko-code:`, `uko-doc:`, `uko-data:`, `uko-infra:` | `https://cleveragents.ai/ontology/uko/code#` etc. | General software — modules, callables, types, tests, imports |
|
||||
| 2 | `uko-oo:`, `uko-func:`, `uko-proc:` | `https://cleveragents.ai/ontology/uko/oo#` etc. | Paradigm-specific — OO classes, interfaces, methods, attributes |
|
||||
| 3 | `uko-py:`, `uko-ts:`, `uko-rs:`, `uko-java:` | `https://cleveragents.ai/ontology/uko/py#` etc. | Technology-specific — DetailLevelMap insertions |
|
||||
|
||||
**Layer 0 root types:**
|
||||
|
||||
- `InformationUnit` — root of every UKO node; anything that can appear in an actor's context.
|
||||
- `Container` — an information unit that contains other units (file, module, class, section).
|
||||
- `Atom` — a leaf-level information unit (function body, paragraph, config value).
|
||||
- `Annotation` — metadata attached to another unit (comment, docstring, attribute).
|
||||
- `Boundary` — an interface point between containers (export, API endpoint, public method signature).
|
||||
|
||||
**Core relationship properties:**
|
||||
|
||||
- `contains` — parent contains child.
|
||||
- `references` — weak reference (e.g. a function mentions a type in a docstring).
|
||||
- `dependsOn` — strong dependency (import, inheritance, call).
|
||||
|
||||
**Detail depths** enable progressive disclosure:
|
||||
|
||||
| Depth | Question Answered | What Gets Included |
|
||||
|-------|-------------------|--------------------|
|
||||
| 0 | *What exists?* | Name/identifier only |
|
||||
| 1 | *How is it organized?* | Names of immediate children |
|
||||
| 2 | *What are the key relationships?* | Children + dependency/reference edges |
|
||||
| 3 | *What is each thing's purpose?* | + Short descriptions/summaries |
|
||||
| 4 | *What is the structural shape?* | + Type info, size/count metadata |
|
||||
| 5-8 | *How does it work?* | Progressively more content |
|
||||
| 9 | *Everything.* | Complete content — nothing omitted |
|
||||
|
||||
When context budget is tight, fragments are downgraded to shallower depths rather than omitted
|
||||
entirely (depth fallback).
|
||||
|
||||
**Provenance properties** on every UKO node:
|
||||
|
||||
- `sourceResource` — the CleverAgents Resource (by ULID) this node was extracted from.
|
||||
- `sourcePath` — file path within the resource.
|
||||
- `sourceRange` — byte or line range within the source file (e.g. `42:1-87:0`).
|
||||
|
||||
**Temporal properties** support versioned knowledge:
|
||||
|
||||
- `validFrom`, `validUntil`, `isCurrent`, `isRevisionOf`.
|
||||
|
||||
#### CRP — Context Request Protocol
|
||||
|
||||
CRP is the structured vocabulary through which actors declare context needs. Each
|
||||
`ContextRequest` triggers a context assembly cycle in the ACMS pipeline.
|
||||
|
||||
**ContextRequest fields:**
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `query` | `str \| None` | `None` | Natural language query |
|
||||
| `entities` | `list[str]` | `[]` | Named entities to focus on |
|
||||
| `uko_types` | `list[str]` | `[]` | UKO types to filter |
|
||||
| `focus` | `list[str]` | `[]` | URIs or identifiers to focus on |
|
||||
| `breadth` | `int` | `2` | Dependency hops outward (>= 0) |
|
||||
| `depth` | `int \| str` | `3` | Detail depth — integer or named level |
|
||||
| `depth_gradient` | `bool` | `True` | Items closer to focus get more detail |
|
||||
| `temporal` | `TemporalScope` | `CURRENT` | Temporal scope for retrieval |
|
||||
| `max_tokens` | `int \| None` | `None` | Maximum token budget (>= 0) |
|
||||
| `preferred_strategies` | `list[str]` | `[]` | Preferred context strategies |
|
||||
| `required_backends` | `list[str]` | `[]` | Required data backends |
|
||||
| `priority` | `float` | `0.5` | 0.0 = background, 1.0 = critical |
|
||||
| `purpose` | `str` | `""` | Why this context is needed |
|
||||
|
||||
**DetailLevelMap** maps named detail levels to integer depths for a UKO domain. Inheritance
|
||||
allows child maps to include all entries from their parent map and insert additional levels.
|
||||
|
||||
Built-in named levels (universal Layer 0):
|
||||
|
||||
| Name | Depth |
|
||||
|------|-------|
|
||||
| `MODULE_LISTING` | 0 |
|
||||
| `SIGNATURES` | 4 |
|
||||
| `FULL_SOURCE` | 9 |
|
||||
|
||||
#### Context Pipeline
|
||||
|
||||
The ACMS pipeline transforms a `ContextRequest` into an `AssembledContext` through five
|
||||
high-level stages:
|
||||
|
||||
1. **Indexing** — Project files are indexed into the UKO ontology by `RepoIndexingService`.
|
||||
Language-specific analyzers extract structural elements (functions, classes, modules, types)
|
||||
into UKO nodes with multiple detail depths. The index is persisted in SQLite and refreshed
|
||||
incrementally on file changes.
|
||||
|
||||
2. **Assembly** — A `ContextRequest` triggers the 10-component assembly pipeline.
|
||||
Strategies run in parallel; their results are fused, deduplicated, scored, and packed.
|
||||
|
||||
3. **Scoping** — Context is filtered by the project's `ProjectContextPolicy`. Each plan phase
|
||||
(`default` -> `strategize` -> `execute` -> `apply`) may override the policy view, controlling
|
||||
which resources and file paths are visible.
|
||||
|
||||
4. **Budget Enforcement** — `ContextBudgetEnforcer` applies `max_file_size` and
|
||||
`max_total_size` constraints. The token budget is a hard ceiling:
|
||||
|
||||
```
|
||||
budget = model_context_window - response_reserve (4096) - tool_definitions - skeleton_allocation
|
||||
```
|
||||
|
||||
Budget refresh is triggered when available budget changes by more than
|
||||
`context.budget.refresh-threshold` (default: 0.30).
|
||||
|
||||
5. **Delivery** — The assembled `AssembledContext` (or `ContextPayload`) is delivered to the
|
||||
actor for its LLM call. Child plans inherit a compressed skeleton of their parent's context
|
||||
(default: 15% of child's budget via `SkeletonCompressor`).
|
||||
|
||||
#### Storage Tiers
|
||||
|
||||
ACMS v1 manages context fragments across three storage tiers:
|
||||
|
||||
| Tier | Backend | Latency | Capacity | Actor Visibility |
|
||||
|------|---------|---------|----------|-----------------|
|
||||
| **Hot** | In-memory cache | Low | Token-budget (`max_tokens_hot`, default: 8,000) | All roles |
|
||||
| **Warm** | SQLite (fast disk) | Medium | Decision-count (`max_decisions_warm`, default: 500) | Strategist, Executor |
|
||||
| **Cold** | File archive (compressed) | High | Decision-count (`max_decisions_cold`, default: 5,000) | Strategist only |
|
||||
|
||||
**Actor roles and tier visibility:**
|
||||
|
||||
| Role | Sees |
|
||||
|------|------|
|
||||
| `strategist` | hot + warm + cold |
|
||||
| `executor` | hot + warm |
|
||||
| `reviewer` | hot only |
|
||||
|
||||
**Tier operations:**
|
||||
|
||||
- `promote(fragment_id)` — cold->warm or warm->hot.
|
||||
- `demote(fragment_id)` — hot->warm or warm->cold (with optional summarisation).
|
||||
- `evict_lru(tier, count)` — evict least-recently-used fragments from a tier.
|
||||
|
||||
> **v1 Note:** In v1, tier labels on `ContextFragment` are sort-priority labels used for
|
||||
> ranking during assembly (`hot > warm > cold`). Full storage-tier semantics with retention
|
||||
> policies, promotion/demotion, and eviction are implemented in `ContextTierService`.
|
||||
|
||||
---
|
||||
|
||||
### Module Definitions
|
||||
|
||||
#### Advanced Invariant Lifecycle
|
||||
ACMS v1 follows the project's layered architecture (Domain -> Application -> Infrastructure ->
|
||||
Presentation).
|
||||
|
||||
**Motivation**: The M1-M6 implementation enforces invariants at phase boundaries but does not version invariant snapshots or provide audit trails for conflict resolution decisions. Enterprise deployments require full auditability of why a particular invariant view was computed.
|
||||
#### Domain Layer
|
||||
|
||||
**Specification**:
|
||||
Domain models are pure Pydantic v2 frozen dataclasses with ULID identifiers and UTC datetimes.
|
||||
No infrastructure dependencies.
|
||||
|
||||
- **Invariant Snapshots**: At each phase boundary enforcement pass, the system captures a versioned `InvariantSnapshot` containing: the full effective invariant view, the conflict resolution decisions made by the Invariant Reconciliation Actor, the precedence chain applied (`plan > action > project > global`), and a timestamp. Snapshots are stored in the decision tree as `invariant_snapshot` decision records.
|
||||
| Class | Module | Description |
|
||||
|-------|--------|-------------|
|
||||
| `ContextFragment` | `cleveragents.domain.models.core.context_fragment` | Atomic unit of context with UKO node, content, depth, token count, relevance, provenance, and tier |
|
||||
| `FragmentProvenance` | `cleveragents.domain.models.core.context_fragment` | Links a fragment back to its originating resource and location |
|
||||
| `ContextBudget` | `cleveragents.domain.models.core.context_fragment` | Token budget with max and reserved tokens |
|
||||
| `ContextPayload` | `cleveragents.domain.models.core.context_fragment` | Assembled payload with fragments, token count, budget usage, context hash, and provenance map |
|
||||
| `ContextView` | `cleveragents.domain.models.core.context_policy` | Per-phase view controlling resource/path filters and size limits |
|
||||
| `ProjectContextPolicy` | `cleveragents.domain.models.core.context_policy` | Per-project policy with phase-based view inheritance |
|
||||
| `ContextRequest` | `cleveragents.domain.models.acms.crp` | Structured context request issued by an actor via CRP |
|
||||
| `DetailLevelMap` | `cleveragents.domain.models.acms.crp` | Maps named detail levels to integer depths for a UKO domain |
|
||||
| `AssembledContext` | `cleveragents.domain.models.acms.crp` | Fused, budget-respecting context payload delivered to an actor |
|
||||
| `TieredFragment` | `cleveragents.domain.models.acms.context_tiers` | Extends `ContextFragment` with tier placement and access tracking metadata |
|
||||
| `TierBudget` | `cleveragents.domain.models.acms.context_tiers` | Controls capacity per tier |
|
||||
| `ActorContextView` | `cleveragents.domain.models.acms.context_tiers` | Per-actor filtered view configuration with role-based default tiers |
|
||||
| `TierMetrics` | `cleveragents.domain.models.acms.context_tiers` | Hit/miss counters and population counts per tier |
|
||||
| `FileRecord` | `cleveragents.domain.models.acms.context_indexing` | Per-file metadata stored during indexing |
|
||||
| `IndexMetadata` | `cleveragents.domain.models.acms.context_indexing` | Summary record for a repository index |
|
||||
| `RepoIndex` | `cleveragents.domain.models.acms.context_indexing` | Composite object returned by index and refresh operations |
|
||||
|
||||
- **Conflict Audit Trail**: When the Invariant Reconciliation Actor resolves a conflict between two invariants at different scopes, it records an `invariant_conflict_resolved` decision with: the conflicting invariants, the winning invariant and its scope, the losing invariant and its scope, and the actor's rationale.
|
||||
#### Application Layer
|
||||
|
||||
- **Invariant Versioning**: Invariants now carry a `version` field (integer, auto-incremented on update). The effective invariant view records the version of each contributing invariant, enabling point-in-time reconstruction of any historical invariant view.
|
||||
Application services orchestrate domain models and delegate to infrastructure adapters.
|
||||
They are registered as singletons in the DI container.
|
||||
|
||||
- **Non-Overridable Propagation**: Global invariants marked `non_overridable` are now explicitly tagged in child plan invariant views, preventing any child plan from silently overriding them even via plan-scope invariants.
|
||||
| Class | Module | Description |
|
||||
|-------|--------|-------------|
|
||||
| `ACMSPipeline` | `cleveragents.application.services.acms_service` | Orchestrates the 10-component context assembly pipeline |
|
||||
| `ContextTierService` | `cleveragents.application.services.context_tiers` | Manages fragment lifecycle across hot/warm/cold tiers with LRU eviction |
|
||||
| `RepoIndexingService` | `cleveragents.application.services.context_indexing` | Scans repository resources and builds persistent file-level indexes |
|
||||
| `ContextBudgetEnforcer` | `cleveragents.application.services.acms_service` | Enforces `max_file_size` and `max_total_size` constraints on assembled context |
|
||||
| `ContextStrategyRegistry` | `cleveragents.application.services.context_strategies` | Manages pluggable retrieval strategies; validates capabilities and handles fallback degradation |
|
||||
| `UKOLoader` | `cleveragents.application.services.uko_loader` | Parses `docs/ontology/uko.ttl` into an in-memory ontology graph; resolves inheritance chains |
|
||||
|
||||
**CLI Extensions**:
|
||||
#### Infrastructure Layer
|
||||
|
||||
Infrastructure adapters implement persistence and external integrations.
|
||||
|
||||
| Class | Module | Description |
|
||||
|-------|--------|-------------|
|
||||
| `UKOStore` | `cleveragents.infrastructure.acms.uko_store` | RDF storage adapter for UKO ontology nodes; supports query and index operations |
|
||||
| `ContextTierManager` | `cleveragents.infrastructure.acms.context_tier_manager` | Manages hot (in-memory), warm (SQLite), and cold (file) storage backends |
|
||||
| `ContextPersistenceAdapter` | `cleveragents.infrastructure.acms.context_persistence` | SQLAlchemy-backed persistence for `repo_indexes` and `indexed_files` tables |
|
||||
| `TextBackend` | `cleveragents.infrastructure.acms.backends` | Full-text search via Tantivy or SQLite FTS5 |
|
||||
| `VectorBackend` | `cleveragents.infrastructure.acms.backends` | Vector similarity search via FAISS or Qdrant |
|
||||
| `GraphBackend` | `cleveragents.infrastructure.acms.backends` | Graph traversal over UKO structural relationships |
|
||||
| `TemporalBackend` | `cleveragents.infrastructure.acms.backends` | Temporal/cold-tier data access for historical context |
|
||||
|
||||
#### Presentation Layer
|
||||
|
||||
CLI commands expose ACMS functionality to users.
|
||||
|
||||
| Class | Module | Description |
|
||||
|-------|--------|-------------|
|
||||
| `ContextCLI` | `cleveragents.presentation.cli.context` | `agents context list/add/show/clear` commands |
|
||||
| `ProjectContextCLI` | `cleveragents.presentation.cli.project_context` | `agents project context set/show/inspect/simulate` commands |
|
||||
|
||||
---
|
||||
|
||||
### Data Models
|
||||
|
||||
#### ContextFragment
|
||||
|
||||
The atomic unit of context returned by strategies and consumed by the assembly pipeline.
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `fragment_id` | `str` | auto ULID | Stable unique identifier |
|
||||
| `uko_node` | `str` | required | UKO URI of the source node |
|
||||
| `content` | `str` | required | Rendered text content (max 1,000,000 chars) |
|
||||
| `detail_depth` | `int` | `0` | Resolved depth: 0 (MODULE_LISTING) through 9 (FULL_SOURCE) |
|
||||
| `token_count` | `int` | required | Actual token count of content (>= 0) |
|
||||
| `relevance_score` | `float` | `0.5` | Score from 0.0 to 1.0 |
|
||||
| `provenance` | `FragmentProvenance` | required | Provenance trace |
|
||||
| `tier` | `str` | `"warm"` | Priority tier: `"hot"`, `"warm"`, `"cold"` |
|
||||
| `metadata` | `dict[str, str]` | `{}` | Arbitrary key-value metadata (max 64 entries) |
|
||||
| `created_at` | `datetime` | auto UTC | Creation timestamp |
|
||||
|
||||
#### ContextBudget
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `max_tokens` | `int` | `4096` | Maximum total tokens (>= 1) |
|
||||
| `reserved_tokens` | `int` | `512` | Tokens reserved for system prompt (>= 0, < `max_tokens`) |
|
||||
| `available_tokens` | `int` | computed | `max_tokens - reserved_tokens` |
|
||||
|
||||
#### ProjectContextPolicy
|
||||
|
||||
Controls what context is available during each ACMS phase via view inheritance:
|
||||
|
||||
```
|
||||
agents invariant history <INVARIANT_ID> # Show version history
|
||||
agents invariant snapshot show <PLAN_ID> <PHASE> # Show snapshot for a phase
|
||||
agents invariant audit <PLAN_ID> # Show full conflict audit trail
|
||||
default -> strategize -> execute -> apply
|
||||
```
|
||||
|
||||
#### ACMS Observability and Tuning
|
||||
Each phase resolves to the first explicitly-set `ContextView` walking up the chain.
|
||||
|
||||
**Motivation**: Context assembly is the most latency-sensitive operation in the plan lifecycle. Production deployments need per-strategy latency budgets, assembly telemetry, and adaptive tier promotion to maintain SLA compliance.
|
||||
**ContextView fields:**
|
||||
|
||||
**Specification**:
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `include_resources` | `list[str]` | `[]` | Resource names/patterns to include (empty = all) |
|
||||
| `exclude_resources` | `list[str]` | `[]` | Resource names/patterns to exclude |
|
||||
| `include_paths` | `list[str]` | `[]` | File path globs to include (empty = all) |
|
||||
| `exclude_paths` | `list[str]` | `[]` | File path globs to exclude |
|
||||
| `max_file_size` | `int \| None` | `None` | Max file size in bytes (None = no limit) |
|
||||
| `max_total_size` | `int \| None` | `None` | Max total context size in bytes (None = no limit) |
|
||||
|
||||
- **Assembly Telemetry**: Each `assemble()` call emits a structured `ContextAssemblyTrace` containing: total assembly latency, per-strategy latency breakdown, fragment counts per strategy, fusion deduplication ratio, cache hit/miss rates per tier, and token budget utilization. Traces are emitted to the observability subsystem and accessible via `agents diagnostics context`.
|
||||
Exclusions always take precedence over inclusions.
|
||||
|
||||
- **Per-Strategy Latency Budgets**: Context strategies can declare a `max_latency_ms` budget. The `StrategyExecutor` enforces this budget via a per-strategy timeout. Strategies that exceed their budget are cancelled and their partial results are discarded (with a warning in the trace). Budget configuration:
|
||||
#### AssembledContext / ContextPayload
|
||||
|
||||
```yaml
|
||||
context_strategies:
|
||||
- name: semantic_embedding
|
||||
max_latency_ms: 500
|
||||
- name: graph_navigation
|
||||
max_latency_ms: 1000
|
||||
- name: temporal_archaeology
|
||||
max_latency_ms: 2000
|
||||
The fused, budget-respecting context payload delivered to an actor.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `payload_id` | `str` | ULID identifier |
|
||||
| `plan_id` | `str` | Plan this payload was assembled for |
|
||||
| `fragments` | `tuple[ContextFragment, ...]` | Ordered context fragments |
|
||||
| `total_tokens` | `int` | Sum of fragment `token_count` values |
|
||||
| `budget` | `ContextBudget` | Budget used for assembly |
|
||||
| `budget_used` | `float` | Fraction of budget consumed (0.0-1.0) |
|
||||
| `strategies_used` | `tuple[str, ...]` | Strategy names that contributed |
|
||||
| `context_hash` | `str` | SHA-256 hash of assembled content |
|
||||
| `preamble` | `str \| None` | Optional structure summary (max 200 tokens) |
|
||||
| `provenance_map` | `dict[str, Any]` | Fragment ID -> provenance mapping |
|
||||
| `assembled_at` | `datetime` | UTC timestamp |
|
||||
|
||||
**Properties:**
|
||||
|
||||
- `is_within_budget` — `True` if total tokens do not exceed available budget.
|
||||
- `remaining_tokens` — Tokens still available in the budget.
|
||||
|
||||
#### IndexMetadata
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `index_id` | `str` | ULID identifier for this index snapshot |
|
||||
| `resource_id` | `str` | ULID of the linked resource |
|
||||
| `indexed_at` | `datetime` | When indexing completed (UTC) |
|
||||
| `file_count` | `int` | Total files in the index |
|
||||
| `token_estimate` | `int` | Sum of all file token counts |
|
||||
| `primary_language` | `str` | Most common language by token count |
|
||||
| `status` | `IndexStatus` | `pending`, `indexing`, `ready`, `stale`, `error` |
|
||||
| `error_message` | `str \| None` | Error details when `status == error` |
|
||||
|
||||
---
|
||||
|
||||
### Key Interfaces
|
||||
|
||||
#### ContextAssemblyService (ACMSPipeline)
|
||||
|
||||
```python
|
||||
class ACMSPipeline:
|
||||
def assemble(
|
||||
self,
|
||||
plan_id: str,
|
||||
fragments: Sequence[ContextFragment],
|
||||
budget: ContextBudget,
|
||||
strategy: str = "relevance",
|
||||
) -> ContextPayload: ...
|
||||
|
||||
def register_strategy(self, name: str, strategy: ContextStrategy) -> None: ...
|
||||
```
|
||||
|
||||
- **Adaptive Tier Promotion**: The ACMS hot tier now supports adaptive promotion: fragments accessed more than `context.hot.promotion_threshold` times within a plan's lifetime are automatically promoted from warm to hot storage. This reduces repeated cold/warm lookups for frequently-accessed context.
|
||||
The pipeline applies a named strategy to rank and filter fragments, then runs all 10
|
||||
pipeline components to produce the final `ContextPayload`.
|
||||
|
||||
- **Context Budget Alerts**: When context assembly consumes more than `context.budget.alert_threshold` (default: 80%) of the token budget, the system emits a `context_budget_alert` event. Plans can configure alert handlers (log, escalate, or truncate-strategy).
|
||||
#### ContextBudgetEnforcer
|
||||
|
||||
#### Plan Hierarchy Enhancements
|
||||
|
||||
**Motivation**: Complex multi-plan workflows require coordination primitives beyond simple parent-child spawning. M7 introduces sibling coordination and resource locking.
|
||||
|
||||
**Specification**:
|
||||
|
||||
- **Cross-Plan Invariant Propagation**: Parent plans can now mark specific invariants as `propagate_to_siblings: true`. When a sibling plan is spawned in the same execution group, it inherits these invariants in addition to its own action-level invariants. Sibling-propagated invariants have lower precedence than the sibling's own plan-scope invariants but higher precedence than project-scope invariants.
|
||||
|
||||
- **Sibling Plan Coordination**: Plans executing in `DEPENDENCY_ORDERED` mode can now declare `coordination_signals` - named boolean flags that one sibling sets and another waits on. This enables producer-consumer patterns without requiring a parent plan to mediate.
|
||||
|
||||
```yaml
|
||||
# In subplan_spawn decision:
|
||||
coordination_signals:
|
||||
produces: [schema_migration_complete]
|
||||
waits_for: [database_ready]
|
||||
```python
|
||||
class ContextBudgetEnforcer:
|
||||
def enforce(
|
||||
self,
|
||||
view: ContextView,
|
||||
policy: ContextPolicy,
|
||||
) -> ContextView: ...
|
||||
```
|
||||
|
||||
- **Plan-Level Resource Locking**: Plans can declare `resource_locks` - exclusive or shared locks on specific resources for the duration of a phase. The lock manager prevents conflicting plans from entering the same phase concurrently on the same resource.
|
||||
Applies `max_file_size` and `max_total_size` constraints from the policy to the assembled
|
||||
view, excluding or summarising fragments that exceed limits.
|
||||
|
||||
```yaml
|
||||
resource_locks:
|
||||
- resource: my-namespace/production-db
|
||||
mode: exclusive # or: shared
|
||||
phases: [execute, apply]
|
||||
#### UKOStore
|
||||
|
||||
```python
|
||||
class UKOStore:
|
||||
def query(
|
||||
self,
|
||||
ontology: UKOOntology,
|
||||
query: str,
|
||||
) -> list[ContextFragment]: ...
|
||||
|
||||
def index(
|
||||
self,
|
||||
project: Project,
|
||||
) -> UKOOntology: ...
|
||||
```
|
||||
|
||||
#### CLI Communication Pattern Migration
|
||||
RDF storage adapter for UKO ontology nodes. Supports SPARQL-like queries and full project
|
||||
indexing.
|
||||
|
||||
**Context**: The current CLI implementation uses direct Python imports to call application-layer services, bypassing the A2A protocol boundary. This creates a reverse dependency (Presentation to Application direct coupling) that violates the layered architecture defined in ADR-001.
|
||||
#### ContextTierService
|
||||
|
||||
**Decision**: A formal migration plan is established for M9 (v3.8.0). Until then, the direct import pattern is permitted under a documented **local CLI exemption**:
|
||||
```python
|
||||
class ContextTierService:
|
||||
def store(self, fragment: TieredFragment) -> None: ...
|
||||
def get(self, fragment_id: str) -> TieredFragment | None: ...
|
||||
def promote(self, fragment_id: str) -> None: ... # cold->warm or warm->hot
|
||||
def demote(self, fragment_id: str) -> None: ... # hot->warm or warm->cold
|
||||
def evict_lru(self, tier: ContextTier, count: int) -> int: ...
|
||||
def get_for_actor(
|
||||
self,
|
||||
role: ActorRole,
|
||||
project_names: list[str],
|
||||
) -> list[TieredFragment]: ...
|
||||
def get_scoped_view(self, project_names: list[str]) -> ScopedBackendView: ...
|
||||
def get_metrics(self) -> TierMetrics: ...
|
||||
```
|
||||
|
||||
- The exemption applies only to the local (non-server) CLI execution path.
|
||||
- All server-mode CLI commands must route through A2A (already enforced).
|
||||
- The `import-linter` configuration is updated to explicitly allow the exempted imports with a `# cli-exemption: local-only` annotation.
|
||||
- No new direct imports may be added without a corresponding issue tracking the M9 migration.
|
||||
#### RepoIndexingService
|
||||
|
||||
**Migration Path (M9)**:
|
||||
```python
|
||||
class RepoIndexingService:
|
||||
def index_resource(
|
||||
self,
|
||||
resource_id: str,
|
||||
root_path: str | Path,
|
||||
*,
|
||||
include_globs: tuple[str, ...] = (),
|
||||
exclude_globs: tuple[str, ...] = (),
|
||||
max_file_size: int | None = None,
|
||||
max_total_size: int | None = None,
|
||||
) -> RepoIndex: ...
|
||||
|
||||
1. Introduce `A2aLocalFacade` as the sole interface between CLI and application layer.
|
||||
2. Replace all direct service imports in CLI command handlers with `facade.call(method, params)`.
|
||||
3. Remove the `import-linter` exemption rules.
|
||||
4. Validate with full integration test suite.
|
||||
def refresh_index(self, resource_id: str, root_path: str | Path, **kwargs) -> RepoIndex: ...
|
||||
def get_index(self, resource_id: str) -> RepoIndex | None: ...
|
||||
def get_index_status(self, resource_id: str) -> IndexMetadata | None: ...
|
||||
def remove_index(self, resource_id: str) -> bool: ...
|
||||
def cleanup_stale_indexing(self) -> int: ...
|
||||
```
|
||||
|
||||
See [ADR-049: CLI Communication Pattern](adr/ADR-049-cli-communication-pattern.md) for the full decision record.
|
||||
---
|
||||
|
||||
### Cross-Cutting Concerns
|
||||
### 10-Component Pipeline Architecture
|
||||
|
||||
#### Backward Compatibility
|
||||
The ACMS pipeline runs three phases, each containing specific components. All 10 components
|
||||
have explicit Protocol definitions and injectable Default implementations.
|
||||
|
||||
All M7 features are additive. Existing plans, actions, invariants, and configurations continue to work without modification. New fields (`version`, `propagate_to_siblings`, `resource_locks`, `coordination_signals`) default to values that preserve existing behavior.
|
||||
**Phase 1 — Strategy Orchestration:**
|
||||
|
||||
#### Performance Targets
|
||||
| # | Component | Responsibility |
|
||||
|---|-----------|---------------|
|
||||
| 1 | `StrategySelector` | Decides which strategies to invoke and with what confidence. Polls `can_handle()` on each registered strategy. |
|
||||
| 2 | `BudgetAllocator` | Distributes the token budget across selected strategies proportionally. |
|
||||
| 3 | `StrategyExecutor` | Runs strategies in parallel with timeouts (default: 30s), circuit breakers (threshold: 3 failures), and configurable max workers (default: 4). |
|
||||
|
||||
| Feature | Target |
|
||||
| :------ | :----- |
|
||||
| Invariant snapshot capture | < 5ms per phase boundary |
|
||||
| Context assembly telemetry overhead | < 1% of total assembly latency |
|
||||
| Adaptive tier promotion decision | < 1ms per fragment access |
|
||||
| Resource lock acquisition | < 10ms (uncontested) |
|
||||
**Phase 2 — Fragment Fusion:**
|
||||
|
||||
#### Security Considerations
|
||||
| # | Component | Responsibility |
|
||||
|---|-----------|---------------|
|
||||
| 4 | `FragmentDeduplicator` | Removes duplicate fragments via content-hash, UKO-identity, or semantic similarity. |
|
||||
| 5 | `DetailDepthResolver` | Resolves conflicts when the same UKO node appears at different detail depths (keeps the most appropriate depth for the budget). |
|
||||
| 6 | `FragmentScorer` | Computes composite relevance scores using weighted factors: relevance (0.4), hierarchy position (0.3), strategy quality (0.2), recency (0.1). |
|
||||
| 7 | `BudgetPacker` | Fits scored fragments into the token budget using a greedy knapsack algorithm with depth fallback (steps: `[9, 4, 2, 0]`). Minimum fragment size: 10 tokens. |
|
||||
| 8 | `FragmentOrderer` | Orders packed fragments for optimal coherence in the context window. |
|
||||
|
||||
- Invariant snapshots are stored in the decision tree and subject to the same access controls as decisions.
|
||||
- Context assembly traces may contain sensitive context fragments; they are stored encrypted at rest when `storage.encrypt_at_rest` is enabled.
|
||||
- Resource locks are scoped to the plan's namespace; cross-namespace locking requires explicit `allow_cross_namespace_locks: true` configuration.
|
||||
**Phase 3 — Context Finalization:**
|
||||
|
||||
| # | Component | Responsibility |
|
||||
|---|-----------|---------------|
|
||||
| 9 | `PreambleGenerator` | Generates provenance summaries prepended to assembled context (max tokens: 200). |
|
||||
| 10 | `SkeletonCompressor` | Compresses parent plan context into skeleton form for child plan inheritance (budget ratio: `context.budget.skeleton-ratio`, default: 0.15). |
|
||||
|
||||
Each component is pluggable via `context.pipeline.*` configuration keys specifying
|
||||
`"module:ClassName"`. All 10 components must execute for every context assembly; components
|
||||
may produce empty output but must not be skipped.
|
||||
|
||||
**v1 defaults:** Phase 1 uses single-strategy selection, full-budget allocation, and
|
||||
synchronous execution. Phase 2 components are pass-through stubs. Phase 3 components are
|
||||
no-op (no preamble, identity compression). Production implementations are planned for future
|
||||
milestones.
|
||||
|
||||
---
|
||||
|
||||
### Context Strategies
|
||||
|
||||
Strategies implement the `ContextStrategy` protocol and are registered in the
|
||||
`ContextStrategyRegistry`.
|
||||
|
||||
**Built-in strategies:**
|
||||
|
||||
| Strategy | Quality | Backends | Description |
|
||||
|----------|---------|----------|-------------|
|
||||
| `simple-keyword` | 0.3 | Text | Full-text search via Tantivy or SQLite FTS5 |
|
||||
| `semantic-embedding` | 0.6 | Vector | Vector similarity search via FAISS or Qdrant |
|
||||
| `breadth-depth-navigator` | 0.85 | Graph | Graph traversal from focus nodes, expanding by `request.breadth` hops |
|
||||
| `arce` | 0.95 | All | Multi-modal: text (40%) + vector (40%) + graph (20%); results merged and deduplicated |
|
||||
| `temporal-archaeology` | 0.5 | Graph + Cold | Two-phase: cold-tier historical nodes + graph traversal |
|
||||
| `plan-decision-context` | 0.7 | Warm/Cold | Walks parent/ancestor plan hierarchy, retrieving decision records |
|
||||
|
||||
Default enabled: `["simple-keyword", "semantic-embedding", "breadth-depth-navigator"]`
|
||||
|
||||
**Fallback degradation path** (when backends are unavailable):
|
||||
|
||||
```
|
||||
arce -> breadth-depth-navigator -> semantic-embedding -> simple-keyword
|
||||
```
|
||||
|
||||
**StrategyCapabilities fields:**
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `uses_text` | `bool` | `False` | Requires TextBackend |
|
||||
| `uses_vector` | `bool` | `False` | Requires VectorBackend |
|
||||
| `uses_graph` | `bool` | `False` | Requires GraphBackend |
|
||||
| `uses_temporal` | `bool` | `False` | Requires temporal/cold-tier data |
|
||||
| `quality_score` | `float` | `0.5` | Intrinsic quality score (0.0-1.0) |
|
||||
| `supports_depth_breadth` | `bool` | `False` | Supports depth/breadth projection |
|
||||
| `supports_plan_hierarchy` | `bool` | `False` | Supports plan hierarchy traversal |
|
||||
|
||||
---
|
||||
|
||||
### CLI Commands
|
||||
|
||||
#### `agents context` — Context Entry Management
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `agents context list` | List context entries for the current project |
|
||||
| `agents context add` | Add a context entry |
|
||||
| `agents context show <id>` | Show details of a specific context entry |
|
||||
| `agents context clear` | Clear all context entries for the current project |
|
||||
|
||||
#### `agents project context` — Project Context Policy and ACMS Management
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `agents project context set PROJECT [OPTIONS]` | Persist a context policy view and ACMS pipeline configuration for a project |
|
||||
| `agents project context show PROJECT [OPTIONS]` | Display the context policy and ACMS pipeline configuration |
|
||||
| `agents project context inspect PROJECT [OPTIONS]` | Inspect the effective context state (tier metrics, fragment distribution, actor visibility) |
|
||||
| `agents project context simulate PROJECT [OPTIONS]` | Run a dry-run context window assembly |
|
||||
|
||||
**`agents project context set` key options:**
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `--view` | `default` | Phase view: default, strategize, execute, apply |
|
||||
| `--include-resource` | — | Resource pattern to include (repeatable) |
|
||||
| `--exclude-resource` | — | Resource pattern to exclude (repeatable) |
|
||||
| `--include-path` | — | File path glob to include (repeatable) |
|
||||
| `--exclude-path` | — | File path glob to exclude (repeatable) |
|
||||
| `--max-file-size` | `None` | Max file size in bytes |
|
||||
| `--max-total-size` | `None` | Max total context size in bytes |
|
||||
| `--hot-max-tokens` | `8000` | Max tokens for the hot tier context window |
|
||||
| `--warm-max-decisions` | `500` | Max decisions for the warm tier |
|
||||
| `--cold-max-decisions` | `5000` | Max decisions for the cold tier |
|
||||
| `--strategy` | — | Preferred context strategy (repeatable) |
|
||||
| `--default-breadth` | `2` | Default breadth for context retrieval |
|
||||
| `--default-depth` | `3` | Default depth (integer or named level) |
|
||||
| `--skeleton-ratio` | `0.2` | Skeleton compression ratio (0.0-1.0) |
|
||||
| `--temporal-scope` | `current` | Temporal scope: current, recent, or all |
|
||||
| `--auto-refresh/--no-auto-refresh` | `True` | Auto-refresh context on resource changes |
|
||||
|
||||
---
|
||||
|
||||
### Integration Points
|
||||
|
||||
- **Decision Tree**: Invariant snapshots and conflict audit records are new decision types integrated into the existing decision tree model.
|
||||
- **ACMS**: Telemetry traces integrate with the existing observability subsystem (ADR-025).
|
||||
- **Plan Lifecycle**: Resource locking integrates with the existing phase transition machinery.
|
||||
- **A2A Protocol**: New `_cleveragents/invariant.snapshot`, `_cleveragents/context.trace`, and `_cleveragents/plan.lock` extension methods are added.
|
||||
#### Plan Lifecycle Integration
|
||||
|
||||
### Milestone Plan
|
||||
Context assembly is triggered at each plan phase transition (see ADR-006). The ACMS pipeline
|
||||
runs before the actor's LLM call for each phase:
|
||||
|
||||
| Feature | Status | Issue |
|
||||
| :------ | :----- | :---- |
|
||||
| Advanced Invariant Lifecycle | Planned | #9899 |
|
||||
| ACMS Observability and Tuning | Planned | #9859 |
|
||||
| Plan Hierarchy Enhancements | Planned | - |
|
||||
| CLI Communication Pattern Migration | Planned | - |
|
||||
1. `strategize` — full context assembly with `breadth-depth-navigator` + `semantic-embedding`.
|
||||
2. `execute` — focused context assembly scoped to the current task.
|
||||
3. `apply` — minimal context assembly; primarily hot-tier fragments.
|
||||
|
||||
Child plans inherit a compressed skeleton of their parent's context via `SkeletonCompressor`.
|
||||
|
||||
#### Resource System Integration
|
||||
|
||||
When a resource is linked to a project via `agents project link-resource`, the
|
||||
`RepoIndexingService` is triggered to index the resource's filesystem tree. The index is
|
||||
stored in SQLite and refreshed incrementally on file changes (see ADR-008).
|
||||
|
||||
#### Actor and Agent Integration
|
||||
|
||||
Actors consume assembled context via the `builtin/context` skill, which provides three tools:
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `request_context` | Request specific context during reasoning |
|
||||
| `query_history` | Query historical context about past decisions |
|
||||
| `get_context_budget` | Check remaining context token budget |
|
||||
|
||||
#### Configuration System Integration
|
||||
|
||||
ACMS behaviour is controlled via `config.toml` under the `[context]` namespace (see ADR-024).
|
||||
|
||||
**Key configuration keys:**
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `context.hot.max-tokens` | `16000` | Hot tier token budget |
|
||||
| `context.warm.max-decisions` | `100` | Warm tier max decisions |
|
||||
| `context.warm.retention-hours` | `24` | Warm tier retention period |
|
||||
| `context.cold.max-decisions` | `500` | Cold tier max decisions |
|
||||
| `context.cold.retention-days` | `90` | Cold tier retention period |
|
||||
| `context.strategies.enabled` | `["simple-keyword", "semantic-embedding", "breadth-depth-navigator"]` | Enabled strategies |
|
||||
| `context.strategies.arce.max-rounds` | `3` | ARCE max refinement rounds |
|
||||
| `context.strategies.breadth-depth-navigator.max-hops` | `4` | Max graph traversal hops |
|
||||
| `context.query.min-relevance` | `0.3` | Minimum relevance score threshold |
|
||||
| `context.file.max-size` | `1048576` | Max file size in bytes (1 MB) |
|
||||
| `context.file.max-total-size` | `52428800` | Max total file size (50 MB) |
|
||||
| `context.budget.refresh-threshold` | `0.30` | Budget change threshold for refresh |
|
||||
| `context.budget.min-useful-budget` | `500` | Minimum useful budget (tokens) |
|
||||
| `context.budget.skeleton-ratio` | `0.15` | Skeleton compression ratio |
|
||||
| `context.pipeline.*` | — | Pluggable pipeline component overrides (`"module:ClassName"`) |
|
||||
|
||||
#### DI Container Integration
|
||||
|
||||
All ACMS services are registered as singletons in the DI container:
|
||||
|
||||
```python
|
||||
from cleveragents.application.container import get_container
|
||||
|
||||
container = get_container()
|
||||
acms_pipeline = container.acms_pipeline()
|
||||
tier_service = container.context_tier_service()
|
||||
indexing_service = container.repo_indexing_service()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Database Schema
|
||||
|
||||
ACMS v1 adds two tables to the SQLite database:
|
||||
|
||||
#### `repo_indexes`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `index_id` | `String(26)` | PK |
|
||||
| `resource_id` | `String(26)` | NOT NULL, UNIQUE, INDEXED |
|
||||
| `indexed_at` | `String(40)` | NOT NULL (ISO-8601 UTC) |
|
||||
| `file_count` | `Integer` | NOT NULL, DEFAULT 0 |
|
||||
| `token_estimate` | `Integer` | NOT NULL, DEFAULT 0 |
|
||||
| `primary_language` | `String(50)` | NOT NULL, DEFAULT "unknown" |
|
||||
| `status` | `String(20)` | NOT NULL, CHECK IN (`pending`, `indexing`, `ready`, `stale`, `error`) |
|
||||
| `error_message` | `Text` | NULLABLE |
|
||||
| `created_at` | `String(40)` | NOT NULL (ISO-8601 UTC) |
|
||||
|
||||
#### `indexed_files`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `index_id` | `String(26)` | PK (composite), FK -> `repo_indexes.index_id` ON DELETE CASCADE |
|
||||
| `path` | `String(1024)` | PK (composite) |
|
||||
| `content_hash` | `String(64)` | NOT NULL |
|
||||
| `token_count` | `Integer` | NOT NULL, DEFAULT 0 |
|
||||
| `size_bytes` | `Integer` | NOT NULL, DEFAULT 0 |
|
||||
| `language` | `String(50)` | NOT NULL, DEFAULT "unknown" |
|
||||
| `last_modified` | `String(40)` | NOT NULL |
|
||||
|
||||
---
|
||||
|
||||
### v1 Known Limitations
|
||||
|
||||
The following are accepted deviations from the full specification in v1, with a planned path
|
||||
to spec conformance in future milestones:
|
||||
|
||||
| Area | v1 Behaviour | Spec Target | Planned |
|
||||
|------|-------------|-------------|---------|
|
||||
| Pipeline components | All 10 Protocol + Default classes defined; defaults are pass-through stubs | Production implementations (parallel execution, dedup, scoring, compression) | Future milestone |
|
||||
| `ContextStrategy` signatures | `can_handle(request: dict)`, `assemble(fragments, budget)` | `can_handle(request: ContextRequest, backends: BackendSet)`, `assemble(request, backends, budget, plan_context)` | M6 strategy registry |
|
||||
| `StrategyCapabilities` fields | `supports_semantic_search`, `supports_graph_navigation`, `max_fragments` | `uses_text`, `uses_vector`, `uses_graph`, `uses_temporal`, `quality_score` | M6 strategy registry |
|
||||
| Tiers | Sort-priority labels for ranking | Storage tiers with retention policies, promotion/demotion | `ContextTierService` future milestone |
|
||||
| `SemanticEmbeddingStrategy` | Character-frequency embedding approximation | Real embedding model integration | Future milestone |
|
||||
| `DetailDepthResolver.resolve()` | Missing `budget` parameter | `resolve(fragments, budget)` | Future milestone |
|
||||
| `FragmentScorer.score()` | Missing `plan_context` param; returns `Sequence[ContextFragment]` | Returns `list[ScoredFragment]` with `plan_context` | Future milestone |
|
||||
| `PreambleGenerator.generate()` | Missing `strategies_used`, `budget_used`, `max_tokens` params | Full parameter set per spec | Future milestone |
|
||||
| `provenance_map` keying | Keyed by `fragment_id` (ULID) | Spec keys by `uko_node` (UKO URI) | Under review |
|
||||
|
||||
---
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
The following criteria must be satisfied for ACMS v1 to be considered complete for milestone
|
||||
v3.4.0:
|
||||
|
||||
**Indexing:**
|
||||
|
||||
- `RepoIndexingService.index_resource()` successfully indexes a repository with 10,000+
|
||||
files without timeout (target: < 60 seconds on standard hardware).
|
||||
- Incremental `refresh_index()` only re-processes changed files (verified by content hash
|
||||
comparison).
|
||||
- Language detection correctly identifies all supported languages.
|
||||
- Index status transitions correctly through `pending -> indexing -> ready`.
|
||||
- `cleanup_stale_indexing()` removes orphan `INDEXING` rows on startup.
|
||||
|
||||
**Assembly:**
|
||||
|
||||
- `ACMSPipeline.assemble()` produces a `ContextPayload` that never exceeds the declared
|
||||
token budget (`is_within_budget == True`).
|
||||
- All three built-in strategies (`relevance`, `recency`, `tiered`) produce correctly
|
||||
ordered fragment sequences.
|
||||
- The pipeline runs all 10 components for every assembly call.
|
||||
- Depth fallback correctly downgrades fragments to shallower depths when budget is tight.
|
||||
- `ContextBudgetEnforcer` correctly applies `max_file_size` and `max_total_size` limits.
|
||||
|
||||
**Tiers:**
|
||||
|
||||
- `ContextTierService.store()` correctly places fragments in the specified tier.
|
||||
- `promote()` and `demote()` correctly move fragments between tiers.
|
||||
- `evict_lru()` removes the correct number of least-recently-used fragments.
|
||||
- Actor role visibility is correctly enforced (reviewer sees hot only; executor sees hot +
|
||||
warm; strategist sees all).
|
||||
- Project scoping correctly isolates fragments by `project_name`.
|
||||
|
||||
**Policy:**
|
||||
|
||||
- `ProjectContextPolicy.resolve_view()` correctly walks the inheritance chain.
|
||||
- Empty policy correctly includes all resources and paths.
|
||||
- Exclusions take precedence over inclusions.
|
||||
- `max_file_size` and `max_total_size` validation rejects zero or negative values.
|
||||
|
||||
**CLI:**
|
||||
|
||||
- `agents context list/add/show/clear` commands function correctly.
|
||||
- `agents project context set` persists policy views and ACMS pipeline configuration.
|
||||
- `agents project context show` displays resolved views and ACMS config.
|
||||
- `agents project context inspect` displays tier metrics, fragment distribution, and actor
|
||||
visibility.
|
||||
- `agents project context simulate` produces a dry-run `AssembledContext` using the
|
||||
project's ACMS configuration.
|
||||
|
||||
**UKO:**
|
||||
|
||||
- `UKOLoader` correctly parses `docs/ontology/uko.ttl` and resolves inheritance chains
|
||||
(including multi-parent `rdfs:subClassOf`).
|
||||
- All four layers (0-3) are correctly identified by prefix.
|
||||
- `resolve_inheritance()` returns the correct BFS-ordered chain for `uko-oo:Class`.
|
||||
|
||||
**Testing:**
|
||||
|
||||
- Budget constraint tests verify assembled context never exceeds declared token budget.
|
||||
- Each of the 10 pipeline components has unit tests for edge cases (empty input, budget
|
||||
exhaustion, duplicate fragments).
|
||||
- Each retrieval strategy has integration tests verifying correctly structured results.
|
||||
- Depth fallback tests verify fragments are downgraded to shallower depths when budget is
|
||||
insufficient.
|
||||
- Configuration tests verify custom pipeline components and strategy registrations are
|
||||
loaded and invoked correctly.
|
||||
- Performance benchmark: index 10,000 files in < 60 seconds; assemble context in < 5
|
||||
seconds.
|
||||
|
||||
---
|
||||
|
||||
*Section added by [AUTO-ARCH-20] — Cycle 27 — Milestone v3.4.0 — ACMS v1 + Context Scaling*
|
||||
|
||||
Reference in New Issue
Block a user