forked from cleveragents/cleveragents-core
a41fc02f11
Implement StrategyCoordinator and FusionEngine as named facades over the existing ACMS pipeline components, providing clean public APIs for parallel strategy execution with proportional budget allocation and fragment fusion with dedup/conflict resolution/knapsack packing. Key changes: - Add StrategyCoordinator with parallel execution and confidence-based budget allocation - Add FusionEngine with URI+hash dedup, max-depth conflict resolution, greedy knapsack packing - Add budget overage guard with lowest-relevance fragment dropping - Add per-strategy max caps enforcement - Wire into existing ContextAssemblyPipeline - Add Behave BDD tests, Robot integration tests, ASV benchmarks - Add docs/reference/acms_fusion.md ISSUES CLOSED: #192
175 lines
7.2 KiB
Markdown
175 lines
7.2 KiB
Markdown
# ACMS Strategy Coordinator & Fusion Engine
|
|
|
|
## Overview
|
|
|
|
The **StrategyCoordinator** and **FusionEngine** are named facades that
|
|
provide clean public APIs over the existing ACMS pipeline Phase 1 and
|
|
Phase 2 components respectively.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ StrategyCoordinator │
|
|
│ ┌─────────────────┐ ┌──────────────────┐ ┌──────────┐ │
|
|
│ │ Confidence │ │ Proportional │ │ Parallel │ │
|
|
│ │ Weighted │ │ Budget │ │ Strategy │ │
|
|
│ │ Selector │ │ Allocator │ │ Executor │ │
|
|
│ └────────┬────────┘ └────────┬─────────┘ └────┬─────┘ │
|
|
│ │ select │ allocate │execute │
|
|
│ └───────────────────┴─────────────────┘ │
|
|
│ coordinate() │
|
|
└─────────────────────────┬───────────────────────────────┘
|
|
│ fragments
|
|
▼
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ FusionEngine │
|
|
│ ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌───────────┐ │
|
|
│ │ Content │ │ Max Depth │ │Weighted │ │ Greedy │ │
|
|
│ │ Hash │ │ Resolver │ │Composite│ │ Knapsack │ │
|
|
│ │ Dedup │ │ │ │ Scorer │ │ Packer │ │
|
|
│ └────┬─────┘ └─────┬─────┘ └────┬────┘ └─────┬─────┘ │
|
|
│ │ dedup │ resolve │ score │ pack │
|
|
│ └──────────────┴────────────┴─────────────┘ │
|
|
│ fuse() │
|
|
│ ┌──────────────────────────────────────────────────┐ │
|
|
│ │ Budget Overage Guard │ │
|
|
│ │ Drop lowest-relevance fragments if over budget │ │
|
|
│ └──────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## StrategyCoordinator
|
|
|
|
### Public API
|
|
|
|
```python
|
|
coordinator = StrategyCoordinator(config=CoordinatorConfig(
|
|
per_strategy_max_cap=500,
|
|
min_useful_budget=64,
|
|
))
|
|
|
|
result = coordinator.coordinate(
|
|
request={"preferred_strategies": ["relevance"]},
|
|
strategies=[strategy_a, strategy_b],
|
|
budget=ContextBudget(max_tokens=4096),
|
|
fragments=input_fragments,
|
|
backends={"vector": True},
|
|
plan_context={"plan_id": "..."},
|
|
)
|
|
|
|
# Result: CoordinationResult
|
|
# .fragments: list[ContextFragment]
|
|
# .strategies_used: list[str]
|
|
# .allocations: list[tuple[str, float, int]]
|
|
# .circuit_broken: list[str]
|
|
```
|
|
|
|
### Budget Allocation
|
|
|
|
Budget tokens are allocated **proportionally by strategy confidence**:
|
|
|
|
- Strategy with confidence 0.8 in a pool of (0.8 + 0.2) = 1.0
|
|
receives 80% of the budget.
|
|
- The largest-remainder method ensures no tokens are lost to rounding.
|
|
- Strategies whose proportional share falls below `min_useful_budget`
|
|
are excluded and their tokens are redistributed.
|
|
|
|
### Per-Strategy Max Caps
|
|
|
|
When `per_strategy_max_cap` is set:
|
|
|
|
1. Any allocation exceeding the cap is clamped.
|
|
2. Excess tokens are redistributed proportionally among uncapped
|
|
strategies.
|
|
3. Redistributed tokens are also capped to prevent cascading overflows.
|
|
|
|
### Error Handling
|
|
|
|
- **Circuit breaker**: After `circuit_breaker_threshold` consecutive
|
|
failures, a strategy is temporarily disabled. Disabled strategies
|
|
appear in `result.circuit_broken`.
|
|
- **Timeouts**: Per-strategy execution timeout of `executor_timeout`
|
|
seconds. Timed-out strategies are treated as failures.
|
|
- **Graceful degradation**: Failed strategies are excluded from results
|
|
without failing the entire coordination round.
|
|
|
|
## FusionEngine
|
|
|
|
### Public API
|
|
|
|
```python
|
|
engine = FusionEngine(config=FusionConfig(
|
|
overage_guard_enabled=True,
|
|
min_fragment_tokens=10,
|
|
))
|
|
|
|
result = engine.fuse(
|
|
fragments=coordinator_output.fragments,
|
|
budget=ContextBudget(max_tokens=4096),
|
|
)
|
|
|
|
# Result: FusionResult
|
|
# .fragments: list[ContextFragment]
|
|
# .dedup_count: int
|
|
# .depth_resolved_count: int
|
|
# .dropped_by_overage_guard: int
|
|
# .total_tokens: int
|
|
# .budget_utilization: float
|
|
```
|
|
|
|
### Fragment Dedup
|
|
|
|
Fragments are grouped by **(UKO URI, SHA-256 hash of content)**.
|
|
Within each group, the fragment with the highest `relevance_score`
|
|
is retained.
|
|
|
|
### Detail Conflict Resolution
|
|
|
|
When the same UKO node appears at multiple detail depths:
|
|
- The highest-depth rendering is retained.
|
|
- Equal depths: prefer higher `relevance_score`.
|
|
|
|
### Greedy Knapsack Packing
|
|
|
|
Fragments are sorted with **deterministic tie-breakers**:
|
|
|
|
1. `relevance_score` descending
|
|
2. `detail_depth` descending
|
|
3. `token_count` ascending
|
|
|
|
The greedy algorithm fills the budget top-down. When a high-value
|
|
fragment doesn't fit, **depth fallback** attempts lower-depth
|
|
renderings of the same UKO node.
|
|
|
|
### Budget Overage Guard
|
|
|
|
A post-packing safety net that:
|
|
|
|
1. Checks if total tokens exceed the available budget.
|
|
2. Drops fragments in ascending order of `relevance_score`.
|
|
3. Emits structured `WARNING` log messages for each dropped fragment.
|
|
4. Continues until total tokens fit within budget.
|
|
|
|
## Configuration Reference
|
|
|
|
### CoordinatorConfig
|
|
|
|
| Field | Type | Default | Description |
|
|
|---|---|---|---|
|
|
| `min_useful_budget` | `int` | 64 | Min tokens per strategy |
|
|
| `executor_timeout` | `float` | 30.0 | Per-strategy timeout (seconds) |
|
|
| `executor_max_workers` | `int` | 4 | Max concurrent threads |
|
|
| `circuit_breaker_threshold` | `int` | 3 | Failures before circuit opens |
|
|
| `preference_boost` | `float` | 1.5 | Confidence multiplier for preferred |
|
|
| `per_strategy_max_cap` | `int \| None` | None | Max tokens per strategy |
|
|
|
|
### FusionConfig
|
|
|
|
| Field | Type | Default | Description |
|
|
|---|---|---|---|
|
|
| `scorer_weights` | `ScorerWeights \| None` | None | Composite scoring weights |
|
|
| `depth_fallback_steps` | `tuple[int, ...]` | (9,4,2,0) | Depth fallback sequence |
|
|
| `min_fragment_tokens` | `int` | 10 | Min tokens for packing |
|
|
| `overage_guard_enabled` | `bool` | True | Enable budget overage guard |
|