cleveragents/cleveragents-core

Fork 3

UAT: ACMSPipeline.assemble() silently narrows multi-strategy selection to a single strategy — StrategySelector, BudgetAllocator, and StrategyExecutor are wired but bypassed #5511

New issue

Open

opened 2026-04-09 07:07:44 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-09 07:07:44 +00:00

Owner

Summary

The ACMSPipeline.assemble() method wires up the full 10-component pipeline (including StrategySelector, BudgetAllocator, and StrategyExecutor) but then immediately filters the selector's output down to a single strategy, defeating the multi-strategy fusion that the spec requires.

What Was Tested

Code-level analysis of src/cleveragents/application/services/acms_service.py, specifically the ACMSPipeline.assemble() method.

Expected Behavior (from spec)

Per docs/reference/context_strategies.md and docs/specification.md §42630-42636:

StrategySelector polls all registered strategies via can_handle() and returns confidence-ranked candidates
BudgetAllocator distributes the token budget proportionally across selected strategies
StrategyExecutor invokes all allocated strategies in parallel and collects all resulting fragments
Multiple strategies can contribute fragments to a single assembly call

Actual Behavior

In ACMSPipeline.assemble() (lines ~350-380 of acms_service.py):

# Phase 1: Strategy Orchestration
all_strategies = list(self._strategies.values())
resolved = self._strategies[strategy_name]
candidates = self._strategy_selector.select(
    all_strategies,
    {"strategy": strategy_name},
)
# ⚠️ IMMEDIATELY FILTERS TO SINGLE STRATEGY:
candidates = [(s, c) for s, c in candidates if s.name == strategy_name] or [
    (resolved, 1.0)
]

The comment in the code acknowledges this is intentional for v1:

"keep only the requested one for v1 single-strategy execution; future multi-strategy support will remove this filter"

However, this means:

The StrategySelector is called but its multi-strategy output is discarded
The BudgetAllocator always receives exactly one candidate
The StrategyExecutor always executes exactly one strategy
The ARCE strategy's multi-modal pipeline (text 40% + vector 40% + graph 20%) is never used as designed — it's treated as a single strategy that gets 100% of the budget

Impact

The spec defines ARCE as: "Multi-modal pipeline: text search (40% budget) + vector similarity (40%) + graph traversal (20%); results merged and deduplicated". This budget split is implemented in ARCEStrategy.assemble() in strategy_stubs.py, but since the ACMSPipeline only ever calls one strategy at a time, the multi-strategy fusion path is never exercised.

The StrategySelector, BudgetAllocator, and StrategyExecutor pipeline components are effectively dead code in the current implementation.

Code Location

src/cleveragents/application/services/acms_service.py — ACMSPipeline.assemble() method, specifically the candidates filtering block.

Steps to Reproduce

Register multiple strategies with ACMSPipeline
Call pipeline.assemble(strategy="arce", ...)
Observe that only the ARCE strategy is invoked — no other strategies contribute fragments
Observe that StrategySelector.select() is called but its multi-strategy output is filtered to one

Suggested Fix

Remove the single-strategy filter (the candidates = [(s, c) for s, c in candidates if s.name == strategy_name] line) and allow the StrategySelector to return multiple candidates. The BudgetAllocator and StrategyExecutor are already implemented to handle multiple strategies correctly.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Summary The `ACMSPipeline.assemble()` method wires up the full 10-component pipeline (including `StrategySelector`, `BudgetAllocator`, and `StrategyExecutor`) but then **immediately filters the selector's output down to a single strategy**, defeating the multi-strategy fusion that the spec requires. ## What Was Tested Code-level analysis of `src/cleveragents/application/services/acms_service.py`, specifically the `ACMSPipeline.assemble()` method. ## Expected Behavior (from spec) Per `docs/reference/context_strategies.md` and `docs/specification.md §42630-42636`: - `StrategySelector` polls all registered strategies via `can_handle()` and returns confidence-ranked candidates - `BudgetAllocator` distributes the token budget proportionally across selected strategies - `StrategyExecutor` invokes all allocated strategies in parallel and collects all resulting fragments - Multiple strategies can contribute fragments to a single assembly call ## Actual Behavior In `ACMSPipeline.assemble()` (lines ~350-380 of `acms_service.py`): ```python # Phase 1: Strategy Orchestration all_strategies = list(self._strategies.values()) resolved = self._strategies[strategy_name] candidates = self._strategy_selector.select( all_strategies, {"strategy": strategy_name}, ) # ⚠️ IMMEDIATELY FILTERS TO SINGLE STRATEGY: candidates = [(s, c) for s, c in candidates if s.name == strategy_name] or [ (resolved, 1.0) ] ``` The comment in the code acknowledges this is intentional for v1: > *"keep only the requested one for v1 single-strategy execution; future multi-strategy support will remove this filter"* However, this means: 1. The `StrategySelector` is called but its multi-strategy output is discarded 2. The `BudgetAllocator` always receives exactly one candidate 3. The `StrategyExecutor` always executes exactly one strategy 4. The ARCE strategy's multi-modal pipeline (text 40% + vector 40% + graph 20%) is never used as designed — it's treated as a single strategy that gets 100% of the budget ## Impact The spec defines ARCE as: *"Multi-modal pipeline: text search (40% budget) + vector similarity (40%) + graph traversal (20%); results merged and deduplicated"*. This budget split is implemented in `ARCEStrategy.assemble()` in `strategy_stubs.py`, but since the `ACMSPipeline` only ever calls one strategy at a time, the multi-strategy fusion path is never exercised. The `StrategySelector`, `BudgetAllocator`, and `StrategyExecutor` pipeline components are effectively dead code in the current implementation. ## Code Location `src/cleveragents/application/services/acms_service.py` — `ACMSPipeline.assemble()` method, specifically the `candidates` filtering block. ## Steps to Reproduce 1. Register multiple strategies with `ACMSPipeline` 2. Call `pipeline.assemble(strategy="arce", ...)` 3. Observe that only the ARCE strategy is invoked — no other strategies contribute fragments 4. Observe that `StrategySelector.select()` is called but its multi-strategy output is filtered to one ## Suggested Fix Remove the single-strategy filter (the `candidates = [(s, c) for s, c in candidates if s.name == strategy_name]` line) and allow the `StrategySelector` to return multiple candidates. The `BudgetAllocator` and `StrategyExecutor` are already implemented to handle multiple strategies correctly. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 07:08:22 +00:00

HAL9000 added this to the v3.6.0 milestone

2026-04-09 07:08:22 +00:00

HAL9000 added

and removed

labels

2026-04-09 07:08:37 +00:00

HAL9000 modified the milestone from v3.6.0 to v3.2.0

2026-04-09 07:12:11 +00:00

HAL9000 referenced this issue

2026-04-09 07:13:22 +00:00

[AUTO-WATCHDOG] System Health Report (Cycle 44) #5370

HAL9000 commented

2026-04-09 07:13:37 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed

labels

2026-04-09 07:17:52 +00:00

HAL9000 commented

2026-04-09 07:18:25 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — ACMSPipeline silently narrowing multi-strategy selection is a behavioral bug that could produce unexpected results without user awareness.
Milestone: v3.2.0 (already assigned — correct)
Story Points: 3 — M — Fixing silent narrowing behavior and adding proper error/warning handling, 4-8 hours.
MoSCoW: MoSCoW/Should have — Silent failures in the ACMS pipeline are a quality issue. Users should be informed when their strategy selection is narrowed. Important but not blocking core functionality.
Parent Epic: Needs linking to appropriate v3.2.0 ACMS Epic

Valid UAT bug. Silent behavioral changes are a usability concern.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — ACMSPipeline silently narrowing multi-strategy selection is a behavioral bug that could produce unexpected results without user awareness. - **Milestone**: v3.2.0 (already assigned — correct) - **Story Points**: 3 — M — Fixing silent narrowing behavior and adding proper error/warning handling, 4-8 hours. - **MoSCoW**: MoSCoW/Should have — Silent failures in the ACMS pipeline are a quality issue. Users should be informed when their strategy selection is narrowed. Important but not blocking core functionality. - **Parent Epic**: Needs linking to appropriate v3.2.0 ACMS Epic Valid UAT bug. Silent behavioral changes are a usability concern. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 referenced this issue

2026-04-09 07:18:38 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 26) #5296

HAL9000 added the

Points

label

2026-04-09 07:18:52 +00:00

HAL9000 referenced this issue

2026-04-09 07:20:57 +00:00

[AUTO-ARCH] Architecture Review (Cycle 1) #5368

HAL9000 referenced this issue

2026-04-09 07:29:13 +00:00