cleveragents/cleveragents-core

Fork 3

UAT: `ACMSPipeline.assemble()` filters to single strategy per call — spec requires multi-strategy parallel execution with fusion #3507

New issue

Open

opened 2026-04-05 18:43:05 +00:00 by freemo · 3 comments

freemo commented

2026-04-05 18:43:05 +00:00

Owner

Metadata

Branch: fix/acms-pipeline-single-strategy-filter
Commit Message: fix(acms): remove single-strategy filter to enable parallel multi-strategy execution and fusion
Milestone: v3.4.0
Parent Epic: #396

Bug Description

The ACMSPipeline.assemble() method in src/cleveragents/application/services/acms_service.py explicitly filters to a single strategy per assembly call, even though the spec requires all enabled strategies to run in parallel with their results fused.

Code location

src/cleveragents/application/services/acms_service.py lines ~770–800:

candidates = self._strategy_selector.select(all_strategies, {"strategy": strategy_name})
# Ensure the explicitly-requested strategy is among candidates.
# If the selector returned multiple, keep only the requested one
# for v1 single-strategy execution; future multi-strategy support
# will remove this filter.
candidates = [(s, c) for s, c in candidates if s.name == strategy_name] or [(resolved, 1.0)]

The comment even acknowledges this is incorrect: "future multi-strategy support will remove this filter."

The same single-strategy limitation exists in ContextAssemblyPipeline in src/cleveragents/application/services/acms_pipeline.py lines ~600–620.

Expected behavior (from spec)

The Context Assembly Pipeline's StrategyExecutor stage must:

Run all enabled strategies in parallel (via ThreadPoolExecutor)
Collect fragments from all strategies
Pass all fragments through the Fragment Fusion phase (deduplication, scoring, budget packing)

The spec explicitly states: "Executes all enabled context strategies in parallel" and "Multiple strategies run in parallel, and their results are fused by the pipeline."

Actual behavior

Only one strategy runs per assembly call. The StrategySelector may return multiple candidates but the pipeline immediately filters to just the single explicitly-named strategy. This means:

The ConfidenceWeightedSelector and ProportionalBudgetAllocator are effectively unused
Multi-strategy context fusion never occurs
The ParallelStrategyExecutor only ever executes one strategy

Steps to reproduce

Register multiple strategies with ACMSPipeline
Call pipeline.assemble(plan_id, fragments, budget) without specifying a strategy
Observe that only the default_strategy ("relevance") runs, not all registered strategies

Subtasks

Remove the single-strategy filter from ACMSPipeline.assemble() in acms_service.py (~line 780)
Remove the single-strategy filter from ContextAssemblyPipeline in acms_pipeline.py (~line 610)
Verify StrategySelector.select() returns all eligible candidates when no explicit strategy is named
Verify ProportionalBudgetAllocator correctly allocates budget across all returned candidates
Verify ParallelStrategyExecutor executes all candidates concurrently via ThreadPoolExecutor
Verify Fragment Fusion (deduplication, scoring, budget packing) operates on the combined fragment set from all strategies
Tests (Behave): Add scenarios for multi-strategy parallel execution producing fused output
Tests (Behave): Add scenario asserting ConfidenceWeightedSelector and ProportionalBudgetAllocator are exercised when multiple strategies are registered
Tests (Robot): Add integration test confirming multi-strategy assembly produces richer context than single-strategy
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

ACMSPipeline.assemble() passes all candidates from StrategySelector to BudgetAllocator and StrategyExecutor — no single-strategy filter remains
ContextAssemblyPipeline likewise passes all candidates through without filtering
ParallelStrategyExecutor executes more than one strategy concurrently when multiple are registered and enabled
Fragment Fusion operates on the merged fragment set from all strategies
ConfidenceWeightedSelector and ProportionalBudgetAllocator are exercised in the normal assembly path
BDD scenarios cover multi-strategy parallel execution and fusion
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/acms-pipeline-single-strategy-filter` - **Commit Message**: `fix(acms): remove single-strategy filter to enable parallel multi-strategy execution and fusion` - **Milestone**: v3.4.0 - **Parent Epic**: #396 ## Bug Description The `ACMSPipeline.assemble()` method in `src/cleveragents/application/services/acms_service.py` explicitly filters to a single strategy per assembly call, even though the spec requires all enabled strategies to run in parallel with their results fused. ### Code location `src/cleveragents/application/services/acms_service.py` lines ~770–800: ```python candidates = self._strategy_selector.select(all_strategies, {"strategy": strategy_name}) # Ensure the explicitly-requested strategy is among candidates. # If the selector returned multiple, keep only the requested one # for v1 single-strategy execution; future multi-strategy support # will remove this filter. candidates = [(s, c) for s, c in candidates if s.name == strategy_name] or [(resolved, 1.0)] ``` The comment even acknowledges this is incorrect: *"future multi-strategy support will remove this filter."* The same single-strategy limitation exists in `ContextAssemblyPipeline` in `src/cleveragents/application/services/acms_pipeline.py` lines ~600–620. ### Expected behavior (from spec) The Context Assembly Pipeline's `StrategyExecutor` stage must: 1. Run **all** enabled strategies in parallel (via `ThreadPoolExecutor`) 2. Collect fragments from all strategies 3. Pass all fragments through the Fragment Fusion phase (deduplication, scoring, budget packing) The spec explicitly states: *"Executes all enabled context strategies in parallel"* and *"Multiple strategies run in parallel, and their results are fused by the pipeline."* ### Actual behavior Only one strategy runs per assembly call. The `StrategySelector` may return multiple candidates but the pipeline immediately filters to just the single explicitly-named strategy. This means: - The `ConfidenceWeightedSelector` and `ProportionalBudgetAllocator` are effectively unused - Multi-strategy context fusion never occurs - The `ParallelStrategyExecutor` only ever executes one strategy ### Steps to reproduce 1. Register multiple strategies with `ACMSPipeline` 2. Call `pipeline.assemble(plan_id, fragments, budget)` without specifying a strategy 3. Observe that only the `default_strategy` ("relevance") runs, not all registered strategies ## Subtasks - [ ] Remove the single-strategy filter from `ACMSPipeline.assemble()` in `acms_service.py` (~line 780) - [ ] Remove the single-strategy filter from `ContextAssemblyPipeline` in `acms_pipeline.py` (~line 610) - [ ] Verify `StrategySelector.select()` returns all eligible candidates when no explicit strategy is named - [ ] Verify `ProportionalBudgetAllocator` correctly allocates budget across all returned candidates - [ ] Verify `ParallelStrategyExecutor` executes all candidates concurrently via `ThreadPoolExecutor` - [ ] Verify Fragment Fusion (deduplication, scoring, budget packing) operates on the combined fragment set from all strategies - [ ] Tests (Behave): Add scenarios for multi-strategy parallel execution producing fused output - [ ] Tests (Behave): Add scenario asserting `ConfidenceWeightedSelector` and `ProportionalBudgetAllocator` are exercised when multiple strategies are registered - [ ] Tests (Robot): Add integration test confirming multi-strategy assembly produces richer context than single-strategy - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] `ACMSPipeline.assemble()` passes all candidates from `StrategySelector` to `BudgetAllocator` and `StrategyExecutor` — no single-strategy filter remains - [ ] `ContextAssemblyPipeline` likewise passes all candidates through without filtering - [ ] `ParallelStrategyExecutor` executes more than one strategy concurrently when multiple are registered and enabled - [ ] Fragment Fusion operates on the merged fragment set from all strategies - [ ] `ConfidenceWeightedSelector` and `ProportionalBudgetAllocator` are exercised in the normal assembly path - [ ] BDD scenarios cover multi-strategy parallel execution and fusion - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.4.0 milestone

2026-04-05 18:43:25 +00:00

freemo added a new dependency

2026-04-05 18:43:27 +00:00

#396 Epic: ACMS Context Pipeline

freemo referenced this issue

2026-04-05 18:51:11 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 18:52:57 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 19:05:02 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo commented

2026-04-05 19:31:54 +00:00

Author

Owner

Transition of issue #3507 in cleveragents-core from State/Unverified to State/Verified completed via automated label update.

What I did:

Read current labels: State/Unverified, Priority/Critical, Type/Bug, Milestone v3.4.0 (open).
Preconditions: No blocking/Paused condition detected; no further checks needed.
Removed old state labels: Unable to remove existing State/Unverified label via available API (no label-removal endpoint in this workflow). The issue still shows State/Unverified.
Added new state label: State/Verified (ID 1321) added to the issue.
Updated milestone: remains v3.4.0.

Current labels after update: State/Unverified, State/Verified, Priority/Critical, Type/Bug.

Notes and recommended follow-up:

To strictly enforce a single State/* label, please remove the remaining State/Unverified label manually or expose a label-removal API in the workflow. This will prevent duplicate state indicators.
If you want me to re-run to remove State/Unverified once label-removal is available, I can do so automatically.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-state-updater

Transition of issue #3507 in cleveragents-core from State/Unverified to State/Verified completed via automated label update. What I did: - Read current labels: State/Unverified, Priority/Critical, Type/Bug, Milestone v3.4.0 (open). - Preconditions: No blocking/Paused condition detected; no further checks needed. - Removed old state labels: Unable to remove existing State/Unverified label via available API (no label-removal endpoint in this workflow). The issue still shows State/Unverified. - Added new state label: State/Verified (ID 1321) added to the issue. - Updated milestone: remains v3.4.0. Current labels after update: State/Unverified, State/Verified, Priority/Critical, Type/Bug. Notes and recommended follow-up: - To strictly enforce a single State/* label, please remove the remaining State/Unverified label manually or expose a label-removal API in the workflow. This will prevent duplicate state indicators. - If you want me to re-run to remove State/Unverified once label-removal is available, I can do so automatically. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-state-updater

freemo referenced this issue

2026-04-05 19:33:21 +00:00

UAT: Dual incompatible ContextStrategy protocols — domain model and application service define conflicting interfaces #3491

freemo commented

2026-04-05 19:33:33 +00:00

Author

Owner

Issue verified and triaged:

Priority: Critical — ACMSPipeline filters to single strategy per call, violating the spec's multi-strategy parallel execution requirement.
Milestone: v3.4.0 (already assigned)
Story Points: 5 (L) — requires removing single-strategy filter from both ACMSPipeline and ContextAssemblyPipeline, and verifying parallel execution and fusion.
Parent Epic: #396 (already linked)
Dependency: Should be resolved after #3491 (protocol consolidation) to ensure strategies can be registered correctly.
Next step: This issue is now ready for implementation.

Automated by CleverAgents Bot
Supervisor: Human Liaison | Agent: ca-human-liaison

Issue verified and triaged: - **Priority**: Critical — ACMSPipeline filters to single strategy per call, violating the spec's multi-strategy parallel execution requirement. - **Milestone**: v3.4.0 (already assigned) - **Story Points**: 5 (L) — requires removing single-strategy filter from both ACMSPipeline and ContextAssemblyPipeline, and verifying parallel execution and fusion. - **Parent Epic**: #396 (already linked) - **Dependency**: Should be resolved after #3491 (protocol consolidation) to ensure strategies can be registered correctly. - **Next step**: This issue is now ready for implementation. --- **Automated by CleverAgents Bot** Supervisor: Human Liaison | Agent: ca-human-liaison

freemo referenced this issue

2026-04-05 19:49:58 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 20:03:42 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:05:52 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:07:13 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo commented

2026-04-05 22:10:38 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels: Type/Bug, Priority/Critical
Reason: Issue had only State/In Progress label. Per CONTRIBUTING.md, every issue must have exactly one State/*, one Type/*, and one Priority/* label. Based on the issue title and body (UAT bug report), Type/Bug is correct. Priority/Critical was inferred from the severity of the bug (spec-required multi-strategy parallel execution completely missing).

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

Label compliance fix applied: - Added missing labels: `Type/Bug`, `Priority/Critical` - Reason: Issue had only `State/In Progress` label. Per CONTRIBUTING.md, every issue must have exactly one `State/*`, one `Type/*`, and one `Priority/*` label. Based on the issue title and body (UAT bug report), `Type/Bug` is correct. `Priority/Critical` was inferred from the severity of the bug (spec-required multi-strategy parallel execution completely missing). --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

freemo referenced this issue

2026-04-05 22:13:37 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 (Resumed) #3654

freemo removed this from the v3.4.0 milestone

2026-04-06 21:04:23 +00:00