cleveragents/cleveragents-core

Fork 3

UAT: ACMSPipeline ContextStrategy protocol uses `dict[str, Any]` instead of spec-required `ContextRequest + BackendSet` #4774

New issue

Open

opened 2026-04-08 18:55:59 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-08 18:55:59 +00:00

Owner

Bug Report

Feature Area: ACMS — Context Assembly Pipeline, ContextStrategy Protocol

Severity: High — the pipeline's internal ContextStrategy protocol is incompatible with the spec-defined protocol, preventing real strategy implementations from being used directly in the pipeline

What Was Tested

Code-level analysis comparing the ContextStrategy protocol in src/cleveragents/application/services/acms_service.py against the spec definition at docs/specification.md §25634-25658 and the domain model in src/cleveragents/domain/models/acms/strategy.py.

Expected Behavior (from spec)

The spec defines the ContextStrategy protocol at docs/specification.md lines 25634-25658:

@runtime_checkable
class ContextStrategy(Protocol):
    @property
    def name(self) -> str: ...

    @property
    def capabilities(self) -> StrategyCapabilities: ...

    def can_handle(self, request: ContextRequest, backends: BackendSet) -> float:
        """Returns 0.0-1.0 confidence that this strategy can usefully
        contribute to this request."""
        ...

    def assemble(self, request: ContextRequest, backends: BackendSet,
                 budget: int, plan_context: PlanContext) -> list[ContextFragment]:
        """Execute the strategy. Must respect the budget."""
        ...

    def explain(self) -> str: ...

The domain model in src/cleveragents/domain/models/acms/strategy.py correctly implements this protocol.

Actual Behavior

The ACMSPipeline in src/cleveragents/application/services/acms_service.py defines a different ContextStrategy protocol (lines 118-148):

class ContextStrategy(Protocol):
    @property
    def name(self) -> str: ...

    @property
    def capabilities(self) -> StrategyCapabilities: ...

    def can_handle(self, request: dict[str, Any]) -> float:
        # ❌ WRONG: takes dict instead of ContextRequest + BackendSet
        ...

    def assemble(
        self,
        fragments: Sequence[ContextFragment],
        budget: ContextBudget,
    ) -> Sequence[ContextFragment]:
        # ❌ WRONG: takes pre-fetched fragments instead of ContextRequest + BackendSet + budget + plan_context
        ...

This means:

The pipeline's ContextStrategy protocol is incompatible with the spec-defined protocol
The 6 built-in strategies in strategy_stubs.py (which implement the correct spec protocol) cannot be used directly in the pipeline — they must be wrapped in SpecStrategyAdapter (lines 286-353)
The SpecStrategyAdapter falls back to relevance-based ranking of pre-fetched fragments, bypassing the actual strategy logic (text search, vector search, graph traversal)
The ConfidenceWeightedSelector calls strategy.can_handle(request) with a dict (line 170), not a ContextRequest + BackendSet as the spec requires

Impact

The 6 spec-required built-in strategies (simple-keyword, semantic-embedding, breadth-depth-navigator, arce, temporal-archaeology, plan-decision-context) are not actually executed when the pipeline runs — they are wrapped in adapters that ignore their actual logic
The pipeline cannot leverage text backends, vector backends, or graph backends through the strategy protocol
Custom strategies implementing the spec protocol cannot be registered directly in the pipeline
The SpecStrategyAdapter.assemble() always falls back to relevance-based ranking regardless of which strategy is "selected"

Code Locations

src/cleveragents/application/services/acms_service.py lines 118-148 (wrong protocol)
src/cleveragents/application/services/acms_service.py lines 286-353 (SpecStrategyAdapter workaround)
src/cleveragents/domain/models/acms/strategy.py lines 194-243 (correct spec protocol)
src/cleveragents/domain/models/acms/strategy_stubs.py (correct implementations)
Issue #3491 is referenced in the code as the tracking issue for this protocol consolidation

Fix Required

The pipeline needs to be refactored to use the spec-defined ContextStrategy protocol:

The ACMSPipeline.assemble() method should pass ContextRequest and BackendSet to strategies
The StrategySelector, BudgetAllocator, and StrategyExecutor should work with the spec protocol
The SpecStrategyAdapter workaround should be removed
The ConfidenceWeightedSelector.select() should pass ContextRequest + BackendSet to can_handle()

This is tracked as issue #3491 in the codebase but needs to be prioritized as it prevents the ACMS from functioning as specified.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** ACMS — Context Assembly Pipeline, ContextStrategy Protocol **Severity:** High — the pipeline's internal `ContextStrategy` protocol is incompatible with the spec-defined protocol, preventing real strategy implementations from being used directly in the pipeline ### What Was Tested Code-level analysis comparing the `ContextStrategy` protocol in `src/cleveragents/application/services/acms_service.py` against the spec definition at `docs/specification.md` §25634-25658 and the domain model in `src/cleveragents/domain/models/acms/strategy.py`. ### Expected Behavior (from spec) The spec defines the `ContextStrategy` protocol at `docs/specification.md` lines 25634-25658: ```python @runtime_checkable class ContextStrategy(Protocol): @property def name(self) -> str: ... @property def capabilities(self) -> StrategyCapabilities: ... def can_handle(self, request: ContextRequest, backends: BackendSet) -> float: """Returns 0.0-1.0 confidence that this strategy can usefully contribute to this request.""" ... def assemble(self, request: ContextRequest, backends: BackendSet, budget: int, plan_context: PlanContext) -> list[ContextFragment]: """Execute the strategy. Must respect the budget.""" ... def explain(self) -> str: ... ``` The domain model in `src/cleveragents/domain/models/acms/strategy.py` correctly implements this protocol. ### Actual Behavior The `ACMSPipeline` in `src/cleveragents/application/services/acms_service.py` defines a **different** `ContextStrategy` protocol (lines 118-148): ```python class ContextStrategy(Protocol): @property def name(self) -> str: ... @property def capabilities(self) -> StrategyCapabilities: ... def can_handle(self, request: dict[str, Any]) -> float: # ❌ WRONG: takes dict instead of ContextRequest + BackendSet ... def assemble( self, fragments: Sequence[ContextFragment], budget: ContextBudget, ) -> Sequence[ContextFragment]: # ❌ WRONG: takes pre-fetched fragments instead of ContextRequest + BackendSet + budget + plan_context ... ``` This means: 1. The pipeline's `ContextStrategy` protocol is **incompatible** with the spec-defined protocol 2. The 6 built-in strategies in `strategy_stubs.py` (which implement the correct spec protocol) **cannot be used directly** in the pipeline — they must be wrapped in `SpecStrategyAdapter` (lines 286-353) 3. The `SpecStrategyAdapter` falls back to relevance-based ranking of pre-fetched fragments, bypassing the actual strategy logic (text search, vector search, graph traversal) 4. The `ConfidenceWeightedSelector` calls `strategy.can_handle(request)` with a `dict` (line 170), not a `ContextRequest + BackendSet` as the spec requires ### Impact - The 6 spec-required built-in strategies (`simple-keyword`, `semantic-embedding`, `breadth-depth-navigator`, `arce`, `temporal-archaeology`, `plan-decision-context`) are **not actually executed** when the pipeline runs — they are wrapped in adapters that ignore their actual logic - The pipeline cannot leverage text backends, vector backends, or graph backends through the strategy protocol - Custom strategies implementing the spec protocol cannot be registered directly in the pipeline - The `SpecStrategyAdapter.assemble()` always falls back to relevance-based ranking regardless of which strategy is "selected" ### Code Locations - `src/cleveragents/application/services/acms_service.py` lines 118-148 (wrong protocol) - `src/cleveragents/application/services/acms_service.py` lines 286-353 (`SpecStrategyAdapter` workaround) - `src/cleveragents/domain/models/acms/strategy.py` lines 194-243 (correct spec protocol) - `src/cleveragents/domain/models/acms/strategy_stubs.py` (correct implementations) - Issue #3491 is referenced in the code as the tracking issue for this protocol consolidation ### Fix Required The pipeline needs to be refactored to use the spec-defined `ContextStrategy` protocol: 1. The `ACMSPipeline.assemble()` method should pass `ContextRequest` and `BackendSet` to strategies 2. The `StrategySelector`, `BudgetAllocator`, and `StrategyExecutor` should work with the spec protocol 3. The `SpecStrategyAdapter` workaround should be removed 4. The `ConfidenceWeightedSelector.select()` should pass `ContextRequest + BackendSet` to `can_handle()` This is tracked as issue #3491 in the codebase but needs to be prioritized as it prevents the ACMS from functioning as specified. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-08 18:57:10 +00:00

UAT: Duplicate StrategyCapabilities class definitions with incompatible fields across ACMS modules #4782

HAL9000 referenced this issue

2026-04-08 18:57:54 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4743

HAL9000 referenced this issue

2026-04-08 20:08:39 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 added the

labels

2026-04-08 23:00:34 +00:00

HAL9000 commented

2026-04-09 03:04:10 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — spec compliance bug identified by UAT testing
Story Points: 3 (M) — targeted fix to align implementation with spec
MoSCoW: Must Have — spec compliance is required for correct system behavior

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — spec compliance bug identified by UAT testing - **Story Points**: 3 (M) — targeted fix to align implementation with spec - **MoSCoW**: Must Have — spec compliance is required for correct system behavior --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 added

and removed

labels

2026-04-09 03:04:22 +00:00

HAL9000 added this to the v3.5.0 milestone