UAT: ACMS pipeline skips backend calls for spec strategies #6978

New issue

Open

opened 2026-04-10 06:10:13 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-10 06:10:13 +00:00

Owner

What was tested

Instantiated SimpleKeywordStrategy from the spec stubs with a recording TextBackend and invoked assemble() directly.
Ran ContextAssemblyPipeline with strategy="simple-keyword" using the same ContextRequest and fragments while the recording backend was registered.

Expected behavior

When the pipeline executes a spec strategy (keyword/semantic/graph/temporal), it should invoke the appropriate backend through the Context Request Protocol so the strategy can retrieve fragments per spec §25207-25216.

Actual behavior

The pipeline delegates to SpecStrategyAdapter, which ignores the supplied ContextRequest/BackendSet and simply re-ranks the pre-fetched fragments. The recording backend is never called, so no retrieval happens. This breaks the spec requirement that these strategies query their backends (text/vector/graph/temporal).

Steps to reproduce

Create a RecordingTextBackend that records calls to search().
Call SimpleKeywordStrategy().assemble() directly with that backend — note that one backend call is recorded.
Instantiate ContextAssemblyPipeline(default_strategy='simple-keyword') and call assemble() with the same ContextRequest and fragments.
Observe that the backend call count does not increase; the pipeline returns the original fragment without ever invoking the backend.

Code location

src/cleveragents/application/services/acms_service.py (SpecStrategyAdapter and ContextAssemblyPipeline)

## What was tested - Instantiated `SimpleKeywordStrategy` from the spec stubs with a recording `TextBackend` and invoked `assemble()` directly. - Ran `ContextAssemblyPipeline` with `strategy="simple-keyword"` using the same `ContextRequest` and fragments while the recording backend was registered. ## Expected behavior - When the pipeline executes a spec strategy (keyword/semantic/graph/temporal), it should invoke the appropriate backend through the Context Request Protocol so the strategy can retrieve fragments per spec §25207-25216. ## Actual behavior - The pipeline delegates to `SpecStrategyAdapter`, which ignores the supplied `ContextRequest`/`BackendSet` and simply re-ranks the pre-fetched fragments. The recording backend is never called, so no retrieval happens. This breaks the spec requirement that these strategies query their backends (text/vector/graph/temporal). ## Steps to reproduce 1. Create a `RecordingTextBackend` that records calls to `search()`. 2. Call `SimpleKeywordStrategy().assemble()` directly with that backend — note that one backend call is recorded. 3. Instantiate `ContextAssemblyPipeline(default_strategy='simple-keyword')` and call `assemble()` with the same `ContextRequest` and fragments. 4. Observe that the backend call count does **not** increase; the pipeline returns the original fragment without ever invoking the backend. ## Code location - `src/cleveragents/application/services/acms_service.py` (`SpecStrategyAdapter` and `ContextAssemblyPipeline`)

Rows
Columns