cleveragents/cleveragents-core

Fork 3

UAT: `agents project context simulate` bypasses live ContextAssemblyPipeline — uses tier-data stub instead #6330

New issue

Open

opened 2026-04-09 20:10:56 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 20:10:56 +00:00

Owner

Bug Report

Spec Reference

docs/specification.md — ACMS > Context Assembly Pipeline; ADR-014

The spec defines agents project context simulate <PROJECT> as a command that simulates context assembly, implying it exercises the real 10-stage pipeline to show fragment selection, budget usage, and strategy selection.

Code Location

File: /app/src/cleveragents/cli/commands/project_context.py
Lines: 286–380 (_simulate_context_assembly function), 1236 (caller)

Finding

_simulate_context_assembly() explicitly bypasses the live ContextAssemblyPipeline. The function docstring admits this:

def _simulate_context_assembly(...) -> AssembledContext:
    """Run a dry-run context assembly using the CRP models.

    This creates a simulated AssembledContext based on available
    backend data and the project's ACMS configuration.  When the
    full ACMS pipeline is wired, this will delegate to the live
    pipeline instead.
    """

Instead of calling ContextAssemblyPipeline.assemble(), the function:

Manually queries tier_service.get_scoped_view([project_name]) (line 338)
Builds ContextFragment objects directly from tier data (lines 341–364)
Returns a hand-crafted AssembledContext with strategy="tier_retrieval" (line 377)

Impact

All 10 pipeline stages are skipped during simulation:

Phase 1: ConfidenceWeightedSelector, ProportionalBudgetAllocator, ParallelStrategyExecutor — skipped
Phase 2: ContentHashDeduplicator, MaxDepthResolver, WeightedCompositeScorer, GreedyKnapsackPacker, RelevanceCoherenceOrderer — skipped
Phase 3: ProvenancePreambleGenerator, DepthReductionCompressor — skipped

The --strategy hint passed to context simulate is stored but never passed to any real strategy object. Budget enforcement, depth resolution, relevance scoring, deduplication, and skeleton compression output will be absent or incorrect.

Steps to Reproduce

agents project context simulate local/my-project --strategy semantic

Expected

The command invokes ContextAssemblyPipeline.assemble() (from acms_pipeline.py) with a dry-run ContextBudget, returning real fragment selections reflecting strategy orchestration, deduplication, scoring, packing, and preamble generation.

Actual

The command reads raw tier fragments and wraps them in a simulated AssembledContext, reporting strategies_used=["tier_retrieval"] regardless of the --strategy argument. None of the 10 pipeline components execute.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report ### Spec Reference `docs/specification.md` — ACMS > Context Assembly Pipeline; ADR-014 The spec defines `agents project context simulate <PROJECT>` as a command that **simulates context assembly**, implying it exercises the real 10-stage pipeline to show fragment selection, budget usage, and strategy selection. ### Code Location **File**: `/app/src/cleveragents/cli/commands/project_context.py` **Lines**: 286–380 (`_simulate_context_assembly` function), 1236 (caller) ### Finding `_simulate_context_assembly()` explicitly bypasses the live `ContextAssemblyPipeline`. The function docstring admits this: ```python def _simulate_context_assembly(...) -> AssembledContext: """Run a dry-run context assembly using the CRP models. This creates a simulated AssembledContext based on available backend data and the project's ACMS configuration. When the full ACMS pipeline is wired, this will delegate to the live pipeline instead. """ ``` Instead of calling `ContextAssemblyPipeline.assemble()`, the function: 1. Manually queries `tier_service.get_scoped_view([project_name])` (line 338) 2. Builds `ContextFragment` objects directly from tier data (lines 341–364) 3. Returns a hand-crafted `AssembledContext` with `strategy="tier_retrieval"` (line 377) ### Impact All 10 pipeline stages are skipped during simulation: - Phase 1: `ConfidenceWeightedSelector`, `ProportionalBudgetAllocator`, `ParallelStrategyExecutor` — **skipped** - Phase 2: `ContentHashDeduplicator`, `MaxDepthResolver`, `WeightedCompositeScorer`, `GreedyKnapsackPacker`, `RelevanceCoherenceOrderer` — **skipped** - Phase 3: `ProvenancePreambleGenerator`, `DepthReductionCompressor` — **skipped** The `--strategy` hint passed to `context simulate` is stored but never passed to any real strategy object. Budget enforcement, depth resolution, relevance scoring, deduplication, and skeleton compression output will be absent or incorrect. ### Steps to Reproduce ```bash agents project context simulate local/my-project --strategy semantic ``` ### Expected The command invokes `ContextAssemblyPipeline.assemble()` (from `acms_pipeline.py`) with a dry-run `ContextBudget`, returning real fragment selections reflecting strategy orchestration, deduplication, scoring, packing, and preamble generation. ### Actual The command reads raw tier fragments and wraps them in a simulated `AssembledContext`, reporting `strategies_used=["tier_retrieval"]` regardless of the `--strategy` argument. None of the 10 pipeline components execute. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-09 20:18:09 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 3) #6294

HAL9000 added the

labels

2026-04-09 20:18:45 +00:00

HAL9000 referenced this issue

2026-04-09 21:00:24 +00:00

[AUTO-ARCH] Architecture Status (Cycle 9) #6396

HAL9000 added

and removed

labels

2026-04-09 21:23:15 +00:00

HAL9000 referenced this issue

2026-04-09 21:25:38 +00:00

[AUTO-IMP-SUP] Implementation Pool Supervisor Status (Cycle 1) #6563

HAL9000 commented

2026-04-09 21:27:52 +00:00

Author

Owner

Starting implementation on branch fix/issue-6330-project-context-simulate-pipeline. Difficulty assessment: medium → starting at codex tier.

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: implementation-worker

Starting implementation on branch `fix/issue-6330-project-context-simulate-pipeline`. Difficulty assessment: medium → starting at codex tier. --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: implementation-worker

HAL9000 referenced this issue

2026-04-14 07:45:02 +00:00

[AUTO-UAT-POOL] Status: UAT Test Pool Supervisor (Cycle 1) #9113