cleveragents/cleveragents-core

Fork 3

UAT: `agents project context simulate` uses stub simulation instead of live ACMS pipeline — does not show token budget breakdown per strategy #5594

New issue

Open

opened 2026-04-09 07:42:49 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 07:42:49 +00:00

Owner

Bug Report

Feature Area: Context Assembly Pipeline — agents project context simulate command

Severity: Critical (milestone-blocking — v3.4.0 deliverable #5: "Returns token budget breakdown without modifying state")

What Was Tested

Code-level analysis of src/cleveragents/cli/commands/project_context.py context_simulate command and _simulate_context_assembly helper against the specification (spec §4325–§4493).

Expected Behavior (from spec §4325–§4493)

agents project context simulate should:

Run a dry-run context assembly using the ACMS pipeline (all 10 stages)
Show token budget breakdown per strategy (e.g., breadth-depth-navigator: 45% of budget, 4 fragments)
Show assembled fragments with UKO URIs, relevance scores, and detail depths
Not modify any state (read-only dry run)

The spec example shows:

╭─ Budget Simulation ──────────────────────────────────────────────╮
│ Total Budget: 16,000 tokens                                      │
│ Skeleton Reserve: 2,400 tokens (15%)                             │
│ Available: 13,600 tokens                                         │
╰──────────────────────────────────────────────────────────────────╯
╭─ Strategy Allocation ────────────────────────────────────────────╮
│ breadth-depth-navigator  45%  6,120 tokens  4 fragments          │
│ semantic-embedding       35%  4,760 tokens  6 fragments          │
│ simple-keyword           20%  2,720 tokens  3 fragments          │
╰──────────────────────────────────────────────────────────────────╯

Actual Behavior (from code)

The _simulate_context_assembly function (line 286 in project_context.py) is a stub that:

Reads existing tier fragments from ContextTierService.get_scoped_view()
Applies a simple token budget cutoff (first-fit, no strategy execution)
Returns a fake AssembledContext with strategy="tier_retrieval" (not a real strategy)
Does not invoke ContextAssemblyPipeline at all

The context_simulate command (line 1164) calls this stub and shows:

Total tokens and budget used percentage
Fragment table (UKO node, tokens, relevance, depth, strategy)
Phase analysis table

Missing:

No strategy allocation breakdown — no per-strategy token budget percentages
No skeleton reserve display — skeleton ratio is read but not shown in output
No real ACMS pipeline execution — strategies are not actually run
No "Budget Simulation" panel with Total Budget, Skeleton Reserve, Available tokens

Code Location

src/cleveragents/cli/commands/project_context.py lines 286–393 (_simulate_context_assembly stub)
src/cleveragents/cli/commands/project_context.py lines 1164–1320 (context_simulate command)

Impact

v3.4.0 deliverable #5 ("Returns token budget breakdown without modifying state") is not met
Strategy allocation percentages are not shown
Skeleton reserve is not displayed
The simulation does not reflect what the real ACMS pipeline would produce

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: Context Assembly Pipeline — `agents project context simulate` command **Severity**: Critical (milestone-blocking — v3.4.0 deliverable #5: "Returns token budget breakdown without modifying state") ### What Was Tested Code-level analysis of `src/cleveragents/cli/commands/project_context.py` `context_simulate` command and `_simulate_context_assembly` helper against the specification (spec §4325–§4493). ### Expected Behavior (from spec §4325–§4493) `agents project context simulate` should: 1. Run a dry-run context assembly using the ACMS pipeline (all 10 stages) 2. Show token budget breakdown **per strategy** (e.g., `breadth-depth-navigator: 45% of budget, 4 fragments`) 3. Show assembled fragments with UKO URIs, relevance scores, and detail depths 4. Not modify any state (read-only dry run) The spec example shows: ``` ╭─ Budget Simulation ──────────────────────────────────────────────╮ │ Total Budget: 16,000 tokens │ │ Skeleton Reserve: 2,400 tokens (15%) │ │ Available: 13,600 tokens │ ╰──────────────────────────────────────────────────────────────────╯ ╭─ Strategy Allocation ────────────────────────────────────────────╮ │ breadth-depth-navigator 45% 6,120 tokens 4 fragments │ │ semantic-embedding 35% 4,760 tokens 6 fragments │ │ simple-keyword 20% 2,720 tokens 3 fragments │ ╰──────────────────────────────────────────────────────────────────╯ ``` ### Actual Behavior (from code) The `_simulate_context_assembly` function (line 286 in `project_context.py`) is a **stub** that: 1. Reads existing tier fragments from `ContextTierService.get_scoped_view()` 2. Applies a simple token budget cutoff (first-fit, no strategy execution) 3. Returns a fake `AssembledContext` with `strategy="tier_retrieval"` (not a real strategy) 4. Does not invoke `ContextAssemblyPipeline` at all The `context_simulate` command (line 1164) calls this stub and shows: - Total tokens and budget used percentage - Fragment table (UKO node, tokens, relevance, depth, strategy) - Phase analysis table Missing: 1. **No strategy allocation breakdown** — no per-strategy token budget percentages 2. **No skeleton reserve display** — skeleton ratio is read but not shown in output 3. **No real ACMS pipeline execution** — strategies are not actually run 4. **No "Budget Simulation" panel** with Total Budget, Skeleton Reserve, Available tokens ### Code Location - `src/cleveragents/cli/commands/project_context.py` lines 286–393 (`_simulate_context_assembly` stub) - `src/cleveragents/cli/commands/project_context.py` lines 1164–1320 (`context_simulate` command) ### Impact - v3.4.0 deliverable #5 ("Returns token budget breakdown without modifying state") is not met - Strategy allocation percentages are not shown - Skeleton reserve is not displayed - The simulation does not reflect what the real ACMS pipeline would produce --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 07:43:23 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 07:45:12 +00:00

HAL9000 added

and removed

labels

2026-04-09 07:46:12 +00:00

HAL9000 commented

2026-04-09 07:46:19 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 referenced this issue

2026-04-09 07:47:09 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 27) #5570

HAL9000 added a new dependency

2026-04-09 07:54:56 +00:00

#5203 EPIC: CLI Output Format Compliance — Global Flags, JSON Envelope & Error Responses

HAL9000 referenced this issue

2026-04-09 07:56:30 +00:00

[AUTO-EPIC] Epic Planning Health Report (Cycle 11) #5298

HAL9000 referenced this issue

2026-04-09 08:20:03 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 1) #5369