UAT: `agents project context simulate` rich output format does not match spec — missing Strategy Results and Fusion Result panels #5069

New issue

Open

opened 2026-04-09 00:54:13 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 00:54:13 +00:00

Owner

Bug Report

Feature Area: ACMS and Context Management — project context simulate command

Severity: Medium (output format deviation; simulation runs but output structure differs)

What Was Tested

Code-level analysis of src/cleveragents/cli/commands/project_context.py — context_simulate() function (lines 1164–1307).

Expected Behavior (from spec)

The specification (§CLI Commands — agents project context simulate, lines 4346–4384) defines the following rich output format:

╭─ ACMS Simulation ────────────────────────────────────────────────╮
│ Project: local/api-service                                       │
│ View: strategize                                                 │
│ Budget: 16,000 tokens (override)                                 │
│ Focus: uko-py:module/src.auth, uko-py:module/src.db              │
╰──────────────────────────────────────────────────────────────────╯

╭─ Strategy Results ───────────────────────────────────────────────╮
│ breadth-depth-navigator                                          │
│   Confidence: 0.85  │  Budget: 7,200 tokens  │  12 fragments     │
│   Top: src/auth/__init__.py (depth 9, 1,240t, score=0.95)        │
│        src/db/models.py (depth 3, 480t, score=0.88)              │
│                                                                  │
│ semantic-embedding                                               │
│   Confidence: 0.60  │  Budget: 5,600 tokens  │  8 fragments      │
│   Top: src/auth/jwt.py (depth 3, 380t, score=0.82)               │
│        src/middleware/auth_check.py (depth 3, 290t, score=0.79)  │
│                                                                  │
│ simple-keyword                                                   │
│   Confidence: 0.30  │  Budget: 3,200 tokens  │  5 fragments      │
│   Top: src/auth/utils.py (depth 6, 210t, score=0.71)             │
╰──────────────────────────────────────────────────────────────────╯

╭─ Fusion Result ──────────────────────────────────────────────────╮
│ Included: 18 fragments (14,820 tokens, 92.6% of budget)          │
│ Excluded: 7 fragments (3,200 tokens, below budget cutoff)        │
│ Deduplicated: 4 fragments merged                                 │
│ Skeleton: 2,400 tokens (15% reserve)                             │
│ Preamble: 180 tokens                                             │
╰──────────────────────────────────────────────────────────────────╯

✓ OK Simulation complete (dry run — no context was assembled)

Actual Behavior (bug)

The implementation renders:

A summary panel with total_tokens, budget_used %, strategies, fragment count, context hash, and preamble
An Assembled Fragments table with columns: #, UKO Node, Tokens, Relevance, Depth, Strategy
A Per-Phase Context Analysis table (strategize/execute/apply breakdown)

Missing:

ACMS Simulation header panel (project, view, budget, focus URIs)
Strategy Results panel (per-strategy: confidence, budget allocation, fragment count, top fragments with depth/tokens/score)
Fusion Result panel (included/excluded fragment counts, deduplication count, skeleton reserve, preamble tokens)

The "Simulation complete (dry run — no context was assembled)" success message is also absent.

Additionally, the simulation is currently a stub that reads from the tier service rather than actually running the ACMS pipeline in dry-run mode. The spec requires it to show "what the Context Assembly Pipeline would include" — i.e., it should invoke the pipeline with dry_run=True and report per-strategy results.

Code Location

src/cleveragents/cli/commands/project_context.py — context_simulate() function, lines 1164–1307

Fix Required

Update the rich output to render the three spec-required panels: ACMS Simulation header, Strategy Results, and Fusion Result.
The Strategy Results panel must show per-strategy confidence, budget allocation, fragment count, and top 2 fragments with their depth, token count, and relevance score.
The Fusion Result panel must show included/excluded fragment counts, deduplication count, skeleton reserve, and preamble tokens.
The success message should read "Simulation complete (dry run — no context was assembled)".

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** ACMS and Context Management — `project context simulate` command **Severity:** Medium (output format deviation; simulation runs but output structure differs) --- ## What Was Tested Code-level analysis of `src/cleveragents/cli/commands/project_context.py` — `context_simulate()` function (lines 1164–1307). --- ## Expected Behavior (from spec) The specification (§CLI Commands — `agents project context simulate`, lines 4346–4384) defines the following rich output format: ``` ╭─ ACMS Simulation ────────────────────────────────────────────────╮ │ Project: local/api-service │ │ View: strategize │ │ Budget: 16,000 tokens (override) │ │ Focus: uko-py:module/src.auth, uko-py:module/src.db │ ╰──────────────────────────────────────────────────────────────────╯ ╭─ Strategy Results ───────────────────────────────────────────────╮ │ breadth-depth-navigator │ │ Confidence: 0.85 │ Budget: 7,200 tokens │ 12 fragments │ │ Top: src/auth/__init__.py (depth 9, 1,240t, score=0.95) │ │ src/db/models.py (depth 3, 480t, score=0.88) │ │ │ │ semantic-embedding │ │ Confidence: 0.60 │ Budget: 5,600 tokens │ 8 fragments │ │ Top: src/auth/jwt.py (depth 3, 380t, score=0.82) │ │ src/middleware/auth_check.py (depth 3, 290t, score=0.79) │ │ │ │ simple-keyword │ │ Confidence: 0.30 │ Budget: 3,200 tokens │ 5 fragments │ │ Top: src/auth/utils.py (depth 6, 210t, score=0.71) │ ╰──────────────────────────────────────────────────────────────────╯ ╭─ Fusion Result ──────────────────────────────────────────────────╮ │ Included: 18 fragments (14,820 tokens, 92.6% of budget) │ │ Excluded: 7 fragments (3,200 tokens, below budget cutoff) │ │ Deduplicated: 4 fragments merged │ │ Skeleton: 2,400 tokens (15% reserve) │ │ Preamble: 180 tokens │ ╰──────────────────────────────────────────────────────────────────╯ ✓ OK Simulation complete (dry run — no context was assembled) ``` --- ## Actual Behavior (bug) The implementation renders: 1. A **summary panel** with total_tokens, budget_used %, strategies, fragment count, context hash, and preamble 2. An **Assembled Fragments** table with columns: #, UKO Node, Tokens, Relevance, Depth, Strategy 3. A **Per-Phase Context Analysis** table (strategize/execute/apply breakdown) Missing: 1. **ACMS Simulation** header panel (project, view, budget, focus URIs) 2. **Strategy Results** panel (per-strategy: confidence, budget allocation, fragment count, top fragments with depth/tokens/score) 3. **Fusion Result** panel (included/excluded fragment counts, deduplication count, skeleton reserve, preamble tokens) The "Simulation complete (dry run — no context was assembled)" success message is also absent. Additionally, the simulation is currently a stub that reads from the tier service rather than actually running the ACMS pipeline in dry-run mode. The spec requires it to show "what the Context Assembly Pipeline would include" — i.e., it should invoke the pipeline with `dry_run=True` and report per-strategy results. --- ## Code Location `src/cleveragents/cli/commands/project_context.py` — `context_simulate()` function, lines 1164–1307 --- ## Fix Required 1. Update the rich output to render the three spec-required panels: ACMS Simulation header, Strategy Results, and Fusion Result. 2. The Strategy Results panel must show per-strategy confidence, budget allocation, fragment count, and top 2 fragments with their depth, token count, and relevance score. 3. The Fusion Result panel must show included/excluded fragment counts, deduplication count, skeleton reserve, and preamble tokens. 4. The success message should read "Simulation complete (dry run — no context was assembled)". --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 00:54:21 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 01:01:44 +00:00

HAL9000 commented

2026-04-09 01:01:55 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — Spec compliance bug; deviates from documented behavior
Milestone: v3.2.0
Story Points: 3 — M
MoSCoW: Must Have — Spec compliance is required

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — Spec compliance bug; deviates from documented behavior - **Milestone**: v3.2.0 - **Story Points**: 3 — M - **MoSCoW**: Must Have — Spec compliance is required --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 referenced this issue

2026-04-09 02:56:12 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 4) #5160

HAL9000 referenced this issue

2026-04-09 03:09:06 +00:00

[AUTO-UAT-POOL] UAT Testing Report (Cycle 5) #5200