UAT: `agents project context simulate` JSON/YAML output uses flat assembled-context structure instead of spec-required `simulation`/`strategy_results`/`fusion_result` keys #6506

New issue

Open

opened 2026-04-09 21:11:44 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-09 21:11:44 +00:00

Owner

Summary

The JSON and YAML output of agents project context simulate uses a flat assembled-context structure instead of the spec-required structure with simulation, strategy_results, and fusion_result keys.

Spec Reference

docs/specification.md §CLI Commands — agents project context simulate JSON output (line ~4428–4490):

{
  "command": "project context simulate",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "simulation": {
      "project": "local/api-service",
      "view": "strategize",
      "budget": 16000,
      "budget_source": "override",
      "focus": ["uko-py:module/src.auth", "uko-py:module/src.db"]
    },
    "strategy_results": [
      {
        "name": "breadth-depth-navigator",
        "confidence": 0.85,
        "budget_tokens": 7200,
        "fragments": 12,
        "top": [...]
      },
      ...
    ],
    "fusion_result": {
      "included_fragments": 18,
      "included_tokens": 14820,
      "budget_utilization": 0.926,
      "excluded_fragments": 7,
      "excluded_tokens": 3200,
      "deduplicated": 4,
      "skeleton_tokens": 2400,
      "preamble_tokens": 180
    }
  }
}

The spec rich output shows:

"ACMS Simulation" panel (project, view, budget, focus)
"Strategy Results" panel (per-strategy: confidence, budget tokens, fragments, top files)
"Fusion Result" panel (included/excluded fragments, deduplication, skeleton, preamble)

Observed Behavior

The context_simulate command (src/cleveragents/cli/commands/project_context.py, line 1164) produces:

{
  "total_tokens": 0,
  "budget_used": 0.0,
  "strategies_used": ["tier_retrieval"],
  "context_hash": "...",
  "preamble": "Simulated context for local/api-service (view: default, budget: 8000 tokens)",
  "fragment_count": 0,
  "fragments": [],
  "project": "local/api-service",
  "view": "default",
  "acms_config": { ... },
  "phase_analysis": { ... }
}

The rich output shows:

"Context Simulation" summary panel (total tokens, budget used, strategies, fragment count, context hash, preamble)
"Assembled Fragments" table
"Per-Phase Context Analysis" table

None of these match the spec-required panels.

Expected Behavior

context simulate should output:

simulation — project, view, budget, budget_source, focus
strategy_results — per-strategy results with confidence, budget_tokens, fragments, top fragments
fusion_result — included/excluded fragments, deduplication, skeleton, preamble

The rich output should show: "ACMS Simulation", "Strategy Results", "Fusion Result" panels.

Additional Note

The current implementation uses _simulate_context_assembly() which is a stub that queries the tier service directly (bypassing the actual ACMS pipeline). The spec requires the simulation to run the full Context Assembly Pipeline in dry-run mode, showing per-strategy contributions.

Code Location

src/cleveragents/cli/commands/project_context.py — context_simulate() (line 1164)
_simulate_context_assembly() (line 286) — stub implementation, not spec-compliant

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Summary The JSON and YAML output of `agents project context simulate` uses a flat assembled-context structure instead of the spec-required structure with `simulation`, `strategy_results`, and `fusion_result` keys. ## Spec Reference `docs/specification.md` §CLI Commands — `agents project context simulate` JSON output (line ~4428–4490): ```json { "command": "project context simulate", "status": "ok", "exit_code": 0, "data": { "simulation": { "project": "local/api-service", "view": "strategize", "budget": 16000, "budget_source": "override", "focus": ["uko-py:module/src.auth", "uko-py:module/src.db"] }, "strategy_results": [ { "name": "breadth-depth-navigator", "confidence": 0.85, "budget_tokens": 7200, "fragments": 12, "top": [...] }, ... ], "fusion_result": { "included_fragments": 18, "included_tokens": 14820, "budget_utilization": 0.926, "excluded_fragments": 7, "excluded_tokens": 3200, "deduplicated": 4, "skeleton_tokens": 2400, "preamble_tokens": 180 } } } ``` The spec rich output shows: - "ACMS Simulation" panel (project, view, budget, focus) - "Strategy Results" panel (per-strategy: confidence, budget tokens, fragments, top files) - "Fusion Result" panel (included/excluded fragments, deduplication, skeleton, preamble) ## Observed Behavior The `context_simulate` command (`src/cleveragents/cli/commands/project_context.py`, line 1164) produces: ```json { "total_tokens": 0, "budget_used": 0.0, "strategies_used": ["tier_retrieval"], "context_hash": "...", "preamble": "Simulated context for local/api-service (view: default, budget: 8000 tokens)", "fragment_count": 0, "fragments": [], "project": "local/api-service", "view": "default", "acms_config": { ... }, "phase_analysis": { ... } } ``` The rich output shows: - "Context Simulation" summary panel (total tokens, budget used, strategies, fragment count, context hash, preamble) - "Assembled Fragments" table - "Per-Phase Context Analysis" table None of these match the spec-required panels. ## Expected Behavior `context simulate` should output: 1. `simulation` — project, view, budget, budget_source, focus 2. `strategy_results` — per-strategy results with confidence, budget_tokens, fragments, top fragments 3. `fusion_result` — included/excluded fragments, deduplication, skeleton, preamble The rich output should show: "ACMS Simulation", "Strategy Results", "Fusion Result" panels. ## Additional Note The current implementation uses `_simulate_context_assembly()` which is a stub that queries the tier service directly (bypassing the actual ACMS pipeline). The spec requires the simulation to run the full Context Assembly Pipeline in dry-run mode, showing per-strategy contributions. ## Code Location - `src/cleveragents/cli/commands/project_context.py` — `context_simulate()` (line 1164) - `_simulate_context_assembly()` (line 286) — stub implementation, not spec-compliant --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 21:31:12 +00:00

HAL9000 referenced this issue

2026-04-09 21:37:14 +00:00

[AUTO-UAT-SUP] UAT Testing Pool Status (Cycle 6) #6372