cleveragents/cleveragents-core

Fork 3

UAT: `agents project context simulate` builds fake context from tier data instead of running the real Context Assembly Pipeline #3516

New issue

Open

opened 2026-04-05 18:46:47 +00:00 by freemo · 2 comments

freemo commented

2026-04-05 18:46:47 +00:00

Owner

Metadata

Branch: fix/m5-simulate-context-assembly-pipeline
Commit Message: fix(cli): wire simulate command to real ContextAssemblyPipeline
Milestone: v3.4.0
Parent Epic: #396

Background

The agents project context simulate CLI command is supposed to run a dry-run of the Context Assembly Pipeline to show what context would be assembled. Instead, it builds a fake AssembledContext from tier storage data, bypassing the pipeline entirely.

Code location: src/cleveragents/cli/commands/project_context.py — the _simulate_context_assembly() function (lines ~200-280)

What the implementation does:

def _simulate_context_assembly(...) -> AssembledContext:
    # Queries tier service for existing fragments
    project_fragments = tier_service.get_scoped_view([project_name])
    # Manually builds AssembledContext from tier data
    # Does NOT call ContextAssemblyPipeline.assemble()
    return AssembledContext(fragments=..., context_hash=hashlib.sha256(...))

Expected behavior (from spec):
The simulate command should:

Build a ContextRequest from the project's ACMS config and any --focus URIs
Call ContextAssemblyPipeline.assemble() with the request
Show which strategies contributed, how the budget was allocated, and what was excluded
Be a true dry-run that reflects what would actually happen during plan execution

The spec says: "Produces a dry-run report showing what the Context Assembly Pipeline would include in the context payload, which strategies contributed, how the budget was allocated, and what was excluded."

Actual behavior:
The simulate command:

Fetches existing tier fragments (not running any strategy)
Manually constructs a fake hash from f"{project_name}:{view}:{total_tokens}"
Reports tier_retrieval as the strategy used (not a real strategy)
Does not show which strategies would have been selected or their confidence scores
Does not show what would have been excluded due to budget constraints

Steps to reproduce:

Run agents project context simulate <project_name>
Observe that the output shows tier_retrieval as the strategy and a hash based on project name + token count
Note that no actual pipeline execution occurs

Subtasks

Audit _simulate_context_assembly() in src/cleveragents/cli/commands/project_context.py and document all deviations from the spec
Build a ContextRequest from the project's ACMS config and any --focus URIs passed via CLI
Wire _simulate_context_assembly() to call ContextAssemblyPipeline.assemble() with the constructed ContextRequest
Implement dry-run / read-only mode in ContextAssemblyPipeline (or pass a flag) so no side-effects occur during simulate
Update simulate output to report: contributing strategies, confidence scores, budget allocation, and excluded items
Remove the fake tier_retrieval strategy label and the manually constructed hash
Tests (Behave): Add/update scenarios for simulate command verifying real pipeline is invoked
Tests (Robot): Add integration test confirming simulate output reflects actual pipeline strategy selection
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
agents project context simulate <project> invokes ContextAssemblyPipeline.assemble() and reports real strategy contributions, budget allocation, and exclusions — not fabricated tier data.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/m5-simulate-context-assembly-pipeline` - **Commit Message**: `fix(cli): wire simulate command to real ContextAssemblyPipeline` - **Milestone**: v3.4.0 - **Parent Epic**: #396 ## Background The `agents project context simulate` CLI command is supposed to run a dry-run of the Context Assembly Pipeline to show what context would be assembled. Instead, it builds a fake `AssembledContext` from tier storage data, bypassing the pipeline entirely. **Code location:** `src/cleveragents/cli/commands/project_context.py` — the `_simulate_context_assembly()` function (lines ~200-280) **What the implementation does:** ```python def _simulate_context_assembly(...) -> AssembledContext: # Queries tier service for existing fragments project_fragments = tier_service.get_scoped_view([project_name]) # Manually builds AssembledContext from tier data # Does NOT call ContextAssemblyPipeline.assemble() return AssembledContext(fragments=..., context_hash=hashlib.sha256(...)) ``` **Expected behavior (from spec):** The `simulate` command should: 1. Build a `ContextRequest` from the project's ACMS config and any `--focus` URIs 2. Call `ContextAssemblyPipeline.assemble()` with the request 3. Show which strategies contributed, how the budget was allocated, and what was excluded 4. Be a true dry-run that reflects what would actually happen during plan execution The spec says: "Produces a dry-run report showing what the Context Assembly Pipeline would include in the context payload, which strategies contributed, how the budget was allocated, and what was excluded." **Actual behavior:** The simulate command: - Fetches existing tier fragments (not running any strategy) - Manually constructs a fake hash from `f"{project_name}:{view}:{total_tokens}"` - Reports `tier_retrieval` as the strategy used (not a real strategy) - Does not show which strategies would have been selected or their confidence scores - Does not show what would have been excluded due to budget constraints **Steps to reproduce:** 1. Run `agents project context simulate <project_name>` 2. Observe that the output shows `tier_retrieval` as the strategy and a hash based on project name + token count 3. Note that no actual pipeline execution occurs ## Subtasks - [ ] Audit `_simulate_context_assembly()` in `src/cleveragents/cli/commands/project_context.py` and document all deviations from the spec - [ ] Build a `ContextRequest` from the project's ACMS config and any `--focus` URIs passed via CLI - [ ] Wire `_simulate_context_assembly()` to call `ContextAssemblyPipeline.assemble()` with the constructed `ContextRequest` - [ ] Implement dry-run / read-only mode in `ContextAssemblyPipeline` (or pass a flag) so no side-effects occur during simulate - [ ] Update simulate output to report: contributing strategies, confidence scores, budget allocation, and excluded items - [ ] Remove the fake `tier_retrieval` strategy label and the manually constructed hash - [ ] Tests (Behave): Add/update scenarios for `simulate` command verifying real pipeline is invoked - [ ] Tests (Robot): Add integration test confirming simulate output reflects actual pipeline strategy selection - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - `agents project context simulate <project>` invokes `ContextAssemblyPipeline.assemble()` and reports real strategy contributions, budget allocation, and exclusions — not fabricated tier data. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.4.0 milestone

2026-04-05 18:46:52 +00:00

freemo added a new dependency

2026-04-05 18:47:00 +00:00

#396 Epic: ACMS Context Pipeline

freemo referenced this issue

2026-04-05 18:51:11 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 18:52:57 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 19:05:02 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo commented

2026-04-05 19:33:36 +00:00

Author

Owner

Issue verified and triaged:

Priority: Critical — simulate command builds fake context from tier data instead of running the real Context Assembly Pipeline.
Milestone: v3.4.0 (already assigned)
Story Points: 5 (L) — requires wiring to real pipeline, implementing dry-run mode, and updating output format.
Parent Epic: #396 (already linked)
Next step: This issue is now ready for implementation.

Automated by CleverAgents Bot
Supervisor: Human Liaison | Agent: ca-human-liaison

Issue verified and triaged: - **Priority**: Critical — simulate command builds fake context from tier data instead of running the real Context Assembly Pipeline. - **Milestone**: v3.4.0 (already assigned) - **Story Points**: 5 (L) — requires wiring to real pipeline, implementing dry-run mode, and updating output format. - **Parent Epic**: #396 (already linked) - **Next step**: This issue is now ready for implementation. --- **Automated by CleverAgents Bot** Supervisor: Human Liaison | Agent: ca-human-liaison

freemo referenced this issue

2026-04-05 19:49:58 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3377

freemo referenced this issue

2026-04-05 20:03:42 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:05:52 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo referenced this issue

2026-04-05 20:07:13 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572

freemo commented

2026-04-05 20:11:48 +00:00

Author

Owner

Starting implementation on branch fix/context-simulate-fake-vs-real-pipeline.

Analysis complete. The _simulate_context_assembly() function in src/cleveragents/cli/commands/project_context.py (lines 286–385) manually builds a fake AssembledContext from tier storage data instead of invoking the real ContextAssemblyPipeline. Key deviations:

Uses tier_retrieval as a fake strategy label
Manually constructs a SHA-256 hash from project name + view + token count
Builds ContextFragment objects directly from TieredFragment tier data
Does not call ContextAssemblyPipeline.assemble() at all

Plan:

Wire _simulate_context_assembly() to call ContextAssemblyPipeline.assemble() with a constructed ContextRequest
Implement dry-run mode by passing a dry_run=True flag or using the pipeline without side-effects
Update output to report contributing strategies, confidence scores, budget allocation, and excluded items
Remove fake tier_retrieval strategy label and manually constructed hash
Add/update Behave tests verifying real pipeline is invoked
Add Robot integration test

Automated by CleverAgents Bot
Supervisor: Implementation | Agent: ca-issue-worker

Starting implementation on branch `fix/context-simulate-fake-vs-real-pipeline`. **Analysis complete.** The `_simulate_context_assembly()` function in `src/cleveragents/cli/commands/project_context.py` (lines 286–385) manually builds a fake `AssembledContext` from tier storage data instead of invoking the real `ContextAssemblyPipeline`. Key deviations: 1. Uses `tier_retrieval` as a fake strategy label 2. Manually constructs a SHA-256 hash from project name + view + token count 3. Builds `ContextFragment` objects directly from `TieredFragment` tier data 4. Does not call `ContextAssemblyPipeline.assemble()` at all **Plan:** - Wire `_simulate_context_assembly()` to call `ContextAssemblyPipeline.assemble()` with a constructed `ContextRequest` - Implement dry-run mode by passing a `dry_run=True` flag or using the pipeline without side-effects - Update output to report contributing strategies, confidence scores, budget allocation, and excluded items - Remove fake `tier_retrieval` strategy label and manually constructed hash - Add/update Behave tests verifying real pipeline is invoked - Add Robot integration test --- **Automated by CleverAgents Bot** Supervisor: Implementation | Agent: ca-issue-worker

freemo removed this from the v3.4.0 milestone

2026-04-06 21:04:23 +00:00