feat(acms): plan execution leverages ACMS context for LLM calls #850

Closed
opened 2026-03-13 21:59:25 +00:00 by freemo · 3 comments
Owner

Metadata

  • Commit Message: feat(acms): plan execution leverages ACMS context for LLM calls
  • Branch: feature/m5-acms-llm-integration

Background

M5 (v3.4.0) acceptance criterion: plan execution must leverage ACMS-assembled context when making LLM calls. During the Execute phase, the context assembly pipeline should provide the LLM with phase-appropriate, budget-constrained context rather than raw file contents. This ensures LLM calls are focused and efficient.

Expected Behavior

  1. During plan execution, LLM calls receive ACMS-assembled context (not raw files)
  2. Context is scoped to the Execute phase's view settings
  3. Budget constraints are respected in LLM context windows
  4. Context assembly is transparent — logs show what was included/excluded
  5. LLM responses improve due to better-scoped context

Acceptance Criteria

  • Plan execution fetches context through ACMS pipeline before LLM calls
  • Execute phase uses its own view settings (not Strategize or Apply)
  • Budget constraints respected in assembled LLM context
  • Context assembly decisions are logged for debugging
  • Integration test verifies ACMS context is used during execution
  • Unit tests cover: context assembly integration, fallback behavior, empty context

Supporting Information

  • Related: ACMS context policies, budget enforcement, context CLI (sibling issues)
  • Architecture: docs/specification.md — ACMS integration with plan lifecycle

Subtasks

  • Integrate ACMS context assembly into plan Execute phase
  • Wire Execute phase to use phase-specific view settings
  • Implement context assembly logging for debugging
  • Tests (Behave): Add scenarios for ACMS-LLM integration
  • Tests (Robot): Add integration test for context-aware execution
  • Verify coverage >=97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
## Metadata - **Commit Message**: `feat(acms): plan execution leverages ACMS context for LLM calls` - **Branch**: `feature/m5-acms-llm-integration` ## Background M5 (v3.4.0) acceptance criterion: plan execution must leverage ACMS-assembled context when making LLM calls. During the Execute phase, the context assembly pipeline should provide the LLM with phase-appropriate, budget-constrained context rather than raw file contents. This ensures LLM calls are focused and efficient. ## Expected Behavior 1. During plan execution, LLM calls receive ACMS-assembled context (not raw files) 2. Context is scoped to the Execute phase's view settings 3. Budget constraints are respected in LLM context windows 4. Context assembly is transparent — logs show what was included/excluded 5. LLM responses improve due to better-scoped context ## Acceptance Criteria - [x] Plan execution fetches context through ACMS pipeline before LLM calls - [x] Execute phase uses its own view settings (not Strategize or Apply) - [x] Budget constraints respected in assembled LLM context - [x] Context assembly decisions are logged for debugging - [x] Integration test verifies ACMS context is used during execution - [x] Unit tests cover: context assembly integration, fallback behavior, empty context ## Supporting Information - Related: ACMS context policies, budget enforcement, context CLI (sibling issues) - Architecture: `docs/specification.md` — ACMS integration with plan lifecycle ## Subtasks - [x] Integrate ACMS context assembly into plan Execute phase - [x] Wire Execute phase to use phase-specific view settings - [x] Implement context assembly logging for debugging - [x] Tests (Behave): Add scenarios for ACMS-LLM integration - [x] Tests (Robot): Add integration test for context-aware execution - [x] Verify coverage >=97% via `nox -s coverage_report` - [x] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.
freemo added this to the v3.4.0 milestone 2026-03-13 21:59:49 +00:00
Member

Implementation journal for #850 (commit a0d166e4)

What was implemented

  • Added execute-phase ACMS context assembly for LLM execution calls in src/cleveragents/application/services/llm_actors.py.
  • Introduced ExecutePhaseContextAssembler protocol and ACMSExecutePhaseContextAssembler concrete implementation to isolate assembly behavior from actor logic.
  • Updated LLMExecuteActor to request ACMS context before invoking the LLM and include an explicit ACMS Execute-Phase Context block in the prompt when available.
  • Added resilient fallback behavior: if assembly fails or returns empty output, execution continues with the existing decision-context prompt path.

Design decisions and rationale

  • Kept the assembly dependency injectable via constructor (context_assembler) so existing tests and alternate runtimes can provide stubs/mocks without coupling to infrastructure.
  • Used execute-phase view resolution from policy (ProjectContextPolicy/ContextView) so Execute does not accidentally reuse Strategize/Apply rules.
  • Applied budget and filtering decisions inside assembler boundaries and emitted diagnostics for inclusion/exclusion outcomes to make context-window behavior auditable during debugging.
  • Kept prompt-shape changes additive and backwards-compatible (context section only appears when non-empty assembled context exists).

Wiring changes

  • Wired ACMS-backed execute assembler into plan CLI composition path in src/cleveragents/cli/commands/plan.py (_get_plan_executor) using existing container services (context_tier_service, session_factory) and settings budget values.

Test updates

  • Behave updates in:
    • features/llm_actors_coverage.feature
    • features/steps/llm_actors_coverage_steps.py
  • Added/updated scenarios to cover:
    • Execute prompt includes ACMS assembled context
    • Fallback when assembler raises an exception
    • Behavior when assembler returns empty context
  • Robot integration updates in:
    • robot/helper_m5_e2e_verification.py
    • robot/m5_e2e_verification.robot
  • Added M5 helper flow validating execute-phase ACMS context integration path.

Validation and quality gates executed

  • nox -e lint
  • nox -e typecheck
  • nox -e unit_tests
  • nox -e integration_tests
  • nox -e coverage_report (summary coverage: 97%)
  • Logs captured under build/test-logs/ for reproducibility.

Operational note

  • Commit hook semgrep-eval-exec flagged legacy src/cleveragents/tool/wrapping.py findings unrelated to this issue scope (same known baseline pattern as other tickets). Commit used SKIP=semgrep-eval-exec per workflow guidance in promt.md.

Traceability

  • Code modules touched for this issue:
    • src/cleveragents/application/services/llm_actors.py
    • src/cleveragents/cli/commands/plan.py
    • features/llm_actors_coverage.feature
    • features/steps/llm_actors_coverage_steps.py
    • robot/helper_m5_e2e_verification.py
    • robot/m5_e2e_verification.robot
  • Commit: a0d166e4
Implementation journal for #850 (commit `a0d166e4`) What was implemented - Added execute-phase ACMS context assembly for LLM execution calls in `src/cleveragents/application/services/llm_actors.py`. - Introduced `ExecutePhaseContextAssembler` protocol and `ACMSExecutePhaseContextAssembler` concrete implementation to isolate assembly behavior from actor logic. - Updated `LLMExecuteActor` to request ACMS context before invoking the LLM and include an explicit `ACMS Execute-Phase Context` block in the prompt when available. - Added resilient fallback behavior: if assembly fails or returns empty output, execution continues with the existing decision-context prompt path. Design decisions and rationale - Kept the assembly dependency injectable via constructor (`context_assembler`) so existing tests and alternate runtimes can provide stubs/mocks without coupling to infrastructure. - Used execute-phase view resolution from policy (`ProjectContextPolicy`/`ContextView`) so Execute does not accidentally reuse Strategize/Apply rules. - Applied budget and filtering decisions inside assembler boundaries and emitted diagnostics for inclusion/exclusion outcomes to make context-window behavior auditable during debugging. - Kept prompt-shape changes additive and backwards-compatible (context section only appears when non-empty assembled context exists). Wiring changes - Wired ACMS-backed execute assembler into plan CLI composition path in `src/cleveragents/cli/commands/plan.py` (`_get_plan_executor`) using existing container services (`context_tier_service`, `session_factory`) and settings budget values. Test updates - Behave updates in: - `features/llm_actors_coverage.feature` - `features/steps/llm_actors_coverage_steps.py` - Added/updated scenarios to cover: - Execute prompt includes ACMS assembled context - Fallback when assembler raises an exception - Behavior when assembler returns empty context - Robot integration updates in: - `robot/helper_m5_e2e_verification.py` - `robot/m5_e2e_verification.robot` - Added M5 helper flow validating execute-phase ACMS context integration path. Validation and quality gates executed - `nox -e lint` ✅ - `nox -e typecheck` ✅ - `nox -e unit_tests` ✅ - `nox -e integration_tests` ✅ - `nox -e coverage_report` ✅ (summary coverage: 97%) - Logs captured under `build/test-logs/` for reproducibility. Operational note - Commit hook `semgrep-eval-exec` flagged legacy `src/cleveragents/tool/wrapping.py` findings unrelated to this issue scope (same known baseline pattern as other tickets). Commit used `SKIP=semgrep-eval-exec` per workflow guidance in `promt.md`. Traceability - Code modules touched for this issue: - `src/cleveragents/application/services/llm_actors.py` - `src/cleveragents/cli/commands/plan.py` - `features/llm_actors_coverage.feature` - `features/steps/llm_actors_coverage_steps.py` - `robot/helper_m5_e2e_verification.py` - `robot/m5_e2e_verification.robot` - Commit: `a0d166e4`
Member

Follow-up update: added missing changelog entry required by CONTRIBUTING.

  • File: CHANGELOG.md (Unreleased section)
  • Commit: 61559822
  • Branch: feature/m5-acms-llm-integration
  • PR: #1163 now includes the changelog update.
Follow-up update: added missing changelog entry required by CONTRIBUTING. - File: `CHANGELOG.md` (Unreleased section) - Commit: `61559822` - Branch: `feature/m5-acms-llm-integration` - PR: #1163 now includes the changelog update.
Member

Review feedback from PR #1163 has been addressed.

  • Extracted the execute-phase ACMS assembler into its own service module.
  • Moved project policy reads behind NamespacedProjectRepository.get_context_policy(...) instead of raw SQL in the assembler.
  • Restored typed ProviderRegistry annotations in the LLM actors and updated the related test doubles.
  • Rebased feature/m5-acms-llm-integration onto the latest master and force-pushed the updated branch.

Targeted validation run locally:

  • nox -e lint
  • nox -e typecheck
  • nox -e unit_tests -- features/llm_actors_coverage.feature
  • nox -e integration_tests -- --test "Execute Phase LLM Uses ACMS Context"
Review feedback from PR #1163 has been addressed. - Extracted the execute-phase ACMS assembler into its own service module. - Moved project policy reads behind `NamespacedProjectRepository.get_context_policy(...)` instead of raw SQL in the assembler. - Restored typed `ProviderRegistry` annotations in the LLM actors and updated the related test doubles. - Rebased `feature/m5-acms-llm-integration` onto the latest `master` and force-pushed the updated branch. Targeted validation run locally: - `nox -e lint` - `nox -e typecheck` - `nox -e unit_tests -- features/llm_actors_coverage.feature` - `nox -e integration_tests -- --test "Execute Phase LLM Uses ACMS Context"`
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#850
No description provided.