feat(actors): wire LLM strategize/execute actors to subplan spawning infrastructure #1207

Open
opened 2026-03-30 06:00:16 +00:00 by hurui200320 · 3 comments
Member

Metadata

  • Commit Message: feat(actors): wire LLM strategize/execute actors to subplan spawning infrastructure
  • Branch: feature/llm-actor-subplan-wiring

Background

PR #815 (merged into master) implements the WF04 E2E test for multi-project dependency updates using the supervised automation profile. The test verifies that the LLM spawns child subplans (one per project), executes them in dependency order, validates per project, and applies in order.

However, the WF04 test always gets skipped because the LLM never produces subplans. Investigation confirms this is not a test bug — the test is correctly implemented. The issue is that the subplan spawning infrastructure is fully built but not connected to the LLM actor execution path.

Current Behavior

  1. LLMStrategizeActor (application/services/llm_actors.py:53) sends a plain text prompt asking the LLM to "break it into numbered steps" and parses the response as text. It does not use LangChain tool/function calling.
  2. LLMExecuteActor (application/services/llm_actors.py:207) sends a plain text prompt asking for FILE: blocks and parses the response as text. Same limitation.
  3. The CLI's _get_plan_executor (cli/commands/plan.py:1265) creates the PlanExecutor without a tool_runner, execution_context, or tool registry — so even if the actors could call tools, there's no runtime to handle them.
  4. SubplanService.spawn() (application/services/subplan_service.py:182) is never called from any application code path — only from tests and benchmarks.
  5. The builtin/plan-subplan tool (tool/builtins/subplan_tool.py) exists and is registered, but is never provided to the LLM during plan execution.

As a result, plan.subplan_statuses remains empty after execution, and the WF04 test skips at wf04_multi_project.robot:459-461.

Expected Behavior

When a multi-project plan is executed, the LLM actors should be able to request subplan spawning via the builtin/plan-subplan tool (or equivalent mechanism), and the execution pipeline should materialize those requests into child plans via SubplanService.spawn().

Acceptance Criteria

  • LLM actors can invoke tools during strategize and/or execute phases (via LangChain tool calling or equivalent)
  • The builtin/plan-subplan tool (or equivalent subplan spawning mechanism) is available to the LLM during plan execution
  • SubplanService.spawn() is called from the execution pipeline when the LLM requests subplan creation
  • After execution, plan.subplan_statuses is populated for multi-project plans
  • WF04 E2E test (robot/e2e/wf04_multi_project.robot) no longer skips — the subplan_count == 0 skip guards are removed and replaced with hard assertions (fail if 0 subplans), so a failure to spawn subplans is reported as a test failure, not a silent skip
  • Existing unit tests, integration tests, and other E2E tests continue to pass

Subtasks

  • Enable tool calling in LLMStrategizeActor and/or LLMExecuteActor (replace plain text llm.invoke() with tool-enabled invocation)
  • Wire builtin/plan-subplan tool into the tool registry available to LLM actors during plan execution
  • Add orchestration code in PlanExecutor (or lifecycle service) to bridge LLM subplan decisions to SubplanService.spawn()
  • Update _get_plan_executor in CLI to provide tool_runner and/or execution_context as needed
  • Update LLM prompts to include multi-project context and subplan spawning instructions when the plan targets multiple projects
  • Tests (Behave): Add scenarios covering the LLM-to-subplan wiring
  • Tests (Robot): Remove the Skip guards for zero subplans in wf04_multi_project.robot (lines 215-217, 234-236, 262-264, 277-279, 459-461) and replace with hard Fail assertions, then verify WF04 passes end-to-end
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Supporting Information

  • WF04 test: robot/e2e/wf04_multi_project.robot
  • WF04 snapshot helper: robot/e2e/wf04_snapshot_helper.py
  • LLM actors: src/cleveragents/application/services/llm_actors.py
  • Subplan tool: src/cleveragents/tool/builtins/subplan_tool.py
  • SubplanService: src/cleveragents/application/services/subplan_service.py
  • PlanExecutor: src/cleveragents/application/services/plan_executor.py
  • CLI executor wiring: src/cleveragents/cli/commands/plan.py:1265 (_get_plan_executor)
  • Related PR: #815 (WF04 E2E test implementation)
  • Related issue: #750 (WF04 test ticket, now closed)
  • Related TODO: # TODO(#650) in llm_actors.py:119,274 — stale reference to structured output wiring
## Metadata - **Commit Message**: `feat(actors): wire LLM strategize/execute actors to subplan spawning infrastructure` - **Branch**: `feature/llm-actor-subplan-wiring` ## Background PR #815 (merged into master) implements the WF04 E2E test for multi-project dependency updates using the supervised automation profile. The test verifies that the LLM spawns child subplans (one per project), executes them in dependency order, validates per project, and applies in order. However, the WF04 test **always gets skipped** because the LLM never produces subplans. Investigation confirms this is not a test bug — the test is correctly implemented. The issue is that the subplan spawning infrastructure is fully built but **not connected** to the LLM actor execution path. ## Current Behavior 1. `LLMStrategizeActor` (`application/services/llm_actors.py:53`) sends a plain text prompt asking the LLM to "break it into numbered steps" and parses the response as text. It does not use LangChain tool/function calling. 2. `LLMExecuteActor` (`application/services/llm_actors.py:207`) sends a plain text prompt asking for `FILE:` blocks and parses the response as text. Same limitation. 3. The CLI's `_get_plan_executor` (`cli/commands/plan.py:1265`) creates the `PlanExecutor` without a `tool_runner`, `execution_context`, or tool registry — so even if the actors could call tools, there's no runtime to handle them. 4. `SubplanService.spawn()` (`application/services/subplan_service.py:182`) is never called from any application code path — only from tests and benchmarks. 5. The `builtin/plan-subplan` tool (`tool/builtins/subplan_tool.py`) exists and is registered, but is never provided to the LLM during plan execution. As a result, `plan.subplan_statuses` remains empty after execution, and the WF04 test skips at `wf04_multi_project.robot:459-461`. ## Expected Behavior When a multi-project plan is executed, the LLM actors should be able to request subplan spawning via the `builtin/plan-subplan` tool (or equivalent mechanism), and the execution pipeline should materialize those requests into child plans via `SubplanService.spawn()`. ## Acceptance Criteria - [ ] LLM actors can invoke tools during strategize and/or execute phases (via LangChain tool calling or equivalent) - [ ] The `builtin/plan-subplan` tool (or equivalent subplan spawning mechanism) is available to the LLM during plan execution - [ ] `SubplanService.spawn()` is called from the execution pipeline when the LLM requests subplan creation - [ ] After execution, `plan.subplan_statuses` is populated for multi-project plans - [ ] WF04 E2E test (`robot/e2e/wf04_multi_project.robot`) no longer skips — the subplan_count == 0 skip guards are removed and replaced with hard assertions (fail if 0 subplans), so a failure to spawn subplans is reported as a test failure, not a silent skip - [ ] Existing unit tests, integration tests, and other E2E tests continue to pass ## Subtasks - [ ] Enable tool calling in `LLMStrategizeActor` and/or `LLMExecuteActor` (replace plain text `llm.invoke()` with tool-enabled invocation) - [ ] Wire `builtin/plan-subplan` tool into the tool registry available to LLM actors during plan execution - [ ] Add orchestration code in `PlanExecutor` (or lifecycle service) to bridge LLM subplan decisions to `SubplanService.spawn()` - [ ] Update `_get_plan_executor` in CLI to provide `tool_runner` and/or `execution_context` as needed - [ ] Update LLM prompts to include multi-project context and subplan spawning instructions when the plan targets multiple projects - [ ] Tests (Behave): Add scenarios covering the LLM-to-subplan wiring - [ ] Tests (Robot): Remove the `Skip` guards for zero subplans in `wf04_multi_project.robot` (lines 215-217, 234-236, 262-264, 277-279, 459-461) and replace with hard `Fail` assertions, then verify WF04 passes end-to-end - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. ## Supporting Information - **WF04 test**: `robot/e2e/wf04_multi_project.robot` - **WF04 snapshot helper**: `robot/e2e/wf04_snapshot_helper.py` - **LLM actors**: `src/cleveragents/application/services/llm_actors.py` - **Subplan tool**: `src/cleveragents/tool/builtins/subplan_tool.py` - **SubplanService**: `src/cleveragents/application/services/subplan_service.py` - **PlanExecutor**: `src/cleveragents/application/services/plan_executor.py` - **CLI executor wiring**: `src/cleveragents/cli/commands/plan.py:1265` (`_get_plan_executor`) - **Related PR**: #815 (WF04 E2E test implementation) - **Related issue**: #750 (WF04 test ticket, now closed) - **Related TODO**: `# TODO(#650)` in `llm_actors.py:119,274` — stale reference to structured output wiring
hurui200320 added this to the v3.3.0 milestone 2026-03-30 06:00:38 +00:00
freemo self-assigned this 2026-04-02 06:13:52 +00:00
Owner

[Backlog Groomer - groomer-1] 📋 Label state mismatch. This issue has State/Unverified but PR #1304 (feat(actors): wire LLM strategize/execute actors to subplan spawning) is open and references this issue. The state should be updated to at least State/Verified (or State/In Review if the PR is ready for review).

**[Backlog Groomer - groomer-1]** 📋 **Label state mismatch.** This issue has `State/Unverified` but PR #1304 (`feat(actors): wire LLM strategize/execute actors to subplan spawning`) is open and references this issue. The state should be updated to at least `State/Verified` (or `State/In Review` if the PR is ready for review).
Owner

PR #1304 has been reviewed. Changes requested: Two # type: ignore[assignment] suppressions in llm_actors.py must be removed (CONTRIBUTING.md violation). The fix is straightforward — use a separate bound_llm: Any variable instead of reassigning llm. PR metadata (milestone, labels) also needs to be set. See the full review on the PR for details.

PR #1304 has been reviewed. **Changes requested**: Two `# type: ignore[assignment]` suppressions in `llm_actors.py` must be removed (CONTRIBUTING.md violation). The fix is straightforward — use a separate `bound_llm: Any` variable instead of reassigning `llm`. PR metadata (milestone, labels) also needs to be set. See the full review on the PR for details.
Owner

PR #1304 has been reviewed (reviewer-pool-2). Changes requested: two # type: ignore[assignment] suppressions in llm_actors.py must be removed per CONTRIBUTING.md. The overall design and test coverage are solid — once the type suppressions are fixed, the PR is ready to merge. See the review comment for full details.

PR #1304 has been reviewed (reviewer-pool-2). **Changes requested**: two `# type: ignore[assignment]` suppressions in `llm_actors.py` must be removed per CONTRIBUTING.md. The overall design and test coverage are solid — once the type suppressions are fixed, the PR is ready to merge. See the [review comment](https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/1304#issuecomment-79194) for full details.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#368 Epic: Subplans & Parallelism
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#1207
No description provided.