Files
cleveragents-core/docs/reference/subplans.md
T
Luis Mendes 9f6fb667c2
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 20s
CI / security (pull_request) Successful in 33s
CI / typecheck (pull_request) Successful in 34s
CI / integration_tests (pull_request) Successful in 2m45s
CI / unit_tests (pull_request) Successful in 13m14s
CI / docker (pull_request) Successful in 38s
CI / benchmark-regression (pull_request) Successful in 24m49s
CI / coverage (pull_request) Successful in 49m5s
CI / lint (push) Successful in 14s
CI / quality (push) Successful in 15s
CI / build (push) Successful in 14s
CI / security (push) Successful in 28s
CI / typecheck (push) Successful in 30s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 2m43s
CI / unit_tests (push) Successful in 13m9s
CI / docker (push) Successful in 39s
CI / benchmark-publish (push) Successful in 14m49s
CI / coverage (push) Successful in 47m53s
feat(subplan): execute and merge subplans
Implement SubplanExecutionService and SubplanMergeService to enable
parent plans to decompose work into coordinated child subplans with
configurable execution modes (sequential, parallel, dependency-ordered)
and merge strategies (git_three_way, sequential_apply, fail_on_conflict,
last_wins).

- SubplanExecutionService: schedules subplan execution with retry
  support via SubplanFailureHandler, max_parallel limits, fail_fast
  semantics, and topological DAG ordering for dependency mode
- SubplanMergeService: wraps sandbox merge infrastructure to combine
  subplan sandbox outputs using the configured merge strategy
- BDD tests: 21 scenarios covering all execution modes, merge
  strategies, validation, and integration flows
- Robot tests: 11 integration test cases with helper module
- ASV benchmarks: performance benchmarks for execution and merge
- Documentation: reference guide in docs/reference/subplans.md

ISSUES CLOSED: #184
2026-03-01 09:07:06 +00:00

8.3 KiB
Raw Blame History

Subplans: Execution and Merge

Subplans allow a parent plan to decompose work into coordinated child plans. Each child plan runs in its own sandbox, and results are merged back into the parent plan using a configurable merge strategy.

Execution Modes

The SubplanConfig.execution_mode field controls how child plans are scheduled:

Mode Description
sequential Execute one at a time in order
parallel Execute concurrently (up to max_parallel)
dependency_ordered Respect DAG dependencies via topological sort

Sequential Mode

Subplans execute one at a time in the order they appear in subplan_statuses. If a subplan fails and fail_fast is enabled, remaining subplans are cancelled.

Parallel Mode

Subplans execute concurrently using a thread pool. The max_parallel setting (default: 5, range: 150) caps the number of concurrent workers. When fail_fast is enabled, remaining subplans are cancelled on first failure. Cancelled futures are reported with CANCELLED status (not ERRORED). Results are returned in completion order, which affects ordering-sensitive merge strategies such as sequential_apply.

Dependency-Ordered Mode

Subplans are sorted topologically based on a dependency graph. Independent subplans (those whose dependencies are all satisfied) within the same topological wave are executed concurrently, respecting the max_parallel limit. Subsequent waves wait for the previous wave to finish. A circular dependency raises a ValueError. The dependency graph maps each subplan_id to the list of subplan_ids it depends on.

Merge Strategies

After subplans complete, their sandbox outputs are merged using the configured SubplanMergeStrategy:

Strategy Description
git_three_way Three-way merge via git merge-file
sequential_apply Apply changes in completion order
fail_on_conflict Raise MergeConflictError on any conflict
last_wins Final subplan's output overwrites earlier ones

Git Three-Way Merge

Uses git merge-file to perform content-level merging. Non-overlapping changes from different subplans are combined automatically. Overlapping changes produce conflict markers in the output.

Sequential Apply

Each subplan's output is applied in the order subplans completed, with each becoming the new base for the next. In sequential mode, this matches the input order. In parallel mode, outputs are ordered by actual completion time, so the subplan that finishes last has its changes applied last. Equivalent to replaying edits sequentially.

Fail on Conflict

Same as git three-way merge, but raises MergeConflictError if any file has unresolved conflicts. Use this when conflicts must be resolved manually.

Last Wins

The final subplan's version of each file overwrites all earlier versions unconditionally. Simple but may discard concurrent work.

Merge Outcomes

The SubplanMergeResult reports:

Field Description
success True if all files merged without conflicts
merged_files Per-file outcomes with content and conflict status
conflict_files Paths of files with unresolved conflicts
total_files Total unique files across all subplan outputs

Each FileMergeOutcome includes:

Field Description
path Relative file path
content Merged content (may contain conflict markers)
has_conflict Whether this file has unresolved conflicts
source_subplan_ids IDs of subplans that modified this file

Timeout Enforcement

When timeout_per_subplan_seconds is set on SubplanConfig, each subplan execution is subject to a wall-clock timeout. If a subplan (including any retries) does not complete within the configured number of seconds, it is marked ERRORED with a SubplanTimeoutError message. Timeouts are enforced in all execution modes (sequential, parallel, and dependency-ordered).

The timeout applies per-subplan, not per-attempt. The entire retry loop for a subplan must complete within the deadline. If the timeout expires during a retry, no further retries are attempted and the subplan is marked as errored immediately.

Failure Handling

The SubplanFailureHandler determines retry and stop behavior:

  • fail_fast: Stop all subplans on first failure (any mode).
  • Sequential mode: Always stops on failure (even without fail_fast).
  • Parallel mode: Other subplans continue unless fail_fast is set.
  • Retry: Retriable errors (TimeoutError, ValidationError, TemporaryResourceError, MergeConflictError) are retried up to max_retries times.
  • Non-retriable: ConfigurationError, AuthenticationError, MissingResourceError, CircularDependencyError are never retried.

Configuration Reference

subplan_config:
  execution_mode: parallel        # sequential | parallel | dependency_ordered
  merge_strategy: git_three_way   # git_three_way | sequential_apply | fail_on_conflict | last_wins
  max_parallel: 5                 # 1-50, for parallel mode
  fail_fast: false                # stop all on first failure
  timeout_per_subplan_seconds: ~  # optional per-subplan timeout
  retry_failed: true              # auto-retry failed subplans
  max_retries: 2                  # 0-5, max retry attempts

Service API

SubplanExecutionService

from cleveragents.application.services import (
    SubplanExecutionService,
    SubplanExecutionOutput,
)

service = SubplanExecutionService(
    config=plan.subplan_config,
    executor_fn=my_executor,  # Callable[[SubplanStatus], SubplanExecutionOutput]
)

result = service.execute_all(
    subplan_statuses=plan.subplan_statuses,
    base_files={"src/main.py": original_content},
    dependency_graph={"sub-2": ["sub-1"]},  # for dependency_ordered mode
)

SubplanMergeService

from cleveragents.application.services import SubplanMergeService

merge_service = SubplanMergeService(strategy=SubplanMergeStrategy.GIT_THREE_WAY)

result = merge_service.merge(
    base_files={"src/main.py": "original content"},
    subplan_outputs=[
        ("sub-1", {"src/main.py": "modified by sub-1"}),
        ("sub-2", {"src/main.py": "modified by sub-2"}),
    ],
)

Design Decisions and Specification Alignment

This section documents intentional deviations from the specification (docs/specification.md) and the rationale behind them.

Execution mode via configuration vs. decision tree

The specification describes execution mode as emerging from the decision tree (subplan_spawn vs subplan_parallel_spawn decision types). This implementation uses a configurable ExecutionMode enum on SubplanConfig instead, which simplifies the service interface and allows the execution mode to be set declaratively without coupling to decision-tree internals.

Single merge strategy per plan vs. per-resource-type

The specification describes merge strategies as resource-type-dependent (e.g., git three-way for git resources, sequential for databases). This implementation uses a single SubplanMergeStrategy per SubplanConfig, applied uniformly to all files. A per-resource-type merge dispatch can be layered on top by callers that select the strategy based on the resource type before constructing the service.

Global concurrency and depth limits

The specification defines plan.concurrency (default 4) and plan.max-child-depth (default 5) as global limits. These are not enforced within this service because they are cross-cutting concerns that belong at the orchestration layer above SubplanExecutionService. The max_parallel setting on SubplanConfig controls per-plan concurrency locally.