feat(plans): implement parallel subplan execution scheduler with max_parallel concurrency control #9609

Merged
HAL9000 merged 6 commits from feat/v3.3.0-parallel-subplan-scheduler into master 2026-06-07 00:44:09 +00:00

6 Commits

Author SHA1 Message Date
HAL9000 06438a02b1 fix(plans): resolve ambiguous step + ordering issues in scheduler BDD tests
CI / push-validation (pull_request) Successful in 28s
CI / build (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 1m14s
CI / typecheck (pull_request) Successful in 1m15s
CI / helm (pull_request) Successful in 1m17s
CI / security (pull_request) Successful in 1m19s
CI / unit_tests (pull_request) Successful in 6m8s
CI / docker (pull_request) Successful in 1m49s
CI / integration_tests (pull_request) Successful in 10m18s
CI / coverage (pull_request) Successful in 10m54s
CI / status-check (pull_request) Successful in 4s
The parallel_subplan_scheduler_steps.py declared a Then step
`all {count:d} subplans should complete successfully` that collided with
the existing `all {n:d} subplans should complete successfully` in
subplan_execution_steps.py, causing behave AmbiguousStep errors that
cascaded across 8 scenarios in subplan_execution.feature plus several
scenarios in parallel_subplan_scheduler.feature.

Resolved by:

* Removing the duplicate Then step; the @when step in
  parallel_subplan_scheduler_steps.py already sets `context.exec_result`
  so the existing shared assertion handles both feature files.
* Reordering result.statuses back to input order in the scheduler
  @when step (parallel execution returns statuses in completion
  order) so index-based shared assertions are deterministic.
* Binding `context.merge_result` and `context.exec_error` for shared
  assertion-step compatibility.
* Using subplan_id lookup instead of positional access in the
  scheduler-specific Then steps (`the second subplan should complete
  successfully`, `the first subplan should be errored with timeout`).
* Differentiating per-subplan content in the overlapping-file-changes
  step with staggered timing so LAST_WINS merge is deterministic.
* Blocking non-first subplans in the first-failure step so fail_fast
  cascade can actually mark them CANCELLED.
* Using a TimeoutError-raising executor for timeout scenarios to
  exercise the scheduler's timeout-handling path deterministically
  under the in-process parallel test runner.

All 77 scenarios in features/parallel_subplan_scheduler.feature and
features/subplan_execution.feature now pass.
2026-06-06 20:26:28 -04:00
cleveragents-auto 712d773836 chore: worker ruff auto-fix (pre-push lint gate) 2026-06-06 20:26:27 -04:00
HAL9000 428ec07951 fix(plans): derive ULID-compatible subplan IDs in scheduler BDD tests
SubplanStatus.subplan_id is pydantic-validated against ^[0-9A-HJKMNP-TV-Z]{26}$.
The scheduler test fixtures constructed SubplanStatus instances with short logical
IDs ("subplan-001", "subplan-A") which failed validation at fixture-construction
time, erroring 22 of 38 originally-failing scenarios in unit_tests CI before the
behavioural assertions could even run.

Derive a deterministic 26-char Crockford-Base32 ID from each logical name via
SHA-256 and translate fail-id sets, block-second dicts, dependency graphs, and
result-status lookups through the same helper so cross-references stay
consistent.

Also add the missing step definitions unique to the parallel_subplan_scheduler
scenarios (staggered-completion fixture, retry-then-succeed fixture, fail_fast-
disabled scheduler, mode-only scheduler, timeout-errored verifier) and remove
duplicate @then registrations that conflict with shared step definitions in
subplan_execution_steps.py.

ISSUES CLOSED: #9555
2026-06-06 20:26:27 -04:00
HAL9000 ebb543a9c3 fix(plans): resolve CI failures in parallel subplan scheduler BDD tests
- Remove duplicate @then decorator on step_verify_peak_concurrency_limit
  (caused AmbiguousStep error crashing all 8 unit test feature files)
- Rename "the subplans should have been executed in order" to
  "the subplans should have been executed in sequential order" to
  avoid conflict with pre-existing step in subplan_execution_steps.py
- Remove 13 additional @then step definitions that duplicated steps in
  subplan_execution_steps.py; alias context.exec_result and
  context.validation_error in @when steps so pre-existing steps work
- Replace two # type: ignore comments (lines 438, 453) with typed
  Any variables per zero-tolerance policy
- Apply ruff format to fix formatting (long import wrapping, list comps)
- Add CHANGELOG entry and CONTRIBUTORS entry for #9555

ISSUES CLOSED: #9609
2026-06-06 20:26:27 -04:00
HAL9000 8648e17472 fix(lint): address ruff lint failures in parallel_subplan_scheduler_steps.py 2026-06-06 20:26:27 -04:00
CleverAgents Automation 705eb5a969 feat(plans): implement parallel subplan execution scheduler with max_parallel concurrency control
- Add ParallelSubplanScheduler class for managing parallel subplan execution
- Implement SubplanQueue for tracking pending, active, and completed subplans
- Implement SchedulerState for immutable scheduler state tracking
- Support configurable max_parallel concurrency limit (1-50)
- Support sequential, parallel, and dependency-ordered execution modes
- Automatic queuing of subplans when max_parallel limit is reached
- Parent plan blocks until all subplans complete
- Comprehensive failure handling and retry logic
- Merge strategy selection for combining subplan outputs
- Add comprehensive BDD test suite with 50+ scenarios
- Test coverage for concurrency control, queue management, and state tracking
2026-06-06 20:26:27 -04:00