## Summary
Add M6 parallel-scaling coverage for 10+ concurrent subplans:
- **15-subplan parallel scenario** with explicit peak-concurrency bound checks (`max_parallel=10`) and thread-safe concurrency tracking via `_build_executor()`.
- **Deep hierarchical decomposition** coverage (4+ levels) with adjusted leaf condition that only stops early when hitting `max_depth` or when the workset is trivially small (`min_files_per_subplan`).
- **Non-progress guard** in `_build_hierarchy` to prevent pathological recursion when clustering cannot meaningfully split the file set.
- **Small-project regression test** (< 50 files) verifying decomposition depth does not increase unexpectedly with the relaxed leaf condition.
- **ASV benchmark** for 15-subplan parallel execution with `max_parallel=10` to track scaling behavior.
### Removed from this PR
The `_build_hierarchy` child-linkage correctness fix (returning `node_id` from recursive calls instead of using `nodes[-1].node_id`) has been **removed** per review feedback — it is a separate bug fix and will be submitted as an independent issue/PR per CONTRIBUTING.md §Atomic Commits.
## Approach
- **Concurrency tracking:** The `_build_executor()` closure in step definitions detects `context.concurrency_counter` / `context.concurrency_lock` and performs thread-safe peak tracking in a try/finally block.
- **Leaf condition:** Replaced the `max_files_per_subplan` / `max_tokens_per_subplan` leaf check with a `min_files_per_subplan` check to allow deeper decomposition for large projects. Added a non-progress guard so clustering that cannot split the file set terminates immediately rather than recursing to `max_depth`.
- **Deterministic IDs:** `_ids_for_count()` preserves legacy fixed IDs for the first 5 subplans and generates additional deterministic IDs for scale scenarios.
## Validation
### Passing
- `nox -s lint` — all checks passed
- `nox -s typecheck` — 0 errors, 0 warnings
- `nox -s unit_tests` — 12,988 scenarios passed, 0 failed
- `nox -s coverage_report` — 97% (passes `--fail-under=97`)
Closes#855
Reviewed-on: cleveragents/cleveragents-core#1201
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Add hierarchical decomposition with 4+ levels and bounded context per
subplan. Implement decomposition heuristics (max_files_per_subplan,
max_tokens_per_subplan, language/dir clustering). Add dependency closure
computation for large graphs and DAG execution ordering. Add bounded
dependency closure with cutoff thresholds and memoization for 10K+
files. Record decomposition decisions in DecisionService
(strategy_choice + subplan_spawn entries).
New modules:
- decomposition_models.py: DecompositionConfig, DecompositionNode,
DecompositionResult, DependencyEdge, DependencyGraph
- decomposition_clustering.py: ClusteringStrategy with directory,
language, and size clustering plus deterministic sort
- decomposition_graph.py: DependencyClosureComputer with bounded
closure, topological sort, cycle detection, and memoization
- decomposition_service.py: DecompositionService orchestrating
hierarchy building and decision recording
Settings: planner_max_depth, planner_max_files_per_subplan,
planner_max_tokens_per_subplan, planner_min_files_per_subplan
Closes#205
brent.edwards
freemo