4 Commits

Author SHA1 Message Date
CoreRasurae 3a95701d9a fix(decomposition): enforce plan.max-child-depth recursion limit to prevent unbounded hierarchy growth
CI / lint (pull_request) Successful in 1m19s
CI / quality (pull_request) Successful in 1m39s
CI / typecheck (pull_request) Successful in 1m43s
CI / security (pull_request) Successful in 1m56s
CI / build (pull_request) Successful in 33s
CI / helm (pull_request) Successful in 40s
CI / unit_tests (pull_request) Successful in 6m36s
CI / integration_tests (pull_request) Successful in 3m57s
CI / push-validation (pull_request) Successful in 19s
CI / docker (pull_request) Successful in 1m28s
CI / coverage (pull_request) Successful in 12m46s
CI / status-check (pull_request) Successful in 4s
CI / lint (push) Failing after 12s
CI / typecheck (push) Failing after 11s
CI / integration_tests (push) Failing after 7s
CI / unit_tests (push) Failing after 12s
CI / quality (push) Failing after 12s
CI / security (push) Failing after 13s
CI / coverage (push) Has been skipped
CI / docker (push) Has been skipped
CI / push-validation (push) Failing after 13s
CI / helm (push) Failing after 14s
CI / build (push) Failing after 15s
CI / status-check (push) Failing after 2s
CI / benchmark-regression (push) Failing after 34s
CI / e2e_tests (push) Successful in 1m1s
CI / benchmark-publish (push) Failing after 35m10s
Add max_child_depth field to DecompositionConfig (default: 5, matching
plan.max-child-depth config) with validation in __post_init__. Update
_build_hierarchy() in DecompositionService to stop recursion and emit
a warning when the depth limit is reached.

BDD tests verify: max_child_depth guards trigger before max_depth
when more restrictive, warnings are logged, default config value is
correct, and invalid values are rejected.

ISSUES CLOSED: #10269
2026-05-19 21:56:28 +01:00
HAL9000 e261ea5abe fix(decomposition): address review feedback for PR #9437
CI / push-validation (pull_request) Successful in 34s
CI / helm (pull_request) Successful in 36s
CI / lint (pull_request) Successful in 3m55s
CI / typecheck (pull_request) Successful in 4m35s
CI / security (pull_request) Successful in 4m43s
CI / build (pull_request) Successful in 3m57s
CI / quality (pull_request) Successful in 4m26s
CI / e2e_tests (pull_request) Successful in 8m57s
CI / integration_tests (pull_request) Successful in 10m49s
CI / unit_tests (pull_request) Successful in 11m56s
CI / docker (pull_request) Failing after 46s
CI / coverage (pull_request) Successful in 14m47s
CI / status-check (pull_request) Failing after 3s
- Remove committed test_reports/ CI artifacts (summary.txt, test_results.json)
- Add test_reports/ to .gitignore to prevent future commits
- Update CHANGELOG.md with Fixed entry for #9401 under [Unreleased]
- Update CONTRIBUTORS.md with credit for directory clustering fix (#9401)
- Add BDD test scenarios for absolute path clustering in
  features/large_project_decomposition.feature:
  * Directory clustering works correctly with absolute file paths
  * _directory_key returns correct key for absolute path with root
  * Directory clustering does not collapse all absolute paths into one bucket
- Add corresponding step implementations in
  features/steps/large_project_decomposition_coverage_steps.py

All 38 scenarios pass (including 3 new absolute path scenarios).
Lint and typecheck pass.

ISSUES CLOSED: #9401
2026-04-17 22:49:46 +00:00
brent.edwards 01b6eb1804 feat(autonomy): parallel execution scales to 10+ concurrent subplans (#1201)
CI / build (push) Successful in 17s
CI / helm (push) Successful in 22s
CI / lint (push) Successful in 28s
CI / typecheck (push) Successful in 47s
CI / benchmark-regression (push) Has been skipped
CI / quality (push) Successful in 3m49s
CI / security (push) Successful in 4m11s
CI / unit_tests (push) Successful in 9m19s
CI / docker (push) Successful in 1m22s
CI / coverage (push) Successful in 12m35s
CI / e2e_tests (push) Successful in 16m17s
CI / integration_tests (push) Successful in 25m5s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (push) Successful in 28m31s
## Summary

Add M6 parallel-scaling coverage for 10+ concurrent subplans:

- **15-subplan parallel scenario** with explicit peak-concurrency bound checks (`max_parallel=10`) and thread-safe concurrency tracking via `_build_executor()`.
- **Deep hierarchical decomposition** coverage (4+ levels) with adjusted leaf condition that only stops early when hitting `max_depth` or when the workset is trivially small (`min_files_per_subplan`).
- **Non-progress guard** in `_build_hierarchy` to prevent pathological recursion when clustering cannot meaningfully split the file set.
- **Small-project regression test** (< 50 files) verifying decomposition depth does not increase unexpectedly with the relaxed leaf condition.
- **ASV benchmark** for 15-subplan parallel execution with `max_parallel=10` to track scaling behavior.

### Removed from this PR

The `_build_hierarchy` child-linkage correctness fix (returning `node_id` from recursive calls instead of using `nodes[-1].node_id`) has been **removed** per review feedback — it is a separate bug fix and will be submitted as an independent issue/PR per CONTRIBUTING.md §Atomic Commits.

## Approach

- **Concurrency tracking:** The `_build_executor()` closure in step definitions detects `context.concurrency_counter` / `context.concurrency_lock` and performs thread-safe peak tracking in a try/finally block.
- **Leaf condition:** Replaced the `max_files_per_subplan` / `max_tokens_per_subplan` leaf check with a `min_files_per_subplan` check to allow deeper decomposition for large projects. Added a non-progress guard so clustering that cannot split the file set terminates immediately rather than recursing to `max_depth`.
- **Deterministic IDs:** `_ids_for_count()` preserves legacy fixed IDs for the first 5 subplans and generates additional deterministic IDs for scale scenarios.

## Validation

### Passing
- `nox -s lint` — all checks passed
- `nox -s typecheck` — 0 errors, 0 warnings
- `nox -s unit_tests` — 12,988 scenarios passed, 0 failed
- `nox -s coverage_report` — 97% (passes `--fail-under=97`)

Closes #855

Reviewed-on: #1201
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
2026-03-31 23:57:39 +00:00
freemo 4232907ab9 feat(plan): add large-project decomposition and dependency closure
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 18s
CI / build (pull_request) Successful in 23s
CI / typecheck (pull_request) Successful in 34s
CI / security (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 3m17s
CI / docker (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 4m3s
CI / coverage (pull_request) Successful in 4m9s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 16s
CI / build (push) Successful in 15s
CI / security (push) Successful in 29s
CI / typecheck (push) Successful in 33s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m23s
CI / integration_tests (push) Successful in 2m48s
CI / docker (push) Successful in 38s
CI / coverage (push) Successful in 4m16s
CI / benchmark-publish (push) Successful in 16m55s
CI / benchmark-regression (pull_request) Successful in 30m14s
Add hierarchical decomposition with 4+ levels and bounded context per
subplan. Implement decomposition heuristics (max_files_per_subplan,
max_tokens_per_subplan, language/dir clustering). Add dependency closure
computation for large graphs and DAG execution ordering. Add bounded
dependency closure with cutoff thresholds and memoization for 10K+
files. Record decomposition decisions in DecisionService
(strategy_choice + subplan_spawn entries).

New modules:
- decomposition_models.py: DecompositionConfig, DecompositionNode,
  DecompositionResult, DependencyEdge, DependencyGraph
- decomposition_clustering.py: ClusteringStrategy with directory,
  language, and size clustering plus deterministic sort
- decomposition_graph.py: DependencyClosureComputer with bounded
  closure, topological sort, cycle detection, and memoization
- decomposition_service.py: DecompositionService orchestrating
  hierarchy building and decision recording

Settings: planner_max_depth, planner_max_files_per_subplan,
planner_max_tokens_per_subplan, planner_min_files_per_subplan

Closes #205
2026-03-03 21:44:20 +00:00