freemo
4232907ab9
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 18s
CI / build (pull_request) Successful in 23s
CI / typecheck (pull_request) Successful in 34s
CI / security (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 3m17s
CI / docker (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 4m3s
CI / coverage (pull_request) Successful in 4m9s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 16s
CI / build (push) Successful in 15s
CI / security (push) Successful in 29s
CI / typecheck (push) Successful in 33s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m23s
CI / integration_tests (push) Successful in 2m48s
CI / docker (push) Successful in 38s
CI / coverage (push) Successful in 4m16s
CI / benchmark-publish (push) Successful in 16m55s
CI / benchmark-regression (pull_request) Successful in 30m14s
Add hierarchical decomposition with 4+ levels and bounded context per subplan. Implement decomposition heuristics (max_files_per_subplan, max_tokens_per_subplan, language/dir clustering). Add dependency closure computation for large graphs and DAG execution ordering. Add bounded dependency closure with cutoff thresholds and memoization for 10K+ files. Record decomposition decisions in DecisionService (strategy_choice + subplan_spawn entries). New modules: - decomposition_models.py: DecompositionConfig, DecompositionNode, DecompositionResult, DependencyEdge, DependencyGraph - decomposition_clustering.py: ClusteringStrategy with directory, language, and size clustering plus deterministic sort - decomposition_graph.py: DependencyClosureComputer with bounded closure, topological sort, cycle detection, and memoization - decomposition_service.py: DecompositionService orchestrating hierarchy building and decision recording Settings: planner_max_depth, planner_max_files_per_subplan, planner_max_tokens_per_subplan, planner_min_files_per_subplan Closes #205
2.2 KiB
2.2 KiB
Large-Project Decomposition
Overview
The large-project decomposition subsystem breaks large file sets into bounded subplans suitable for independent execution. It provides hierarchical decomposition with 4+ levels, clustering heuristics, dependency closure computation, and DAG execution ordering.
Configuration
| Setting | Default | Description |
|---|---|---|
planner_max_depth |
4 | Maximum recursion depth |
planner_max_files_per_subplan |
500 | Upper file count per leaf |
planner_max_tokens_per_subplan |
100000 | Upper token count per leaf |
planner_min_files_per_subplan |
10 | Threshold below which decomposition is skipped |
Clustering Strategies
Three complementary strategies partition files into bounded clusters:
- Directory — Groups files by common directory prefix. Splits oversized
buckets into chunks of
max_files_per_subplan. - Language — Groups files by file extension. Useful for polyglot projects.
- Size — Groups files by cumulative estimated token count.
A deterministic sort ensures clusters are always produced in the same order.
Dependency Closure
DependencyClosureComputer computes bounded transitive closures on file
dependency graphs:
compute_closure(graph, roots, cutoff)— BFS forward closure with optional size limit. Results are memoised for repeated queries.topological_sort(graph)— Kahn's algorithm for DAG execution ordering.detect_cycles(graph)— DFS colouring to find elementary cycles.
Decision Recording
DecompositionService.record_decisions() persists two types of decisions:
strategy_choice— The overall decomposition strategy.subplan_spawn— One per leaf node in the decomposition tree.
API
from cleveragents.application.services.decomposition_service import (
DecompositionService,
)
from cleveragents.application.services.decomposition_models import (
DecompositionConfig,
)
svc = DecompositionService(decision_service=ds)
result = svc.decompose(files, DecompositionConfig(max_depth=4))
svc.record_decisions(plan_id, result)
Related
- Subplan Service
- Decision Service
- Forgejo issue #205