forked from HAL9000/cleveragents-core
freemo
4232907ab9
Add hierarchical decomposition with 4+ levels and bounded context per subplan. Implement decomposition heuristics (max_files_per_subplan, max_tokens_per_subplan, language/dir clustering). Add dependency closure computation for large graphs and DAG execution ordering. Add bounded dependency closure with cutoff thresholds and memoization for 10K+ files. Record decomposition decisions in DecisionService (strategy_choice + subplan_spawn entries). New modules: - decomposition_models.py: DecompositionConfig, DecompositionNode, DecompositionResult, DependencyEdge, DependencyGraph - decomposition_clustering.py: ClusteringStrategy with directory, language, and size clustering plus deterministic sort - decomposition_graph.py: DependencyClosureComputer with bounded closure, topological sort, cycle detection, and memoization - decomposition_service.py: DecompositionService orchestrating hierarchy building and decision recording Settings: planner_max_depth, planner_max_files_per_subplan, planner_max_tokens_per_subplan, planner_min_files_per_subplan Closes #205
67 lines
2.2 KiB
Markdown
67 lines
2.2 KiB
Markdown
# Large-Project Decomposition
|
|
|
|
## Overview
|
|
|
|
The large-project decomposition subsystem breaks large file sets into bounded
|
|
subplans suitable for independent execution. It provides hierarchical
|
|
decomposition with 4+ levels, clustering heuristics, dependency closure
|
|
computation, and DAG execution ordering.
|
|
|
|
## Configuration
|
|
|
|
| Setting | Default | Description |
|
|
|---------|---------|-------------|
|
|
| `planner_max_depth` | 4 | Maximum recursion depth |
|
|
| `planner_max_files_per_subplan` | 500 | Upper file count per leaf |
|
|
| `planner_max_tokens_per_subplan` | 100000 | Upper token count per leaf |
|
|
| `planner_min_files_per_subplan` | 10 | Threshold below which decomposition is skipped |
|
|
|
|
## Clustering Strategies
|
|
|
|
Three complementary strategies partition files into bounded clusters:
|
|
|
|
- **Directory** — Groups files by common directory prefix. Splits oversized
|
|
buckets into chunks of `max_files_per_subplan`.
|
|
- **Language** — Groups files by file extension. Useful for polyglot projects.
|
|
- **Size** — Groups files by cumulative estimated token count.
|
|
|
|
A deterministic sort ensures clusters are always produced in the same order.
|
|
|
|
## Dependency Closure
|
|
|
|
`DependencyClosureComputer` computes bounded transitive closures on file
|
|
dependency graphs:
|
|
|
|
- `compute_closure(graph, roots, cutoff)` — BFS forward closure with optional
|
|
size limit. Results are memoised for repeated queries.
|
|
- `topological_sort(graph)` — Kahn's algorithm for DAG execution ordering.
|
|
- `detect_cycles(graph)` — DFS colouring to find elementary cycles.
|
|
|
|
## Decision Recording
|
|
|
|
`DecompositionService.record_decisions()` persists two types of decisions:
|
|
|
|
1. `strategy_choice` — The overall decomposition strategy.
|
|
2. `subplan_spawn` — One per leaf node in the decomposition tree.
|
|
|
|
## API
|
|
|
|
```python
|
|
from cleveragents.application.services.decomposition_service import (
|
|
DecompositionService,
|
|
)
|
|
from cleveragents.application.services.decomposition_models import (
|
|
DecompositionConfig,
|
|
)
|
|
|
|
svc = DecompositionService(decision_service=ds)
|
|
result = svc.decompose(files, DecompositionConfig(max_depth=4))
|
|
svc.record_decisions(plan_id, result)
|
|
```
|
|
|
|
## Related
|
|
|
|
- [Subplan Service](subplan_service.md)
|
|
- [Decision Service](decision_service.md)
|
|
- Forgejo issue #205
|