forked from cleveragents/cleveragents-core
Cleaned up and improved the specification and plan slightly
This commit is contained in:
+129
-1
@@ -75,6 +75,26 @@ While CleverAgents leverages LangGraph and LangChain for the underlying LLM runt
|
||||
- Generate fresh documentation via Docusaurus integrated within the `docs/` directory of the CleverAgents project.
|
||||
- Delete the plandex/ reference directory once migration is complete - it serves only as temporary reference material.
|
||||
|
||||
### Core Architectural Requirements
|
||||
|
||||
**Scalability**: The system must handle massive codebases (50,000+ files) through:
|
||||
- Three-tier memory architecture (hot/warm/cold)
|
||||
- Hierarchical task decomposition
|
||||
- Bounded dependency closures
|
||||
- Lazy resource sandboxing
|
||||
|
||||
**Reliability**: Prevent cascading failures through:
|
||||
- Complete execution isolation via sandboxes
|
||||
- Multi-layer semantic error prevention
|
||||
- Checkpoint-based rollback capabilities
|
||||
- Invariant enforcement throughout execution
|
||||
|
||||
**Autonomy with Control**: Progressive automation through:
|
||||
- Three-level automation system (manual, review-before-apply, full)
|
||||
- Decision correction without full re-execution
|
||||
- Confidence-based escalation
|
||||
- Semantic understanding of when human input is needed
|
||||
|
||||
---
|
||||
|
||||
## Continuous Testing and Documentation Policy
|
||||
@@ -111,6 +131,32 @@ Action → Strategize → Execute → Apply → Applied (terminal)
|
||||
|
||||
**Strategize / Execute / Apply phase states**: `queued`, `processing`, `errored`, `complete`, `cancelled`
|
||||
|
||||
### Key Architectural Components
|
||||
|
||||
**Multi-tier Memory System**:
|
||||
- **Hot tier**: Immediate working context in LLM context window
|
||||
- **Warm tier**: Recent decisions and contexts from current plan tree
|
||||
- **Cold tier**: Historical decisions from past plans, queryable but not in active memory
|
||||
- Context snapshots with cryptographic hashes preserve complete decision context
|
||||
|
||||
**Dependency Closure Computation**:
|
||||
- Resource-aware analysis during Strategize
|
||||
- Hierarchical scoping with explicit resource lists
|
||||
- Lazy expansion prevents closure explosion
|
||||
- Interface-based boundaries for modular changes
|
||||
|
||||
**Execution Coordination**:
|
||||
- Complete isolation via per-plan sandboxes
|
||||
- Resource-specific sandbox strategies (git worktrees, transactions, etc.)
|
||||
- Hierarchical merge resolution
|
||||
- Checkpoint-based coordination for rollback
|
||||
|
||||
**Semantic Error Prevention**:
|
||||
- Decision-time validation during Strategize
|
||||
- Execution-time semantic guards in actors
|
||||
- Invariant enforcement throughout
|
||||
- Pattern-based predictive error prevention
|
||||
|
||||
### Namespace Rules
|
||||
|
||||
| Namespace | Scope | Storage |
|
||||
@@ -514,8 +560,19 @@ Execute all required tests through the appropriate `nox` sessions—never call `
|
||||
- [ ] Implement `FilesystemSandbox` (copy-on-write) for local files
|
||||
- [ ] Implement `NoSandbox` for non-sandboxable resources
|
||||
- [ ] Add sandbox lifecycle (create, commit, rollback, cleanup)
|
||||
- [ ] Implement isolation mechanisms:
|
||||
- [ ] Each plan gets its own sandbox containing only resources it edits
|
||||
- [ ] Parallel plans cannot see each other's intermediate states
|
||||
- [ ] Lazy sandboxing - resources sandboxed only when accessed
|
||||
- [ ] Implement merge strategies:
|
||||
- [ ] Git-style three-way merge for code resources
|
||||
- [ ] Sequential application for databases
|
||||
- [ ] Smart JSON/YAML merge for config files
|
||||
- [ ] Parent plan merge resolution for conflicts
|
||||
- [ ] Location: `src/cleveragents/infrastructure/sandbox/`
|
||||
- [ ] Tests: Integration tests for each sandbox type
|
||||
- [ ] Tests: Parallel execution isolation tests
|
||||
- [ ] Tests: Merge conflict resolution tests
|
||||
|
||||
- [ ] **Stage B4: Resource Integration** (Day 4-5)
|
||||
- [ ] Code: Connect resources to plan execution
|
||||
@@ -586,11 +643,20 @@ Execute all required tests through the appropriate `nox` sessions—never call `
|
||||
- [ ] **Stage C5: Plan-Actor Integration** (Day 6-8)
|
||||
- [ ] Code: Connect actors to plan lifecycle
|
||||
- [ ] Strategize phase invokes `strategy_actor` with project context
|
||||
- [ ] Implement dependency closure computation
|
||||
- [ ] Resource-aware analysis (imports, includes, symbols, tests, build refs)
|
||||
- [ ] Hierarchical task decomposition
|
||||
- [ ] Bounded context generation per task
|
||||
- [ ] Execute phase invokes `execution_actor` with strategy output
|
||||
- [ ] Receives precise dependency closure
|
||||
- [ ] Works within sandbox boundaries
|
||||
- [ ] Can spawn subplans based on strategy decisions
|
||||
- [ ] Actors receive plan description, project resources, and arguments
|
||||
- [ ] Actor output flows to next phase
|
||||
- [ ] Location: `src/cleveragents/application/services/plan_lifecycle_service.py`
|
||||
- [ ] Tests: End-to-end tests for full plan lifecycle with actors
|
||||
- [ ] Tests: Dependency closure computation accuracy
|
||||
- [ ] Tests: Large codebase task decomposition
|
||||
|
||||
**M3 SUCCESS CRITERIA**:
|
||||
- [ ] Can define actors in YAML with skills
|
||||
@@ -620,8 +686,17 @@ Execute all required tests through the appropriate `nox` sessions—never call `
|
||||
- [ ] Each decision gets a ULID and is persisted
|
||||
- [ ] Build decision tree from parent_decision_id relationships
|
||||
- [ ] Root decision is the plan's prompt_definition
|
||||
- [ ] Capture complete context snapshots:
|
||||
- [ ] Hot context hash (cryptographic)
|
||||
- [ ] Full context snapshot storage
|
||||
- [ ] List of all relevant resources
|
||||
- [ ] Actor state checkpoint
|
||||
- [ ] Record alternatives considered and confidence scores
|
||||
- [ ] Implement semantic validation at decision time
|
||||
- [ ] Location: `src/cleveragents/application/services/decision_service.py`
|
||||
- [ ] Tests: Verify decision tree is built during Strategize
|
||||
- [ ] Tests: Context snapshot preservation and retrieval
|
||||
- [ ] Tests: Decision replay from context
|
||||
|
||||
- [ ] **Stage D3: Decision CLI** (Day 11)
|
||||
- [ ] Code: Decision viewing and correction commands
|
||||
@@ -742,11 +817,29 @@ Execute all required tests through the appropriate `nox` sessions—never call `
|
||||
- [ ] **Stage G1: Automation Levels** (Day 26-27)
|
||||
- [ ] Code: Implement automation level support
|
||||
- [ ] Manual: User prompted for each decision
|
||||
- [ ] Show context, alternatives, recommendation
|
||||
- [ ] Accept explicit choice or custom guidance
|
||||
- [ ] Record user decisions in decision tree
|
||||
- [ ] Review-before-apply: Auto decisions, user reviews before Apply
|
||||
- [ ] AI makes all decisions autonomously
|
||||
- [ ] Execution completes in sandbox
|
||||
- [ ] Human reviews complete diff before apply
|
||||
- [ ] User can approve, reject, or correct specific decisions
|
||||
- [ ] Full automation: All decisions automatic
|
||||
- [ ] AI makes all decisions
|
||||
- [ ] Execution proceeds through apply
|
||||
- [ ] Human notified of completion
|
||||
- [ ] Rollback available if issues detected
|
||||
- [ ] Configurable per plan/session/global
|
||||
- [ ] Progressive trust building:
|
||||
- [ ] Track decision success rates
|
||||
- [ ] Build codebase familiarity scores
|
||||
- [ ] Confidence-based escalation even in full auto mode
|
||||
- [ ] Semantic understanding of when to escalate
|
||||
- [ ] Location: Update plan lifecycle service
|
||||
- [ ] Tests: Tests for each automation level
|
||||
- [ ] Tests: Confidence-based escalation scenarios
|
||||
- [ ] Tests: Progressive automation upgrade paths
|
||||
|
||||
- [ ] **Stage G2: Checkpointing & Rollback** (Day 28-29)
|
||||
- [ ] Code: Enhanced checkpointing
|
||||
@@ -756,14 +849,49 @@ Execute all required tests through the appropriate `nox` sessions—never call `
|
||||
- [ ] Location: `src/cleveragents/infrastructure/checkpoint/`
|
||||
- [ ] Tests: Checkpoint/rollback tests
|
||||
|
||||
- [ ] **Stage G2.5: Semantic Validation Framework** (Day 29-30)
|
||||
- [ ] Code: Multi-layer semantic error prevention
|
||||
- [ ] Decision-time validation in Strategize
|
||||
- [ ] Alternatives evaluation with confidence scores
|
||||
- [ ] Validation checks before committing to decision
|
||||
- [ ] Execution-time semantic guards
|
||||
- [ ] API compatibility checking
|
||||
- [ ] Breaking change detection
|
||||
- [ ] Auto-migration generation where possible
|
||||
- [ ] Invariant enforcement system
|
||||
- [ ] User-defined invariants
|
||||
- [ ] Invariant checking during execution
|
||||
- [ ] Violation handling and recovery
|
||||
- [ ] Error pattern database
|
||||
- [ ] Learn from historical failures
|
||||
- [ ] Predictive error prevention
|
||||
- [ ] Pattern matching for known issues
|
||||
- [ ] Location: `src/cleveragents/application/services/semantic_validation_service.py`
|
||||
- [ ] Tests: Semantic validation scenarios
|
||||
- [ ] Tests: Invariant violation detection
|
||||
- [ ] Tests: Error pattern learning
|
||||
|
||||
- [ ] **Stage G3: Context Tiers** (Day 30-31)
|
||||
- [ ] Code: Hot/warm/cold context management
|
||||
- [ ] Hot context: immediate LLM access
|
||||
- [ ] Hot context: immediate LLM access (10-20 files for active work)
|
||||
- [ ] Warm context: quick retrieval
|
||||
- [ ] Recent decisions from current plan tree
|
||||
- [ ] Indexed embeddings and vector search results
|
||||
- [ ] Decision chain that led to current work
|
||||
- [ ] Cold context: archived/compressed
|
||||
- [ ] Historical decisions from past plans
|
||||
- [ ] Past refactoring patterns
|
||||
- [ ] Cross-project learnings
|
||||
- [ ] Per-actor context views
|
||||
- [ ] Strategist: architecture docs, READMEs, module boundaries
|
||||
- [ ] Executor: precise code sections for edits
|
||||
- [ ] Reviewer: diffs, tests, risk zones
|
||||
- [ ] Promotion/demotion algorithms
|
||||
- [ ] Context locality preservation for massive codebases
|
||||
- [ ] Location: `src/cleveragents/application/services/context_tier_service.py`
|
||||
- [ ] Tests: Context tier tests
|
||||
- [ ] Tests: Large codebase simulation (50k+ files)
|
||||
- [ ] Tests: Context view filtering by actor type
|
||||
|
||||
- [ ] **Stage G4: Cost & Risk Estimation** (Day 32-33)
|
||||
- [ ] Code: Optional estimation actor
|
||||
|
||||
+284
-55
@@ -1,4 +1,4 @@
|
||||
# CleverAgents v3 Documentation (Detailed Spec)
|
||||
# CleverAgents Documentation (Detailed Spec)
|
||||
|
||||
## Source Material
|
||||
|
||||
@@ -10,15 +10,15 @@ Key themes include:
|
||||
* A scalable **context/memory architecture** (hot/warm/cold tiers, per-actor views).
|
||||
* A future-facing correction model where the user can "edit the decision tree" and only recompute affected subtrees.
|
||||
|
||||
# Big Picture: What CleverAgents v3 *is*
|
||||
# Big Picture: What CleverAgents *is*
|
||||
|
||||
CleverAgents v3 is your **command center for AI agents**—a unified platform for orchestrating any task you want agents to accomplish, from developing large software projects to writing comprehensive technical papers, administering databases, managing cloud infrastructure, or any complex multi-step workflow. The core value proposition is enabling **long-running, complex, large-scale tasks to execute autonomously with minimal human intervention**, making it ideal for building entire software systems, producing extensive documentation, or managing sophisticated operations largely hands-off.
|
||||
CleverAgents is your **command center for AI agents**—a unified platform for orchestrating any task you want agents to accomplish, from developing large software projects to writing comprehensive technical papers, administering databases, managing cloud infrastructure, or any complex multi-step workflow. The core value proposition is enabling **long-running, complex, large-scale tasks to execute autonomously with minimal human intervention**, making it ideal for building entire software systems, producing extensive documentation, or managing sophisticated operations largely hands-off.
|
||||
|
||||
In **server mode**, CleverAgents becomes a collaborative hub where teams can share resources—prompts, actors, actions, and projects—while executing plans in the cloud. This enables a consistent experience across all your devices: start a complex task on your laptop, check progress from your phone, and review results from any machine.
|
||||
|
||||
While CleverAgents leverages LangGraph and LangChain for the underlying LLM runtime primitives (tool calling, graphs, routing), its value lies in what it builds on top:
|
||||
|
||||
* **CleverAgents v3** provides:
|
||||
* **CleverAgents** provides:
|
||||
|
||||
* A **first-class plan lifecycle** (Action/Strategize/Execute/Apply) for breaking down and tracking complex work,
|
||||
* A **project + resource model** for grounding tasks in real codebases, databases, documents, and infrastructure,
|
||||
@@ -48,7 +48,7 @@ While CleverAgents leverages LangGraph and LangChain for the underlying LLM runt
|
||||
|
||||
## Plan
|
||||
|
||||
A **plan** is the fundamental unit of orchestration and traceability in v3.
|
||||
A **plan** is the fundamental unit of orchestration and traceability.
|
||||
|
||||
### Plan Lifecycle Phases
|
||||
|
||||
@@ -67,7 +67,7 @@ This four-stage model is explicitly called out as the new architecture replacing
|
||||
|
||||
### Phase Transition Verbs (CLI / UX Contract)
|
||||
|
||||
Your draft proposes verbs that trigger phase transitions. v3 should standardize these verbs as the **public API** (CLI, TUI, web):
|
||||
Your draft proposes verbs that trigger phase transitions. CleverAgents should standardize these verbs as the **public API** (CLI, TUI, web):
|
||||
|
||||
| Current Phase | Command Verb | Next Phase |
|
||||
| ------------- | ------------ | ---------- |
|
||||
@@ -122,6 +122,30 @@ A single plan should usually represent the smallest "complete" unit of work (sim
|
||||
|
||||
This is core to the long-term objective: tackling large tasks while only recomputing parts of the decision tree when corrected.
|
||||
|
||||
#### Hierarchical Decomposition for Scale
|
||||
|
||||
When handling massive tasks (e.g., converting Firefox to Rust), the system uses hierarchical decomposition:
|
||||
|
||||
1. **Root Level**: High-level architectural decisions
|
||||
- "Convert Firefox Renderer to Rust"
|
||||
- Decision: "Start with leaf modules, work inward"
|
||||
- Context: Module dependency graph (2,847 modules)
|
||||
|
||||
2. **Subsystem Level**: Major component decisions
|
||||
- "Phase 1: Convert utility libraries (no external deps)"
|
||||
- Each subsystem gets its own bounded context
|
||||
|
||||
3. **Module Level**: Individual module conversions
|
||||
- "Convert string_utils module"
|
||||
- Context: Only the 47 functions and 12 dependent files
|
||||
- Decision: "Use Rust's String type"
|
||||
|
||||
4. **File Level**: Specific file changes
|
||||
- Actual code transformations
|
||||
- Minimal context needed
|
||||
|
||||
At each level, only the relevant context is loaded. The persistent decision graph means we can always reconstruct why we're converting a particular module and what constraints apply from higher-level decisions.
|
||||
|
||||
#### Subplan Spawning Mechanism
|
||||
|
||||
In the actor definition for the execution actor, there are **nodes that act as skills** whose purpose is to generate subplans. The execution actor can call these skills to trigger subplans.
|
||||
@@ -170,7 +194,7 @@ The way results are merged depends on the resource type:
|
||||
|
||||
### The Plan "Decision Tree" and Visualization
|
||||
|
||||
CleverAgents v3 intends to record enough information to render:
|
||||
CleverAgents intends to record enough information to render:
|
||||
|
||||
* an **ASCII tree** in the TUI, and
|
||||
* optionally a GUI tree via visualization tools (D3/Cytoscape) once the data exists.
|
||||
@@ -335,10 +359,18 @@ Decision:
|
||||
|
||||
# Context Snapshot (for replay)
|
||||
context_snapshot:
|
||||
hot_context_hash: str # Hash of hot context at decision time
|
||||
hot_context_ref: str # Reference to stored snapshot
|
||||
relevant_resources: list[ResourceRef]
|
||||
actor_state_ref: str # Reference to LangGraph checkpoint
|
||||
hot_context_hash: str # Cryptographic hash of the exact context
|
||||
hot_context_ref: str # Pointer to the full stored snapshot
|
||||
relevant_resources: list[ResourceRef] # Every file/symbol that influenced this decision
|
||||
actor_state_ref: str # Complete LangGraph checkpoint
|
||||
|
||||
# When the system decides "refactor the authentication module to use async patterns,"
|
||||
# it permanently records:
|
||||
# - Which files were examined to make that decision
|
||||
# - What symbols and dependencies were traced
|
||||
# - The exact code state that was analyzed
|
||||
# - The reasoning chain that led to this choice
|
||||
# - Alternative approaches that were considered but rejected
|
||||
|
||||
# Rationale
|
||||
rationale: str # Why this option was chosen
|
||||
@@ -621,6 +653,37 @@ Strategize is:
|
||||
|
||||
This "architect vs coder" separation is explicitly described as a core motivation.
|
||||
|
||||
**Resource-aware dependency analysis**:
|
||||
During the Strategize phase, the strategy actor employs specialized mechanisms to compute precise dependency closures:
|
||||
|
||||
```python
|
||||
# Pseudocode of what happens inside a strategy actor
|
||||
def compute_closure_for_refactoring(target_module):
|
||||
closure = ResourceClosure()
|
||||
|
||||
# Direct file dependencies
|
||||
closure.add_files(find_imports(target_module))
|
||||
closure.add_files(find_includes(target_module))
|
||||
|
||||
# Symbol dependencies
|
||||
for symbol in extract_exported_symbols(target_module):
|
||||
closure.add_files(find_symbol_usage(symbol, scope='project'))
|
||||
|
||||
# Test dependencies
|
||||
closure.add_files(find_tests_for_module(target_module))
|
||||
|
||||
# Build system dependencies
|
||||
closure.add_files(find_build_references(target_module))
|
||||
|
||||
return closure
|
||||
```
|
||||
|
||||
The system leverages several key insights:
|
||||
- **Modular boundaries exist**: Even in legacy codebases, there are natural boundaries
|
||||
- **Changes are incremental**: We don't convert 50,000 files atomically
|
||||
- **Dependencies are sparse**: Most modules depend on a small fraction of the codebase
|
||||
- **Interfaces are narrow**: Public APIs are much smaller than implementations
|
||||
|
||||
### Strategize Data Model
|
||||
|
||||
A plan in Strategize contains all Action fields plus:
|
||||
@@ -804,6 +867,40 @@ When a plan accesses multiple resources:
|
||||
* Apply commits each sandbox separately
|
||||
* If any sandbox Apply fails, others may still succeed (partial apply)
|
||||
|
||||
**Complete isolation during execution prevents compound errors**:
|
||||
Each plan executes in its own sandbox, which means:
|
||||
```
|
||||
Plan A (refactoring auth module):
|
||||
- Sandbox A1: Contains only auth/*.cpp, auth_tests/*.cpp
|
||||
- Cannot see Plan B's intermediate states
|
||||
- Cannot accidentally depend on Plan B's half-done work
|
||||
|
||||
Plan B (updating API endpoints):
|
||||
- Sandbox B1: Contains only api/*.cpp, api_tests/*.cpp
|
||||
- Makes changes assuming current auth interface
|
||||
- Protected from Plan A's intermediate refactoring
|
||||
```
|
||||
|
||||
**Hierarchical merge resolution**:
|
||||
When subplans complete, the parent plan performs intelligent merging:
|
||||
```python
|
||||
def merge_subplan_results(subplan_results):
|
||||
# Group by resource type
|
||||
by_resource = group_by_resource_type(subplan_results)
|
||||
|
||||
# Apply resource-specific merge strategies
|
||||
for resource_type, changes in by_resource:
|
||||
if resource_type == 'git_repo':
|
||||
merge_git_changes(changes) # Three-way merge
|
||||
elif resource_type == 'database':
|
||||
merge_db_changes(changes) # Sequential application
|
||||
elif resource_type == 'config_files':
|
||||
merge_config_changes(changes) # Smart JSON/YAML merge
|
||||
|
||||
# Validate merged state
|
||||
run_integration_tests()
|
||||
```
|
||||
|
||||
### Execution Data Model
|
||||
|
||||
A plan in Execute contains:
|
||||
@@ -905,7 +1002,7 @@ The notes call out planned improvements:
|
||||
* Improve file structure generation.
|
||||
* Handle invalid python responses better than wrapping in docstrings.
|
||||
|
||||
Therefore, v3 should define a **standard output parsing pipeline**:
|
||||
Therefore, CleverAgents should define a **standard output parsing pipeline**:
|
||||
|
||||
1. Parse model output.
|
||||
2. Identify code blocks and associated filenames (if present).
|
||||
@@ -916,6 +1013,96 @@ Therefore, v3 should define a **standard output parsing pipeline**:
|
||||
* request correction from actor, or
|
||||
* quarantine the output as a "draft artifact" rather than writing it as real source.
|
||||
|
||||
## Semantic Error Prevention
|
||||
|
||||
CleverAgents provides multiple layers of proactive error prevention that catch semantic errors before they can propagate through the system.
|
||||
|
||||
### Layer 1: Decision-time Validation During Strategize
|
||||
|
||||
Every decision includes semantic validation:
|
||||
```yaml
|
||||
Decision: Refactor payment module to async
|
||||
alternatives_considered:
|
||||
- "Convert to async/await patterns" (chosen)
|
||||
- "Use thread pool with channels" (rejected: doesn't integrate with async ecosystem)
|
||||
- "Keep synchronous with timeout" (rejected: doesn't solve core latency issue)
|
||||
confidence_score: 0.85
|
||||
validation_performed:
|
||||
- Checked all payment API consumers can handle async
|
||||
- Verified database driver supports async operations
|
||||
- Confirmed no regulatory requirement for sync processing
|
||||
```
|
||||
|
||||
### Layer 2: Execution-time Semantic Guards
|
||||
|
||||
The execution actor configuration includes validation nodes that understand semantics:
|
||||
|
||||
```yaml
|
||||
actors:
|
||||
code_executor:
|
||||
type: graph
|
||||
nodes:
|
||||
- name: semantic_validator
|
||||
type: tool
|
||||
config:
|
||||
tools:
|
||||
- name: validate_api_compatibility
|
||||
code: |
|
||||
# Not just syntax checking - semantic validation
|
||||
old_api = extract_api_signature(previous_version)
|
||||
new_api = extract_api_signature(current_version)
|
||||
|
||||
breaking_changes = find_breaking_changes(old_api, new_api)
|
||||
if breaking_changes:
|
||||
# Don't just fail - understand the impact
|
||||
affected_consumers = find_api_consumers(breaking_changes)
|
||||
migration_plan = generate_migration(breaking_changes)
|
||||
|
||||
if can_auto_migrate(affected_consumers, migration_plan):
|
||||
apply_migration(migration_plan)
|
||||
else:
|
||||
raise SemanticError(
|
||||
"Breaking API changes require manual review",
|
||||
changes=breaking_changes,
|
||||
affected=affected_consumers
|
||||
)
|
||||
```
|
||||
|
||||
### Layer 3: Invariant Enforcement
|
||||
|
||||
The system maintains semantic invariants:
|
||||
```python
|
||||
class RefactoringInvariants:
|
||||
# User-defined invariants for the codebase
|
||||
invariants = [
|
||||
"All public APIs must maintain backward compatibility",
|
||||
"Database transactions must complete within 5 seconds",
|
||||
"Authentication must always use OAuth2",
|
||||
"Payment processing must be idempotent"
|
||||
]
|
||||
|
||||
def check_invariant_preservation(self, changes):
|
||||
for invariant in self.invariants:
|
||||
if not self.verify_invariant(invariant, changes):
|
||||
return InvariantViolation(invariant, changes)
|
||||
return Success()
|
||||
```
|
||||
|
||||
### Layer 4: Predictive Error Prevention
|
||||
|
||||
The system learns from past failures:
|
||||
```yaml
|
||||
Error Pattern Database:
|
||||
- pattern: "Async conversion in payment module"
|
||||
historical_failures:
|
||||
- "Race condition in payment confirmation"
|
||||
- "Timeout handling breaks idempotency"
|
||||
preventive_checks:
|
||||
- "Add explicit transaction boundaries"
|
||||
- "Verify idempotency keys are preserved"
|
||||
- "Check distributed lock acquisition"
|
||||
```
|
||||
|
||||
## Applied (Apply Phase)
|
||||
|
||||
### What Apply Does
|
||||
@@ -1194,7 +1381,7 @@ This enables a pattern where:
|
||||
|
||||
### What an Actor Is
|
||||
|
||||
An **actor** is the v3 abstraction that generalizes "agent" into "anything conversational."
|
||||
An **actor** is the abstraction that generalizes "agent" into "anything conversational."
|
||||
|
||||
* It can be as small as a single LLM agent.
|
||||
* It can also be an entire graph that itself calls other actors/tools.
|
||||
@@ -1321,7 +1508,7 @@ This enables hierarchical composition where:
|
||||
|
||||
The notes describe an actor as a configuration profile with fields like name, provider, model, configuration, blob, and graph description.
|
||||
|
||||
A robust v3 actor schema should include:
|
||||
A robust actor schema should include:
|
||||
|
||||
* `name` (namespaced)
|
||||
* `provider` (LLM provider or runtime target)
|
||||
@@ -1362,7 +1549,7 @@ This is a powerful simplification: "everything is a node."
|
||||
|
||||
### Agent Definition
|
||||
|
||||
In CleverAgents v3, an **agent** is a specialized actor with:
|
||||
In CleverAgents, an **agent** is a specialized actor with:
|
||||
|
||||
* a conversational interface,
|
||||
* tool-calling capability,
|
||||
@@ -1518,7 +1705,7 @@ A session should:
|
||||
|
||||
The notes include a known issue: conversation history can be lost between CLI invocations depending on connection string configuration, implying the system needs a stable memory service backend.
|
||||
|
||||
Therefore, v3 should specify:
|
||||
Therefore, CleverAgents should specify:
|
||||
|
||||
* sessions have stable IDs
|
||||
* sessions can be resumed
|
||||
@@ -1527,7 +1714,7 @@ Therefore, v3 should specify:
|
||||
|
||||
## Server
|
||||
|
||||
### What a Server Is (in v3)
|
||||
### What a Server Is
|
||||
|
||||
A server is an optional mode that enables:
|
||||
|
||||
@@ -1697,7 +1884,7 @@ This enables:
|
||||
|
||||
## Context
|
||||
|
||||
Context in v3 is not "dump all files into an LLM." It is a system that:
|
||||
Context in is not "dump all files into an LLM." It is a system that:
|
||||
|
||||
* finds relevant information from resources,
|
||||
* injects appropriate subsets into each actor/node,
|
||||
@@ -1714,22 +1901,25 @@ The transcript implies:
|
||||
|
||||
### Tiered Context Architecture (Hot/Warm/Cold)
|
||||
|
||||
A proposed architecture:
|
||||
The system uses a sophisticated three-tier memory architecture that enables working with massive codebases without holding everything in memory:
|
||||
|
||||
* **Hot context (hot cache)**
|
||||
The small set of immediately relevant chunks injected into the current actor prompt.
|
||||
The small set of immediately relevant chunks injected into the current actor prompt. When working on a 50,000 file codebase, hot context focuses on the immediate task (e.g., 10-20 files for a specific refactoring).
|
||||
|
||||
* **Warm context**
|
||||
Indexed embeddings / vector search / graph store representation of the codebase.
|
||||
Recent decisions and their contexts from this plan tree - quickly accessible. Includes indexed embeddings, vector search results, and graph store representations. Maintains the decision chain that led to the current work.
|
||||
|
||||
* **Cold storage**
|
||||
Long-term storage in SQLite or caching systems: history, prior summaries, older plan artifacts.
|
||||
Historical decisions from past plans on this codebase - queryable but not in active memory. Long-term storage in SQLite or caching systems containing prior summaries, older plan artifacts, and historical patterns (e.g., "last time we refactored auth, we also had to update these services").
|
||||
|
||||
Promotion/demotion behavior:
|
||||
|
||||
* System analyzes current query,
|
||||
* promotes relevant data upward (cold → warm → hot),
|
||||
* demotes stale data out of hot to keep prompts tight.
|
||||
* System analyzes current query
|
||||
* Promotes relevant data upward (cold → warm → hot)
|
||||
* Demotes stale data out of hot to keep prompts tight
|
||||
* Preserves complete context snapshots for every decision
|
||||
|
||||
This architecture leverages the key insight that software development is inherently local - even in huge codebases, individual changes typically touch a bounded set of files. The Decision Tree captures these localities.
|
||||
|
||||
### Actor-Specific Context Views
|
||||
|
||||
@@ -1756,7 +1946,7 @@ A practical approach mentioned:
|
||||
* Initial context is high-level (repo tree, language, overview),
|
||||
* Then the system searches for relevant details iteratively via RAG.
|
||||
|
||||
This strongly suggests v3 should define:
|
||||
This strongly suggests CleverAgents should define:
|
||||
|
||||
* an "initial context recipe" per project type (codebase vs documents vs infra),
|
||||
* iterative context refinement loops during strategize/execute.
|
||||
@@ -1771,9 +1961,41 @@ Automation levels determine which phase transitions happen automatically.
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| **Manual** | User explicitly triggers: create → use → execute → apply. Every phase transition requires a command. |
|
||||
| **Review-before-apply** | Strategize + Execute happen automatically. System pauses before Apply to show diff and ask for approval. |
|
||||
| **Full automation** | System runs all phases automatically end-to-end. |
|
||||
| **Manual** | User explicitly triggers: create → use → execute → apply. Every phase transition requires a command. Every decision point pauses for human input. User sees context, alternatives, and recommendation. User provides explicit choice or custom guidance. |
|
||||
| **Review-before-apply** | Strategize + Execute happen automatically. AI makes all decisions autonomously. System pauses before Apply to show diff and ask for approval. User can approve, reject, or correct specific decisions. |
|
||||
| **Full automation** | System runs all phases automatically end-to-end. AI makes all decisions. Execution proceeds through apply. Human notified of completion. Rollback available if issues detected. |
|
||||
|
||||
### Progressive Trust Building
|
||||
|
||||
New users typically follow this progression:
|
||||
1. Start with manual mode to understand system behavior
|
||||
2. Move to review-before-apply as confidence builds
|
||||
3. Enable full automation for specific task types
|
||||
4. Gradually expand full automation scope
|
||||
|
||||
### Semantic Escalation
|
||||
|
||||
Even in full automation mode, the system understands when it needs help:
|
||||
|
||||
```python
|
||||
class AutonomyController:
|
||||
def assess_decision_confidence(self, decision, context):
|
||||
factors = {
|
||||
'past_success_rate': self.get_historical_success(decision.type),
|
||||
'codebase_familiarity': self.get_familiarity_score(context.project),
|
||||
'risk_assessment': self.evaluate_risk(decision),
|
||||
'invariant_complexity': self.analyze_invariants(decision)
|
||||
}
|
||||
|
||||
confidence = self.compute_confidence(factors)
|
||||
|
||||
if confidence < self.threshold:
|
||||
if self.automation_level == 'full':
|
||||
# Even in full automation, critical decisions escalate
|
||||
return RequestHumanGuidance(decision, factors)
|
||||
|
||||
return ProceedAutonomously(decision)
|
||||
```
|
||||
|
||||
### Automation Level Hierarchy
|
||||
|
||||
@@ -1848,15 +2070,15 @@ The notes mention future concerns:
|
||||
|
||||
* API call limits
|
||||
* cost caps
|
||||
So v3 should define:
|
||||
So CleverAgents should define:
|
||||
* per-plan budgets
|
||||
* per-session budgets
|
||||
* per-org budgets
|
||||
* per-actor max tool calls / max retries
|
||||
|
||||
## Correcting Plans (Core v3 Feature)
|
||||
## Correcting Plans (Core Feature)
|
||||
|
||||
Correcting plans is where v3 becomes more than "a fancy prompt runner."
|
||||
Correcting plans is where CleverAgents becomes more than "a fancy prompt runner."
|
||||
|
||||
### The Goal
|
||||
|
||||
@@ -1981,7 +2203,7 @@ This keeps history reproducible and prevents accidental destructive edits.
|
||||
|
||||
Even though the direction is "more autonomous," the transcript explicitly recognizes that real workflows require engineers to collaborate with the system, editing code while it works, and using better UX integration (TUI/web/IDE).
|
||||
|
||||
So v3 should aim for:
|
||||
So ClverAgents should aim for:
|
||||
|
||||
* visibility: what is it doing now?
|
||||
* interruptibility: pause/cancel/retry
|
||||
@@ -2019,7 +2241,7 @@ And should later allow exporting the tree as image (PNG) or JSON for other visua
|
||||
|
||||
## Storage and Persistence
|
||||
|
||||
v3 should define where each concept lives:
|
||||
CleverAgents should define where each concept lives:
|
||||
|
||||
* Actions: stored in a registry (local files or server DB)
|
||||
* Actors: stored similarly (config files + DB indexing)
|
||||
@@ -2063,6 +2285,13 @@ To debug large plans:
|
||||
* Context should evolve toward **hot/warm/cold tiers** and **actor-specific context views**.
|
||||
* The system is designed for large tasks where the user can **correct a decision** and only recompute downstream work, visualizable as a plan decision tree.
|
||||
|
||||
The system can handle Firefox-scale projects not through magic, but through:
|
||||
- **Hierarchical decomposition** breaking massive tasks into bounded work
|
||||
- **Persistent decision graphs** maintaining context across any scale
|
||||
- **Isolated execution** preventing cascading failures
|
||||
- **Semantic validation** catching errors before propagation
|
||||
- **Progressive automation** building trust through incremental success
|
||||
|
||||
If you want, I can also produce:
|
||||
|
||||
* a canonical **JSON/YAML schema** for Actions, Actors, Projects, Plans, Skills, and Context Views,
|
||||
@@ -2071,15 +2300,15 @@ If you want, I can also produce:
|
||||
|
||||
# Work Remaining to Make CleverAgents Fully Functional
|
||||
|
||||
This section describes—**exhaustively and in implementation terms**—what remains to be done to bring the current CleverAgents codebase up to the intended CleverAgents v3 behavior described in the transcript/meeting notes (Action → Strategize → Execute → Apply; actors as composable graphs; sandbox + diff review; context scaling; checkpointable skills; etc.).
|
||||
This section describes—**exhaustively and in implementation terms**—what remains to be done to bring the current CleverAgents codebase up to the intended CleverAgents behavior described in the transcript/meeting notes (Action → Strategize → Execute → Apply; actors as composable graphs; sandbox + diff review; context scaling; checkpointable skills; etc.).
|
||||
|
||||
It is based primarily on the attached "what's missing / code analysis" document (last updated **Jan 29, 2026**) which outlines the current master branch's functional gaps, architectural limitations, and regressions introduced by new reactive/langgraph code.
|
||||
|
||||
---
|
||||
|
||||
## Current State vs Intended v3: The Gap in One Sentence
|
||||
## Current State vs Intended: The Gap in One Sentence
|
||||
|
||||
Right now, the codebase still behaves like a **linear, single-file "LLM dump to generated.py" pipeline** (with tiny context and stubbed validation), while intended v3 requires a **plan lifecycle engine that can reliably generate, validate, sandbox, diff-review, and apply multi-file/multi-project changes—backed by checkpointable skills, rich context, and composable actor graphs**.
|
||||
Right now, the codebase still behaves like a **linear, single-file "LLM dump to generated.py" pipeline** (with tiny context and stubbed validation), while intended CleverAgents requires a **plan lifecycle engine that can reliably generate, validate, sandbox, diff-review, and apply multi-file/multi-project changes—backed by checkpointable skills, rich context, and composable actor graphs**.
|
||||
|
||||
Everything below is the concrete work required to close that gap.
|
||||
|
||||
@@ -2252,7 +2481,7 @@ But the "plan must work" path should be stable with deterministic orchestration.
|
||||
|
||||
The analysis notes that apply currently writes files directly and can leave the codebase inconsistent on partial failures; diff preview is missing or insufficient; rollback is not robust.
|
||||
|
||||
### What "done" looks like (v3 expectation)
|
||||
### What "done" looks like (expectation)
|
||||
|
||||
* Execute never mutates the real project directly.
|
||||
* All changes happen in a sandbox.
|
||||
@@ -2357,7 +2586,7 @@ The system needs to know:
|
||||
* checkpointability
|
||||
* side effects
|
||||
|
||||
The meeting notes explicitly call out that MCP doesn't provide enough writing-scope metadata for safe automation; v3 needs an internal extension/registry.
|
||||
The meeting notes explicitly call out that MCP doesn't provide enough writing-scope metadata for safe automation; CleverAgents needs an internal extension/registry.
|
||||
|
||||
---
|
||||
|
||||
@@ -2586,7 +2815,7 @@ If templating is needed, use a sandboxed template engine or restrict tokens seve
|
||||
|
||||
### Problem (today)
|
||||
|
||||
The analysis describes the current master as still conceptually the linear tell/build/apply pipeline. Meanwhile, intended v3 introduces reusable Actions decoupled from Projects, and separate Strategize/Execute/Apply phases with automation levels.
|
||||
The analysis describes the current master as still conceptually the linear tell/build/apply pipeline. Meanwhile, intended CleverAgents introduces reusable Actions decoupled from Projects, and separate Strategize/Execute/Apply phases with automation levels.
|
||||
|
||||
### What "done" looks like
|
||||
|
||||
@@ -2757,7 +2986,7 @@ This ordering matches the reality that without multi-file output and safe apply,
|
||||
|
||||
## Acceptance Criteria for "Fully Functional" (Concrete)
|
||||
|
||||
A realistic "fully functional" bar (aligned to intended v3 behavior) is:
|
||||
A realistic "fully functional" bar (aligned to intended behavior) is:
|
||||
|
||||
* Can **strategize** read-only using real context across many files.
|
||||
* Can **execute** in sandbox, producing a multi-file ChangeSet for non-trivial features.
|
||||
@@ -2771,11 +3000,11 @@ A realistic "fully functional" bar (aligned to intended v3 behavior) is:
|
||||
|
||||
Everything listed earlier maps directly to closing the gaps documented in the attached analysis and to achieving the behaviors described in the transcript.
|
||||
|
||||
# CleverAgents v3 Architecture FAQ
|
||||
# CleverAgents Architecture FAQ
|
||||
|
||||
## Q: How does CleverAgents v3 handle persistent repository knowledge beyond ephemeral context windows?
|
||||
## Q: How does CleverAgents handle persistent repository knowledge beyond ephemeral context windows?
|
||||
|
||||
**What exists today architecturally**: The v3 specification defines a sophisticated multi-tier memory system that goes far beyond ephemeral context windows. At its core is the Decision Tree structure (lines 187-426) which provides a durable, queryable record of every choice made during planning, along with the complete context that informed those choices.
|
||||
**What exists today architecturally**: The specification defines a sophisticated multi-tier memory system that goes far beyond ephemeral context windows. At its core is the Decision Tree structure which provides a durable, queryable record of every choice made during planning, along with the complete context that informed those choices.
|
||||
|
||||
**How the persistent model works in practice**:
|
||||
|
||||
@@ -2838,7 +3067,7 @@ Even months later, we can query: "Why did we use Rust's String type?" and get th
|
||||
|
||||
## Q: How does the system compute task-specific dependency closures for large-scale operations?
|
||||
|
||||
**What exists today architecturally**: The v3 specification defines multiple mechanisms for computing and maintaining minimal dependency closures. The execution blueprint (line 671) produced during the Strategize phase doesn't just list steps - it includes a complete dependency graph with explicit scoping for each operation.
|
||||
**What exists today architecturally**: The specification defines multiple mechanisms for computing and maintaining minimal dependency closures. The execution blueprint produced during the Strategize phase doesn't just list steps - it includes a complete dependency graph with explicit scoping for each operation.
|
||||
|
||||
**How dependency closure computation works**:
|
||||
|
||||
@@ -2867,7 +3096,7 @@ During the Strategize phase, the strategy actor employs several mechanisms to co
|
||||
return closure
|
||||
```
|
||||
|
||||
2. **Hierarchical scoping**: When spawning subplans (lines 127-170), each subplan receives:
|
||||
2. **Hierarchical scoping**: When spawning subplans, each subplan receives:
|
||||
- An explicit `relevant_resources` list
|
||||
- A `sandbox_strategy` appropriate for those resources
|
||||
- Clear boundaries of what it can and cannot modify
|
||||
@@ -2926,7 +3155,7 @@ The Firefox example would decompose into ~1,000 bounded subplans, each touching
|
||||
|
||||
## Q: What mechanisms enforce global consistency during parallel execution across many files?
|
||||
|
||||
**What exists today architecturally**: The sandbox model (lines 723-817) combined with hierarchical plan execution provides strong guarantees about consistency during parallel execution. This isn't just process isolation - it's semantic isolation with intelligent merge strategies.
|
||||
**What exists today architecturally**: The sandbox model combined with hierarchical plan execution provides strong guarantees about consistency during parallel execution. This isn't just process isolation - it's semantic isolation with intelligent merge strategies.
|
||||
|
||||
**How the coordination mechanism prevents compound errors**:
|
||||
|
||||
@@ -2996,7 +3225,7 @@ PARALLEL EXECUTION WITHOUT COORDINATION (what we prevent):
|
||||
- Service C changes error handling → breaks Services D, E, F
|
||||
- Cascade of failures!
|
||||
|
||||
CLEVERAGENTS V3 COORDINATED EXECUTION:
|
||||
CLEVERAGENTS COORDINATED EXECUTION:
|
||||
Parent Plan: Refactor auth library
|
||||
├── Subplan 1: Update auth library interface
|
||||
│ Sandbox: Only auth library files
|
||||
@@ -3051,7 +3280,7 @@ Parent Plan: Refactor auth library
|
||||
|
||||
## Q: How does the system proactively prevent semantic errors before they propagate?
|
||||
|
||||
**What exists today architecturally**: The v3 specification defines multiple layers of proactive error prevention that go far beyond traditional testing. This is a comprehensive defense-in-depth approach that catches semantic errors before they can propagate.
|
||||
**What exists today architecturally**: The specification defines multiple layers of proactive error prevention that go far beyond traditional testing. This is a comprehensive defense-in-depth approach that catches semantic errors before they can propagate.
|
||||
|
||||
**Layer 1: Decision-time validation during Strategize**:
|
||||
|
||||
@@ -3196,7 +3425,7 @@ The validation nodes enforce these semantics, not just test passage.
|
||||
|
||||
## Q: How does the system balance human supervision with autonomous operation?
|
||||
|
||||
**What exists today architecturally**: The v3 specification defines a sophisticated gradation of automation levels (lines 203-213) that precisely controls when human intervention is needed. This isn't a binary human/AI split - it's a spectrum that can be adjusted per task, per project, or per organization.
|
||||
**What exists today architecturally**: The specification defines a sophisticated gradation of automation levels that precisely controls when human intervention is needed. This isn't a binary human/AI split - it's a spectrum that can be adjusted per task, per project, or per organization.
|
||||
|
||||
**How the automation levels work in practice**:
|
||||
|
||||
@@ -3224,7 +3453,7 @@ Full Automation Mode:
|
||||
|
||||
**The decision correction mechanism enables progressive automation**:
|
||||
|
||||
The `agents plan correct` command (lines 263-294) is crucial for building trust:
|
||||
The `agents plan correct` command is crucial for building trust:
|
||||
|
||||
```bash
|
||||
# User observes AI made suboptimal choice
|
||||
@@ -3337,16 +3566,16 @@ The system becomes more autonomous through:
|
||||
|
||||
## Q: What's actually implemented today versus planned for the future?
|
||||
|
||||
**Concrete implementations in the v3 architecture**:
|
||||
**Concrete implementations in the architecture**:
|
||||
|
||||
1. **Decision Tree with Complete Context Capture**
|
||||
- Full schema defined (lines 373-426)
|
||||
- Full schema defined
|
||||
- Storage model specified
|
||||
- Correction mechanism detailed
|
||||
- Query patterns established
|
||||
|
||||
2. **Hierarchical Plan/Subplan System**
|
||||
- Spawning mechanism defined (lines 127-170)
|
||||
- Spawning mechanism defined
|
||||
- Execution semantics specified
|
||||
- Merge strategies documented
|
||||
- Failure handling described
|
||||
@@ -3428,4 +3657,4 @@ The difference between handling a 1,000 file project and a 100,000 file project
|
||||
- Better context selection (improves with use but works with heuristics)
|
||||
- More validation patterns (accumulate over time)
|
||||
|
||||
This isn't speculative architecture astronautics - it's applying proven distributed systems principles to AI agent coordination. The innovation is in the integration, not in requiring fundamental breakthroughs.
|
||||
This isn't speculative architecture astronautics - it's applying proven distributed systems principles to AI agent coordination. The innovation is in the integration, not in requiring fundamental breakthroughs.
|
||||
|
||||
Reference in New Issue
Block a user