diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 000000000..05380481f --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,722 @@ +# CleverAgents Architecture FAQ + +### Q: How does CleverAgents handle persistent repository knowledge beyond ephemeral context windows? + +**What exists today architecturally**: The specification defines the Advanced Context Management System (ACMS) — a sophisticated, pluggable, strategy-driven framework that goes far beyond ephemeral context windows. At its core are three innovations: (1) the Universal Knowledge Ontology (UKO), an inheritance-based RDF ontology that represents any resource at multiple abstraction levels with full provenance and temporal versioning; (2) the Context Request Protocol (CRP), through which actors dynamically declare what information they need at what detail depth; and (3) a Context Assembly Pipeline — a 10-component pluggable pipeline that orchestrates strategy selection, budget allocation, parallel execution, fragment deduplication, depth resolution, scoring, budget packing, ordering, preamble generation, and skeleton compression. Each component is backed by a replaceable Protocol implementation configurable at global, project, or plan scope. Underlying this is the Decision Tree structure which provides a durable, queryable record of every choice made during planning. + +**How the persistent model works in practice**: + +When a strategy actor analyzes a codebase during the Strategize phase, the ACMS assembles context dynamically using the CRP. The actor issues `request_context` calls specifying focus nodes (UKO URIs), breadth (hop count), depth (an integer or named level like `SIGNATURES`, `FULL_SOURCE`), and optional depth gradients. The Context Assembly Pipeline then orchestrates the full assembly: the StrategySelector chooses applicable strategies (e.g., `breadth-depth-navigator` for graph traversal, `semantic-embedding` for similarity search, `plan-decision-context` for historical decisions), the BudgetAllocator distributes the token budget, the StrategyExecutor runs them in parallel, and the fusion phase (FragmentDeduplicator, DetailDepthResolver, FragmentScorer, BudgetPacker, FragmentOrderer) deduplicates, resolves detail conflicts, scores, and packs the results into the available budget. + +Each decision creates a comprehensive Decision record that includes: + +

+context_snapshot:
+  hot_context_hash: str # Cryptographic hash of the exact context
+  hot_context_ref: str # Pointer to the full stored snapshot
+  relevant_resources: list[ResourceRef] # Every file/symbol that influenced this decision
+  actor_state_ref: str # Complete LangGraph checkpoint
+  uko_focus_nodes: list[UKO_URI] # UKO nodes that were the focus of this context assembly
+  strategies_used: list[str] # Which ACMS strategies contributed to this context
+
+ +This means when the system decides "refactor the authentication module to use async patterns," it permanently records: +- Which UKO nodes (files, classes, functions) were the focus of context assembly +- Which strategies retrieved context and at what detail depths +- What symbols and dependencies were traced via UKO graph traversal +- The exact code state that was analyzed (with temporal versioning via `validFrom`/`validUntil`) +- The reasoning chain that led to this choice +- Alternative approaches that were considered but rejected + +**The three-tier memory architecture enables scale**: + +1. **Hot tier**: Immediate working context assembled by the ACMS — the Context Assembly Pipeline's output loaded into the LLM context window, dynamically computed based on the actor's model context window minus response reserve and tool definitions +2. **Warm tier**: Recent decisions and their contexts from this plan tree — quickly accessible, retained for `context.tiers.warm.retention-hours` (default: 24h), queryable via the `plan-decision-context` strategy +3. **Cold tier**: Historical decisions from past plans on this codebase — queryable via `temporal-archaeology` strategy, retained for `context.tiers.cold.retention-days` (default: 90 days) + +When working on a 50,000 file codebase, the system doesn't need to hold all files in memory. Instead: +- Hot context focuses on the immediate task via demand-driven CRP requests (e.g., `request_context` with focus on 10-20 UKO nodes, breadth=2, depth_gradient={0:9, 1:4, 2:0}) +- Warm context maintains the decision chain via the `plan-decision-context` strategy, with skeleton compression propagating parent plan context to child plans +- Cold context provides historical patterns via the `temporal-archaeology` strategy ("last time we refactored auth, we also had to update these services") + +**Why this scales to massive codebases**: + +The key insight is that software development is inherently local - even in huge codebases, individual changes typically touch a bounded set of files. The Decision Tree captures these localities. When converting Firefox to Rust (your example), the system would: + +1. Make high-level architectural decisions and enforce invariants (captured as root decision nodes) +2. Decompose into major subsystem conversions (each a `subplan_parallel_spawn` or `subplan_spawn` decision spawning child plans) +3. Each subsystem plan makes decisions about its modules, inheriting applicable invariants +4. Module plans make decisions about individual files + +At each level, only the relevant context is loaded. The persistent decision graph means we can always reconstruct why we're converting a particular module and what constraints apply from higher-level decisions. + +**Concrete example of persistence in action**: + +

+Plan: Convert Firefox Renderer to Rust
+├── [invariant_enforced] "Maintain API compatibility with existing C++ callers"
+├── [invariant_enforced] "All converted modules must pass existing C++ test suites"
+├── [strategy_choice] Architecture approach: Start with leaf modules, work inward
+│   Context: Analyzed module dependency graph, 2,847 modules total
+│   Resources: module_graph.json, architecture_docs.md
+│   
+├── [subplan_parallel_spawn] Phase 1: Convert utility libraries (no external deps)
+│   └── [subplan_spawn] Convert string_utils module
+│       └── Plan: 01KH29R8WPKPBHRY7Q0NA9XW86
+│           ├── [prompt_definition] "Convert string_utils module to Rust"
+│           ├── [invariant_enforced] "Maintain API compatibility with existing C++ callers"
+│           ├── [implementation_choice] Use Rust's String type, not custom implementation
+│           │   Context: Analyzed 47 string_utils.cpp functions
+│           │   Resources: string_utils.cpp, string_utils.h, 12 dependent files
+│           │   Rationale: Rust's String provides same guarantees with better ergonomics
+│           └── ...
+
+ +Even months later, we can query: "Why did we use Rust's String type?" and get the exact context and reasoning, without reprocessing the entire codebase. + +### Q: How does the system compute task-specific dependency closures for large-scale operations? + +**What exists today architecturally**: The specification defines multiple mechanisms for computing and maintaining minimal dependency closures. The execution blueprint produced during the Strategize phase doesn't just list steps - it includes a complete dependency graph with explicit scoping for each operation. + +**How dependency closure computation works**: + +During the Strategize phase, the strategy actor employs several mechanisms to compute precise dependency closures: + +1. **Resource-aware analysis**: The actor uses specialized skills to trace dependencies: + 

+   # Pseudocode of what happens inside a strategy actor
+   def compute_closure_for_refactoring(target_module):
+       closure = ResourceClosure()
+       
+       # Direct file dependencies
+       closure.add_files(find_imports(target_module))
+       closure.add_files(find_includes(target_module))
+       
+       # Symbol dependencies
+       for symbol in extract_exported_symbols(target_module):
+           closure.add_files(find_symbol_usage(symbol, scope='project'))
+       
+       # Test dependencies
+       closure.add_files(find_tests_for_module(target_module))
+       
+       # Build system dependencies
+       closure.add_files(find_build_references(target_module))
+       
+       return closure
+
+ +2. **Hierarchical scoping**: When spawning child plans (via `subplan_spawn` or `subplan_parallel_spawn`), each child plan receives: + - An explicit `relevant_resources` list + - A `sandbox_strategy` appropriate for those resources + - Clear boundaries of what it can and cannot modify + - The parent plan's effective invariant view (already reconciled from action, project, and global scopes) + +3. **Decision-based tracking**: Each `subplan_spawn` decision records the child plan it creates, and `subplan_parallel_spawn` decisions group parallel child plans: + 

+   # Individual child plan spawn
+   decision_type: subplan_spawn
+   chosen_option: "Refactor authentication module"
+   downstream_plan_ids: ["plan-auth-refactor-123"]
+   artifacts_produced: 
+     - auth_module_files: ["auth.rs", "auth_test.rs", "auth_types.rs"]
+     - api_updates: ["api/v2/login.rs", "api/v2/logout.rs"]
+   
+   # Parallel group of child plans
+   decision_type: subplan_parallel_spawn
+   chosen_option: "Convert utility libraries in parallel"
+   # Contains subplan_spawn children, each with their own downstream_plan_ids
+
+ +**Concrete example - Converting a subsystem to Rust**: + +Let's trace how the system handles "Convert Firefox's Network Stack to Rust": + +

+STRATEGIZE PHASE:
+1. Analyze network stack structure
+   - Identifies 847 C++ files in netwerk/ directory
+   - Traces public API surface (237 exported functions)
+   - Maps internal dependencies (1,432 internal calls)
+   
+2. Compute minimal closure for Phase 1 (DNS resolver):
+   - Core files: dns_resolver.cpp, dns_cache.cpp, dns_config.cpp (3 files)
+   - Direct dependencies: 12 files in netwerk/base/
+   - Test files: 8 test files specific to DNS
+   - Build files: 2 moz.build files
+   - Total closure: 25 files (not 847!)
+   
+3. Generate execution blueprint with child plans:
+   - [subplan_parallel_spawn] DNS module conversions:
+     - convert-dns-types: Closure of 5 files (type definitions)
+     - convert-dns-cache: Closure of 8 files (cache + tests)  
+     - convert-dns-resolver: Closure of 12 files (resolver + integration)
+
+ +**Why this is tractable even for massive codebases**: + +The system leverages several key insights about real software: + +1. **Modular boundaries exist**: Even in legacy codebases, there are natural boundaries +2. **Changes are incremental**: We don't convert 50,000 files atomically +3. **Dependencies are sparse**: Most modules depend on a small fraction of the codebase +4. **Interfaces are narrow**: Public APIs are much smaller than implementations + +The Firefox example would decompose into ~1,000 bounded child plans (grouped via `subplan_parallel_spawn` decisions where independent), each touching 10-100 files. The parent plan tracks the overall architecture and enforces invariants, while each child plan maintains its focused closure. + +**How we prevent closure explosion**: + +- **Lazy expansion**: Dependencies are traced only as deep as needed for correctness +- **Interface-based boundaries**: When possible, work against stable interfaces +- **Incremental validation**: Each child plan validates its changes don't break dependents +- **Hierarchical merge strategies**: Parent plans resolve conflicts between child plan changes + +### Q: What mechanisms enforce global consistency during parallel execution across many files? + +**What exists today architecturally**: The sandbox model combined with hierarchical plan execution provides strong guarantees about consistency during parallel execution. This isn't just process isolation - it's semantic isolation with intelligent merge strategies. + +**How the coordination mechanism prevents compound errors**: + +1. **Complete isolation during execution**: + Each plan executes in its own sandbox, which means: + 

+   Plan A (refactoring auth module):
+   - Sandbox A1: Contains only auth/*.cpp, auth_tests/*.cpp
+   - Cannot see Plan B's intermediate states
+   - Cannot accidentally depend on Plan B's half-done work
+   
+   Plan B (updating API endpoints):
+   - Sandbox B1: Contains only api/*.cpp, api_tests/*.cpp  
+   - Makes changes assuming current auth interface
+   - Protected from Plan A's intermediate refactoring
+
+ +2. **Resource-specific sandbox strategies provide natural coordination**: + 

+   Git repositories:
+     - Strategy: git worktrees
+     - Coordination: Git's three-way merge algorithm
+     - Conflict detection: Built into Git
+     - Rollback: git reset/checkout
+   
+   Databases:
+     - Strategy: Transaction isolation  
+     - Coordination: MVCC (multi-version concurrency control)
+     - Conflict detection: Serialization failures
+     - Rollback: Transaction abort
+   
+   Cloud Infrastructure:
+     - Strategy: Terraform workspaces
+     - Coordination: State locking
+     - Conflict detection: Resource conflicts in plan
+     - Rollback: Previous state restoration
+
+ +3. **Hierarchical merge resolution**: + When child plans complete, the parent plan performs intelligent merging: + 

+   def merge_subplan_results(subplan_results):
+       # Group by resource type
+       by_resource = group_by_resource_type(subplan_results)
+       
+       # Apply resource-specific merge strategies
+        for resource_type, changes in by_resource:
+            if resource_type == 'git-checkout':
+                merge_git_changes(changes)  # Three-way merge
+            elif resource_type == 'fs-mount':
+                merge_fs_changes(changes)   # Copy-on-write reconciliation
+            elif resource_type.startswith('database'):
+                merge_db_changes(changes)   # Sequential application
+       
+       # Validate merged state
+       run_integration_tests()
+
+ +**Concrete example - Preventing cascading failures**: + +Consider refactoring a shared authentication library used by 15 services: + +

+PARALLEL EXECUTION WITHOUT COORDINATION (what we prevent):
+- Service A refactors to async auth → breaks Service B
+- Service B compensates with workaround → breaks Service C  
+- Service C changes error handling → breaks Services D, E, F
+- Cascade of failures!
+
+CLEVERAGENTS COORDINATED EXECUTION:
+Parent Plan: Refactor auth library
+├── [invariant_enforced] "Auth library public API must remain backward compatible during transition"
+├── [subplan_spawn] Plan 1: Update auth library interface
+│   Sandbox: Only auth library files
+│   Output: New interface definition
+│   
+├── Barrier: Wait for Plan 1 completion
+│   
+├── [subplan_parallel_spawn] Plans 2-16: Update each service (in parallel)
+│   Each sandbox: Only that service's files
+│   Each uses: New interface from Plan 1
+│   No inter-service dependencies during execution
+│   
+└── Merge Phase:
+    - Collect all service updates
+    - Apply to main branch in order
+    - Run integration tests
+    - If conflicts: Parent plan resolves using semantic understanding
+
+ +**Advanced coordination patterns**: + +1. **Optimistic concurrency with semantic conflict resolution**: + 

+   Two child plans both modify api/user.rs:
+   - Plan A: Adds async fn get_user_profile()
+   - Plan B: Adds fn validate_user_permissions()
+   
+   Merge strategy:
+   - Git merge succeeds (different functions)
+   - Semantic validation ensures both functions work together
+   - Parent plan adds integration glue if needed
+
+ +2. **Checkpoint-based coordination**: + 

+   Execution timeline:
+   T1: Plan A creates checkpoint before major refactor
+   T2: Plan B creates checkpoint before API changes
+   T3: Plan A encounters error, rolls back to T1
+   T4: Plan B completes successfully
+   T5: Plan A retries with knowledge of B's success
+
+ +3. **Resource locking for critical sections**: + 

+   When modifying shared schema files:
+   - Acquire exclusive lock on schema resources
+   - Make changes atomically
+   - Release lock with new version
+   - Other plans rebase on new schema
+
+ +### Q: How does the system proactively prevent semantic errors before they propagate? + +**What exists today architecturally**: The specification defines multiple layers of proactive error prevention that go far beyond traditional testing. This is a comprehensive defense-in-depth approach that catches semantic errors before they can propagate. + +**Layer 1: Decision-time validation during Strategize**: + +Every decision includes semantic validation: +

+Decision: Refactor payment module to async
+alternatives_considered:
+  - "Convert to async/await patterns" (chosen)
+  - "Use thread pool with channels" (rejected: doesn't integrate with async ecosystem)
+  - "Keep synchronous with timeout" (rejected: doesn't solve core latency issue)
+confidence_score: 0.85
+validation_performed:
+  - Checked all payment API consumers can handle async
+  - Verified database driver supports async operations
+  - Confirmed no regulatory requirement for sync processing
+
+ +**Layer 2: Execution-time semantic guards**: + +The execution actor uses a tool node that references the independently registered `local/validate-api-compat` tool: + +

+actors:
+  code_executor:
+    type: graph
+    skills:
+      - local/semantic-validators    # Contains validation tools for LLM tool-calling
+    nodes:
+      - name: semantic_validator
+        type: tool
+        tool: local/validate-api-compat # Named tool from Tool Registry
+
+ +The `local/validate-api-compat` tool (independently registered via `agents tool add`) performs semantic validation — not just syntax checking. It extracts API signatures, finds breaking changes, checks affected consumers, and either auto-migrates or raises a `SemanticError` for manual review. + +**Layer 3: Invariant enforcement through the decision tree**: + +Invariants are attached at four scopes (global, project, action, plan) and managed via the unified `agents invariant` command or `--invariant` flags on creation commands. When a plan enters Strategize, the **Invariant Reconciliation Actor** (set via `--invariant-actor` on actions/plans/projects, or globally via `agents config set actor.default.invariant`) computes the effective invariant view by applying precedence rules (plan > project > global) to resolve conflicts. Each effective invariant is recorded as an `invariant_enforced` decision, making them visible, correctable, and auditable: + +

+# The system collects, reconciles, and enforces semantic invariants
+class InvariantEnforcer:
+    def compute_effective_invariants(self, plan):
+        """Compute the effective invariant view using the Invariant Reconciliation Actor."""
+        raw = self.collect_all_invariants(plan)
+        reconciler = (
+            self.get_plan_invariant_actor(plan)
+            or self.get_project_invariant_actor(plan)
+            or self.get_global_invariant_actor()
+        )
+        # Apply precedence: plan > project > global
+        return reconciler.reconcile(raw, precedence=['plan', 'project', 'global'])
+    
+    def collect_all_invariants(self, plan):
+        """Collect invariants from all scopes accessible to this plan."""
+        invariants = []
+        invariants.extend(self.get_global_invariants())
+        for project in plan.projects:
+            invariants.extend(self.get_project_invariants(project))
+        invariants.extend(self.get_action_invariants(plan.action))
+        invariants.extend(self.get_plan_invariants(plan))
+        return invariants
+    
+    # Example invariants at different scopes:
+    # Global:  "Payment processing must be idempotent"
+    # Project: "Database transactions must complete within 5 seconds"
+    # Action:  "Test files must not import production secrets"
+    # Plan:    "All API calls over TCP must be mocked"
+    
+    def check_invariant_preservation(self, changes, enforced_invariants):
+        for invariant in enforced_invariants:
+            if not self.verify_invariant(invariant, changes):
+                return InvariantViolation(invariant, changes)
+        return Success()
+
+ +**Layer 4: Predictive error prevention through pattern matching**: + +The system learns from past failures: +

+Error Pattern Database:
+  - pattern: "Async conversion in payment module"
+    historical_failures:
+      - "Race condition in payment confirmation"
+      - "Timeout handling breaks idempotency"
+    preventive_checks:
+      - "Add explicit transaction boundaries"
+      - "Verify idempotency keys are preserved"
+      - "Check distributed lock acquisition"
+
+ +**Concrete example - Preventing a subtle distributed systems bug**: + +Scenario: Refactoring a service to use event sourcing: + +

+PROACTIVE CONTAINMENT IN ACTION:
+
+1. Strategy Phase Semantic Analysis:
+   - Decision: "Convert order service to event sourcing"
+   - Semantic check: "Event sourcing requires eventual consistency"
+   - Identifies: 3 services assume immediate consistency
+   - Adds decision: "Update dependent services for eventual consistency"
+
+2. Execution Phase Invariant Checking:
+   - Detects: PaymentService.chargeCard() called after OrderCreated event
+   - Semantic issue: Payment before order confirmation violates business rules
+   - Automatic fix: Insert OrderConfirmed event requirement
+   
+3. Validation Node Catches Edge Case:
+   - Discovers: Audit service expects synchronous order numbers
+   - Impact: Async events break compliance reporting
+   - Resolution: Add audit event buffer with guaranteed ordering
+   
+4. Pre-Apply Semantic Verification:
+   - Simulates production event flow
+   - Detects: Under high load, events can arrive out of order
+   - Adds: Event ordering guarantees via vector clocks
+
+ +**Why this prevents issues that traditional testing misses**: + +Traditional tests check "does the code work?" Our semantic containment asks: +- Does it preserve business invariants? +- Does it maintain architectural patterns? +- Does it respect distributed systems principles? +- Does it handle the edge cases we've seen before? + +**Integration with Definition of Done (DoD)**: + +Each plan's DoD includes semantic requirements: +

+definition_of_done:
+  must:
+    - "All API changes maintain backward compatibility"
+    - "No increase in p99 latency"
+    - "Audit trail remains complete"
+  should:
+    - "Improve code coverage by 10%"
+    - "Reduce cyclomatic complexity"
+  may:
+    - "Optimize for memory usage"
+
+ +The validation nodes enforce these semantics, not just test passage. + +### Q: How does the system balance human supervision with autonomous operation? + +**What exists today architecturally**: The specification defines a comprehensive automation profile system with 10 confidence thresholds (0.0–1.0) and 3 boolean safety flags controlling every aspect of human-vs-automated operation. This isn't a binary human/AI split — it's a continuously adjustable matrix where each threshold specifies the minimum confidence level for automatic operation, tunable per task, per project, or per organization. + +**How the automation profiles work in practice**: + +

+manual Profile (all thresholds = 1.0):
+  - Every phase transition pauses for human action
+  - Every decision point pauses for human input
+  - User sees: Context, alternatives, recommendation, confidence score
+  - User provides: Explicit choice or custom guidance
+  - Use case: New users, critical production changes, first-time codebase exploration
+
+review Profile (phase thresholds = 0.0, decision thresholds = 1.0):
+  - Phases run automatically (Strategize, Execute proceed without pausing)
+  - Every decision within those phases pauses for human input
+  - Apply is manual; transient retries and child plans are automatic
+  - Use case: Teams that want to stay in the decision loop without triggering phases
+
+supervised Profile (strategize thresholds = 0.0, execute/apply = 1.0):
+  - Strategize runs automatically with autonomous decisions
+  - System pauses before Execute for human review of strategy
+  - Transient failures retried automatically
+  - Use case: Projects where you trust planning but want to review before execution
+
+cautious Profile (intermediate thresholds = 0.6-0.8):
+  - System proceeds automatically only when Semantic Escalation reports high confidence
+  - Escalates to human when uncertain (confidence below threshold)
+  - Higher bars (0.8) for risky operations, lower bars (0.6) for safer ones
+  - Use case: Gradual automation adoption, mixed-complexity projects
+
+trusted Profile (most thresholds = 0.0, apply = 1.0):
+  - Strategize and Execute run automatically with autonomous decisions
+  - Validation failures self-fixed within strategy bounds
+  - Child plans spawned automatically
+  - System pauses before Apply for human diff review
+  - Use case: Normal feature development, refactoring
+
+autonomous Profile (all thresholds = 0.0 except apply = 1.0):
+  - Everything automatic except Apply
+  - Can revise its own strategy if execution hits a wall
+  - Restores from checkpoints on failure
+  - Use case: Well-understood projects, batch operations
+
+ci Profile (all thresholds = 0.0, sandbox + checkpoints required):
+  - Complete end-to-end automation including Apply
+  - Sandbox and checkpoints remain mandatory for safety
+  - Unsafe tools blocked
+  - Use case: CI/CD pipelines, headless batch jobs, scheduled automation
+
+full-auto Profile (all thresholds = 0.0):
+  - Complete end-to-end automation including Apply
+  - No sandbox or checkpoint requirements
+  - Unsafe tools allowed
+  - Use case: Routine updates, environments with external safety mechanisms
+
+Custom profiles with intermediate thresholds (e.g., 0.3, 0.5, 0.7):
+  - Fine-grained control: auto-execute with confidence >= 0.5 but escalate below
+  - Different thresholds per flag: low bar for retries, high bar for strategy revision
+  - Enables "trust but verify" patterns without binary on/off
+  - Use case: Teams building confidence gradually, mixed-risk projects
+
+ +**The decision correction mechanism enables progressive automation**: + +The `agents plan correct` command is crucial for building trust: + +

+# User observes AI made suboptimal choice
+agents plan tree <plan_id>
+# Sees: [Decision] "Use REST API for service communication"
+
+# User knows gRPC would be better for this use case
+agents plan correct <decision_id> --mode=revert \
+  --guidance "Use gRPC instead of REST. This service requires streaming 
+             updates and binary protocol efficiency. Set up protocol 
+             buffer definitions and generate client/server stubs."
+
+# System:
+# 1. Marks original decision as superseded
+# 2. Creates new decision with user guidance
+# 3. Recomputes ONLY affected downstream decisions
+# 4. Preserves all unrelated work
+
+ +**Progressive trust building through automation profiles**: + +New users typically follow this progression: +1. Start with `manual` to understand system behavior +2. Move to `supervised` as confidence in the planning phase builds, or customize a profile with intermediate thresholds (e.g., 0.5) for gradual autonomy +3. Adopt `trusted` for routine development tasks +4. Enable `auto` for well-understood projects +5. Use `full-auto` for low-risk batch operations + +**Real autonomy through semantic understanding**: + +True autonomy isn't about removing humans - it's about the system understanding when it needs help: + +

+class AutonomyController:
+    def assess_decision_confidence(self, decision, context, profile):
+        factors = {
+            'past_success_rate': self.get_historical_success(decision.type),
+            'codebase_familiarity': self.get_familiarity_score(context.project),
+            'risk_assessment': self.evaluate_risk(decision),
+            'invariant_complexity': self.analyze_invariants(decision)
+        }
+        
+        confidence = self.compute_confidence(factors)  # Returns 0.0–1.0
+        threshold = profile.get_threshold(decision.flag)  # From profile
+        
+        if confidence >= threshold:
+            return ProceedAutonomously(decision, confidence)
+        else:
+            # Confidence below profile threshold — escalate to human
+            return RequestHumanGuidance(decision, confidence, threshold, factors)
+
+ +**Concrete example - Autonomous handling of a complex refactoring**: + +

+Scenario: "Modernize legacy e-commerce system"
+
+INITIAL PLAN (Full Automation Mode):
+1. System analyzes 50,000 line codebase
+2. Identifies modernization opportunities
+3. Creates plan with 47 child plans
+
+AUTONOMOUS EXECUTION WITH SMART ESCALATION:
+
+[subplan_parallel_spawn] Plans 1-15: Update utility functions (executes autonomously)
+- Confidence: 0.95 (straightforward transformations)
+- Result: Success
+
+[subplan_spawn] Plan 16: Refactor payment processing
+- Confidence: 0.4 (critical business logic)
+- [invariant_enforced] "Preserve exact penny rounding behavior"
+- Action: ESCALATES to human
+- Human provides: "Preserve exact penny rounding behavior"
+- Continues autonomously with constraint
+
+[subplan_parallel_spawn] Plans 17-30: UI component updates (executes autonomously)
+- Confidence: 0.9 (isolated changes)
+- Result: Success
+
+[subplan_spawn] Plan 31: Database schema migration
+- Detects: Would require 6-hour downtime
+- Action: ESCALATES to human
+- Human provides: "Use online migration with feature flags"
+- Re-plans with zero-downtime approach
+
+[subplan_parallel_spawn] Plans 32-47: Complete autonomously
+
+ +**The path to greater autonomy**: + +The system becomes more autonomous through: + +1. **Learning from corrections**: + - Every correction teaches the system about user preferences + - Patterns emerge: "This team always prefers gRPC for microservices" + - Future decisions incorporate these learnings + +2. **Building project-specific context**: + - Each successful plan adds to project knowledge + - System learns codebase patterns, team conventions, business rules + - Confidence increases with familiarity + +3. **Hierarchical delegation**: + - Proven child plan patterns become fully autonomous + - Human focuses on high-level decisions + - System handles implementation details + +4. **Semantic safety nets**: + - Comprehensive invariant checking reduces risk + - Rollback capabilities provide recovery path + - Humans can trust system won't cause catastrophic failures + +### Q: What's actually implemented today versus planned for the future? + +**Concrete implementations in the architecture**: + +1. **Decision Tree with Complete Context Capture** + - Full schema defined + - Storage model specified + - Correction mechanism detailed + - Query patterns established + +2. **Hierarchical Plan System** + - Spawning mechanism defined (sequential `subplan_spawn` and parallel `subplan_parallel_spawn`) + - Execution semantics specified (child plans are full Plans with their own decision trees) + - Invariant inheritance from parent plans, projects, and global scope + - Merge strategies documented + - Failure handling described + +3. **Resource-Aware Sandbox Isolation** + - Multiple strategies defined (git worktree, filesystem overlay, transactions) + - Lazy sandboxing for efficiency + - Resource-specific merge algorithms + - Cleanup behavior specified + +4. **Multi-Layer Error Prevention** + - Decision validation during planning + - Semantic validation nodes + - Invariant enforcement + - Definition of Done checking + +5. **Graduated Automation Controls** + - Three levels clearly defined + - Decision correction without full re-execution + - Confidence-based escalation + - Progressive trust building + +**Near-term implementations** (architecture complete, engineering straightforward): + +1. **Memory Tier Management** + - Hot/warm/cold distinction clear + - Context loading patterns defined + - Just needs LRU cache and storage backend + +2. **Cross-Plan Learning** + - Decision history provides training data + - Pattern extraction is standard ML + - Confidence scoring is well-understood + +3. **Cost/Risk Estimation** + - Dedicated estimation actor role defined + - Historical data provides baselines + - Standard prediction problem + +4. **Extended Validation Patterns** + - Pluggable validation architecture + - Project-specific rules as configuration + - Industry patterns can be packaged + +**Research territory** (requires innovation but architecture supports): + +1. **Optimal Context Selection for 100K+ file codebases** + - Current: Heuristic-based selection + - Research: ML-driven relevance ranking + - Architecture supports: Any selection algorithm can plug in + +2. **Automated Invariant Discovery** + - Current: User-defined invariants + - Research: Mining invariants from code patterns + - Architecture supports: Invariants are just validation rules + +3. **Cross-Project Knowledge Transfer** + - Current: Project-specific learning + - Research: Generalized pattern recognition + - Architecture supports: Cold tier can span projects + +4. **Fully Autonomous Recovery Strategies** + - Current: Rollback and retry with guidance + - Research: Automatic error understanding and fixing + - Architecture supports: Recovery is just another plan type + +**Why we can confidently handle Firefox-scale projects**: + +The architecture doesn't require magical AI breakthroughs. It requires: +- **Hierarchical decomposition**: ✓ Fully specified +- **Bounded context operations**: ✓ Dependency closure computation defined +- **Parallel execution with isolation**: ✓ Sandbox model complete +- **Semantic validation**: ✓ Multi-layer approach specified +- **Progressive automation**: ✓ Automation profiles and correction defined + +The difference between handling a 1,000 file project and a 100,000 file project is: +- More child plans (hierarchical decomposition handles this) +- Larger cold storage (standard database scaling) +- Better context selection (improves with use but works with heuristics) +- More validation patterns (accumulate over time) + +This isn't speculative architecture astronautics - it's applying proven distributed systems principles to AI agent coordination. The innovation is in the integration, not in requiring fundamental breakthroughs. + diff --git a/docs/specification.md b/docs/specification.md index deedb35b4..8919957cb 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -26442,7 +26442,7 @@ Retrieves context from parent and ancestor plan decisions. This is how child pla #### ACMS Extensibility -The ACMS-specific extension points (analyzers, backends, UKO vocabularies, strategies, and pipeline components) are documented in the **Extensibility** section below. See **Extensibility > ACMS-Specific Extensions** for the full details. +The ACMS-specific extension points (analyzers, backends, UKO vocabularies, strategies, and pipeline components) are documented in the **Extensibility** section below. See **Extensibility > ACMS Extensions** for the full details. #### Real-time Index Synchronization @@ -27400,7 +27400,7 @@ The sandbox layer supports custom isolation strategies for specialized resource Custom strategies are mapped to resource types via the resource type configuration's `sandbox_strategy` field, or globally via `sandbox.strategy` config. -#### ACMS-Specific Extensions +#### ACMS Extensions The ACMS provides four categories of extension points: UKO analyzers, index backends, context strategies, UKO vocabularies, and Context Assembly Pipeline components. @@ -27592,1187 +27592,3 @@ The following table shows which Protocol each pipeline slot implements and what | **Pipeline: PreambleGenerator** | `config.toml` `context.pipeline.preamble-generator` or project/plan YAML | TOML/YAML + Python module | Configuration-driven (scope chain) | | **Pipeline: SkeletonCompressor** | `config.toml` `context.pipeline.skeleton-compressor` or project/plan YAML | TOML/YAML + Python module | Configuration-driven (scope chain) | -## Work Remaining - -This section describes what remains to be done to bring the current CleverAgents codebase up to the specification. - -**Last Updated:** February 6, 2026 - -### Current State Assessment - -#### What's Implemented - -Based on analysis of the current codebase: - -1. **Plan Lifecycle Foundation** - The 3-phase lifecycle (Strategize → Execute → Apply), instantiated from Action templates, is partially implemented in `plan_lifecycle_service.py` -2. **Database Models** - Models exist for projects, plans, contexts, changes, and actors -3. **LangGraph Integration** - Graph-based workflow support exists but is not fully connected to the plan workflow -4. **Reactive System** - A reactive system with stream routing is present -5. **Actor System** - Actor models and services exist but lack full behavioral definitions -6. **Basic Context Analysis** - Simple context loading and analysis capabilities -7. **Change Tracking** - Basic `Change` and `ChangeSet` models exist - -#### What's Missing - -Critical gaps between the specification and current implementation: - -1. **No Resource Abstraction** - The unified resource layer for files, databases, APIs is entirely missing -2. **No Sandboxing** - Execute phase writes directly to files without isolation -3. **No Decision Tree** - No decision tracking, storage, or correction mechanism -4. **No Skills/MCP Integration** - No skill registry, skill YAML parsing, tool adapters, or MCP protocol support -5. **Single-File Limitation** - Hard-coded to generate exactly one file per plan -6. **Text-Based Code Generation** - Still parsing LLM output instead of tool-based approach -7. **No Checkpointing** - No rollback or checkpoint capabilities -8. **No Code Intelligence / ACMS** - Missing ACMS implementation: UKO ontology analyzers, Context Assembly Pipeline (the 10-component pluggable pipeline for strategy orchestration, fragment fusion, and context finalization), CRP skill, context budget calculator, and backend indices (text, vector, graph). The full design is specified in the "Advanced Context Management System (ACMS)" section. -9. **No Context Tiers** - No hot/warm/cold memory architecture, no warm-to-cold demotion, no temporal versioning of UKO nodes -10. **Limited Validation** - Basic stub validation instead of semantic checks - -### Work Items by Priority - -### 1) Implement Tool-Based Resource Modification (Critical Foundation) - -#### Problem -The current system generates code as a single text blob that gets written to one file. The specification requires a modern tool-based approach where LLMs invoke discrete operations on resources. - -#### Implementation Steps - -1. **Create Resource Abstraction Layer** - 

-   # New modules needed:
-   src/cleveragents/domain/models/resources.py
-   src/cleveragents/domain/resources/handlers.py
-   src/cleveragents/domain/resources/sandbox.py
-
- -2. **Implement Built-in Tools (via built-in skills)** - 

-   # Core file operations (provided by built-in skill groups)
-   read_file(path: str) -> str
-   write_file(path: str, content: str) -> None
-   edit_file(path: str, changes: list[Edit]) -> None
-   delete_file(path: str) -> None
-   move_file(src: str, dst: str) -> None
-   create_directory(path: str) -> None
-   list_files(pattern: str) -> list[str]
-   search_files(pattern: str, content_pattern: str) -> list[Match]
-
- -3. **Connect Tools to ChangeSet** - - Each tool invocation that modifies resources creates a `Change` record - - ChangeSet accumulates these changes during execution - - No more parsing LLM text output for code - -#### Estimated Effort: 2-3 weeks - -### 2) Implement Sandbox Infrastructure (Critical for Safety) - -#### Problem -Execute phase currently writes directly to the project. The specification requires all changes happen in an isolated sandbox that can be reviewed before applying. - -#### Implementation Steps - -1. **Define Sandbox Interface** - 

-   class Sandbox(Protocol):
-       def create() -> SandboxRef
-       def read(path: str) -> Content
-       def write(path: str, content: Content) -> Change
-       def diff() -> DiffView
-       def commit() -> None
-       def rollback() -> None
-
- -2. **Implement Sandbox Strategies** - - `GitWorktreeSandbox` - For git repositories (preferred) - - `FilesystemCopySandbox` - For non-git projects - - `TransactionSandbox` - For databases - - `NoOpSandbox` - For non-sandboxable resources - -3. **Lazy Sandbox Creation** - - Only create sandboxes when resources are accessed - - Each plan gets its own sandbox namespace - -#### Estimated Effort: 2 weeks - -### 3) Build Decision Tree System (Core Innovation) - -#### Problem -No decision tracking exists. The specification's key innovation is recording every decision with full context, enabling correction without full re-execution. - -#### Implementation Steps - -1. **Create Decision Models** - 

-   -- New tables needed
-   CREATE TABLE decisions (
-       decision_id TEXT PRIMARY KEY,  -- ULID
-       plan_id TEXT NOT NULL,
-       parent_decision_id TEXT,
-       decision_type TEXT NOT NULL,
-       question TEXT NOT NULL,
-       chosen_option TEXT NOT NULL,
-       alternatives_considered TEXT,  -- JSON array
-       context_snapshot TEXT NOT NULL,  -- JSON
-       created_at TEXT NOT NULL
-   );
-
- -2. **Implement Decision Recording** - - Every choice during Strategize creates a Decision record - - Capture complete context snapshot with each decision - - Track downstream dependencies - -3. **Build Correction Mechanism** - 

-   agents plan correct <decision_id> --mode=revert --guidance "<new decision>"
-
- - Mark decision as superseded - - Recompute only affected subtree - - Preserve unaffected work - -#### Estimated Effort: 3 weeks - -### 4) Implement Advanced Context Management System (ACMS) (Scalability Enabler) - -#### Problem -Current context is limited to a few hundred characters from a few files. Large codebases require intelligent, pluggable context discovery with multiple retrieval strategies, a universal knowledge representation, and demand-driven context assembly. The full ACMS design is specified in the "Advanced Context Management System (ACMS)" section of this document. - -#### Implementation Steps - -1. **Universal Knowledge Ontology (UKO) Foundation** - - Implement the four-layer UKO ontology hierarchy (Layer 0: universal foundation, Layer 1: domain specializations, Layer 2: paradigm/format specializations, Layer 3: technology-specific) - - Implement the DetailDepth / DetailLevelMap system with inheritance-based resolution (Layer 0 universal integer semantics, per-domain named level maps at Layer 1, paradigm refinements at Layer 2, language-specific insertions at Layer 3) - - Build source code analyzers for Python, TypeScript, Rust, Java (extract UKO nodes from source code via `uko-code:`, `uko-oo:`, `uko-func:`, `uko-py:`, etc.) - - Implement document analyzers for Markdown, reStructuredText, HTML (`uko-doc:`) with semantic awareness (topic extraction, implicit cross-reference inference via `uko-doc:discussesTopic`, citation resolution) - - Implement data schema analyzers for PostgreSQL, MySQL, SQLite (`uko-data:`) with schema introspection, foreign key graph construction, view dependency analysis, and stored procedure parsing - - Implement infrastructure analyzers for Docker Compose, Kubernetes (`uko-infra:`) with service dependency graph construction and config reference resolution - - Implement temporal versioning (`validFrom`, `validUntil`, `isCurrent`, `isRevisionOf`) - - Integrate eager indexing on resource add via the Resource Registry lifecycle hooks - -2. **Three-Index Backend Architecture** - - **Text Index**: Tantivy/SQLite FTS for exact keyword matches - - **Vector Index**: FAISS/Qdrant for semantic embedding search - - **Graph Store**: RDF store for UKO relationship traversal - - Implement `ScopedBackendView` to auto-filter all queries to plan project resources - -3. **Context Request Protocol (CRP)** - - Implement the `builtin/context` skill with tools: `request_context`, `query_history`, `get_context_budget` - - Implement `ContextRequest` and `ContextResponse` data structures - - Integrate CRP with actor execution loop - -4. **Context Assembly Pipeline (10-component pluggable pipeline)** - - Implement `ContextStrategy` base class and strategy registration - - Build six built-in strategies: `simple-keyword` (0.3), `semantic-embedding` (0.6), `breadth-depth-navigator` (0.85), `arce` (0.95), `temporal-archaeology` (0.5), `plan-decision-context` (0.7) - - **Phase 1 — Strategy Orchestration**: Implement `StrategySelectorProtocol` / `ConfidenceWeightedSelector`, `BudgetAllocatorProtocol` / `ProportionalBudgetAllocator`, `StrategyExecutorProtocol` / `ParallelStrategyExecutor` (with circuit breaker) - - **Phase 2 — Fragment Fusion**: Implement `FragmentDeduplicatorProtocol` / `ContentHashDeduplicator`, `DetailDepthResolverProtocol` / `MaxDepthResolver`, `FragmentScorerProtocol` / `WeightedCompositeScorer`, `BudgetPackerProtocol` / `GreedyKnapsackPacker`, `FragmentOrdererProtocol` / `RelevanceCoherenceOrderer` - - **Phase 3 — Context Finalization**: Implement `PreambleGeneratorProtocol` / `ProvenancePreambleGenerator`, `SkeletonCompressorProtocol` / `DepthReductionCompressor` - - Implement `ContextAssemblyPipeline` orchestrator and `PipelineConfigResolver` (scope chain: plan > project > global > built-in) - -6. **Context Budget Calculator & Refresh Manager** - - Dynamic budget computation from model context window, response reserve, tool definitions, system prompt - - Skeleton compression and plan context inheritance - - Automatic re-assembly on budget change exceeding refresh threshold - -7. **Tiered Context System** - - **Hot**: Current working set assembled by ACMS (in LLM context) - - **Warm**: Recent decisions and search results (warm retention window) - - **Cold**: Historical data, patterns, and archived UKO snapshots (queryable via retrieval) - -#### Estimated Effort: 8-10 weeks - -### 5) Complete Actor System with Behavioral Definitions - -#### Problem -Actors exist but don't define behavior beyond model selection. The specification requires actors to be composable graphs with tools, memory, and context policies. - -#### Implementation Steps - -1. **Extend Actor Configuration** - 

-   actors:
-     my_strategist:
-       type: graph
-       config:
-         provider: anthropic
-         model: claude-3-opus
-         memory_policy: per_plan
-         context_view: architect # High-level view
-       skills:
-         - local/file-ops             # provides read_file
-         - local/code-intelligence    # provides search_code, analyze_dependencies
-       routes:
-         strategize:
-           entry_point: analyze
-           nodes:
-             - name: analyze
-               type: llm
-             - name: plan
-               type: llm
-
- -2. **Implement Tool Access Policies** - - Strategy actors: read-only tools - - Execute actors: read/write within sandbox - - Apply actors: commit tools - -3. **Actor-Specific Context Views** - - Strategist: Architecture, dependencies, patterns - - Executor: Implementation details, specific files - - Reviewer: Diffs, tests, risk analysis - -#### Estimated Effort: 2 weeks - -### 6) Resource System, Resource Types, and Resource Registry - -#### Problem -Resources are currently defined inline within projects. The specification requires resources to be independently registered first-class entities with a type system, DAG relationships (physical/virtual), auto-discovery, and content-identity tracking. Tools need resource bindings to declare and resolve their resource dependencies. - -#### Implementation Steps - -1. **Resource Type Registry and CLI** - - Implement 24 built-in resource types: 15 physical (`git-checkout`, `git`, `git-remote`, `git-branch`, `git-tag`, `git-commit`, `git-tree`, `git-tree-entry`, `git-stash`, `git-submodule`, `fs-mount`, `fs-directory`, `fs-file`, `fs-symlink`, `fs-hardlink`) and 9 virtual (`file`, `directory`, `symlink`, `commit`, `branch`, `tag`, `remote`, `submodule`, `tree`) - - Implement resource type YAML parsing and validation for custom types - - Implement `agents resource type add [--update]/remove/list/show` commands - - Implement dynamic CLI subcommand registration (custom types create new `agents resource add` subcommands) - -2. **Resource Registry and CLI** - - Implement `agents resource add /remove/list/show/tree` commands - - Implement Resource Registry persistence in database - - Implement DAG parent/child relationships with type constraints and cycle detection - - Implement `agents resource link-child`/`unlink-child` for manual DAG management - -3. **Auto-Discovery System** - - Implement handler-driven child resource discovery (git-checkout discovers git + fs-directory worktree root; git discovers remotes, branches, tags, commits, stashes, submodules; git-commit discovers root git-tree; git-tree discovers tree entries + subtrees; fs-mount discovers root fs-directory; fs-directory discovers subdirectories, files, symlinks, hardlinks) - - Implement resource reuse during discovery (link existing resources instead of duplicating) - - Implement refresh mechanism for keeping discovered children up to date - -4. **Physical/Virtual Resource Model** - - Implement content hashing for identity tracking - - Implement virtual resource linking (shared virtual parents for identical physical resources) - - Implement divergence detection (unlink when content changes) - -5. **Project-Resource Linking** - - Replace `agents project add-resource`/`remove-resource` with `link-resource`/`unlink-resource` - - Implement project-level overrides (read-only, alias) - - Migrate any existing inline resource data to Resource Registry - -6. **Resource Handlers** - - Implement `GitCheckoutHandler` with auto-discovery, sandbox creation, checkpoint support - - Implement `GitHandler` for git repository structure discovery (remotes, branches, tags, commits, stashes, submodules) - - Implement `GitObjectHandler` for git-commit, git-tree, git-tree-entry (read-only access to git objects) - - Implement `GitRefHandler` for git-branch, git-tag, git-stash (read/write ref manipulation) - - Implement `GitConfigHandler` for git-remote, git-submodule (read-only config access) - - Implement `FilesystemHandler` with auto-discovery for fs-mount, fs-directory, fs-file, fs-symlink, fs-hardlink - - Implement handler plugin system for custom resource types - -#### Estimated Effort: 5 weeks - -### 7) MCP Integration, Tool System, Skill System, and Tool-Resource Bindings - -#### Problem -No tool or skill abstraction exists. The specification defines tools as independently registered operations managed via `agents tool` CLI commands, and skills as namespaced collections of tools managed via `agents skill` CLI commands. Tools can be sourced from MCP servers, Agent Skills folders, built-ins, and custom code. Tools also need resource bindings to declare and resolve their dependencies on resources. - -#### Implementation Steps - -1. **Tool Registry and CLI** - - Implement tool YAML parsing and validation (including `resources` section for resource slots) - - Implement `agents tool add [--update]/remove/list/show` commands - - Implement Tool Registry persistence in database - - Implement anonymous tool support (inline definitions without registration) - -2. **Skill Registry and CLI** - - Implement skill YAML parsing and validation - - Implement `agents skill add [--update]/remove/list/show/tools` commands - - Implement hierarchical skill composition (includes) with per-tool metadata overrides - - Implement Skill Registry persistence in database - - Implement named tool references (resolving from Tool Registry) and anonymous inline tools - -3. **Tool-Resource Binding System** - - Implement resource slot parsing and validation in tool YAML - - Implement three binding modes: contextual, static, parameter - - Implement binding resolution at activation time (contextual/static) and invocation time (parameter) - - Implement resource type compatibility validation - - Implement built-in tool implicit resource bindings - - Inject bound resources into `ToolExecutionContext.resources` - -4. **MCP Tool Adapter** - 

-   class MCPToolAdapter:
-       def wrap_mcp_tool(self, tool) -> Tool:
-           # Add sandbox interception
-           # Add change tracking
-           # Add capability metadata
-           # Add resource binding support
-
- -5. **Tool Capability Metadata** - 

-   class ToolCapability:
-       read_only: bool
-       write_scope: list[str]       # References resource slot names
-       checkpointable: bool
-       idempotent: bool
-       side_effects: list[str]
-
- -6. **External MCP Server Support** - 

-   # In tool or skill YAML configuration
-   mcp_servers:
-     - name: github
-       command: "npx @anthropic/mcp-github"
-       env: {GITHUB_TOKEN: "${GITHUB_TOKEN}"}
-
- -7. **Agent Skills Adapter** - - Implement SKILL.md frontmatter parsing for discovery - - Implement progressive disclosure (metadata → instructions → resources) - - Implement sandboxed script execution - -8. **Metadata Override System** - - Implement shallow-merge override logic for tool capability metadata - - Support overrides at skill-level (tool refs), include-level (tool_overrides), and actor graph node-level - - Built-in tool metadata is not overridable - -#### Estimated Effort: 5 weeks - -### 8) Implement Validation and Semantic Error Prevention - -#### Problem -Current validation is a stub. The specification requires multi-layer semantic validation. Validations are now a specialized subtype of Tool — they extend the Tool class with a `mode` (required/informational) and a structured JSON return format (`passed`, `message`, `data`). They can be attached to resources directly, through projects, or through plans. - -#### Implementation Steps - -1. **Decision-Time Validation** - - Validate choices during Strategize - - Check alternatives for feasibility - - Record validation in decision metadata - -2. **Execution-Time Guards** - 

-   # Actor references a skill containing validation tools
-   skills:
-     - local/semantic-validators   # validate_api_compatibility,
-                                   # check_invariants, verify_test_coverage
-
- -3. **Validation as Tool Subtype** - 

-   # Register validations (tool subtypes)
-   agents validation add --config ./validations/run-tests.yaml --required local/run-tests
-   agents validation add --config ./validations/lint-check.yaml --required local/lint-check
-
-   # Attach to resource (directly or scoped to a project)
-   agents validation attach --project my-api my-api-repo local/run-tests
-   agents validation attach my-api-repo local/lint-check
-
- -4. **Validation Class Inheritance** - - Implement Validation as a class extending Tool - - Reuse Tool Registry, binding resolution, and execution flow - - Add only: mode field, return format enforcement, attachment scoping - -5. **Tool Wrapping** - - Implement `wraps` field resolution: validate wrapped Tool exists at registration time, store reference - - Implement `transform` function: sandboxed Python execution that converts wrapped Tool output to `{ "passed", "message", "data" }` - - Implement property inheritance from wrapped Tool (`input_schema`, `resource_slots`, `timeout`) with override semantics - - Enforce read-only semantics on wrapped Tool invocation regardless of wrapped Tool's `writes` flag - - Emit registration-time warning when wrapping a Tool with `writes: true` - - Implement `argument_mapping` field: map Validation input arguments to wrapped Tool input arguments with support for renaming and fixed values - - (Deferred) Wrapped Tool deduplication: when multiple Validations wrap the same Tool with identical forwarded arguments, invoke it once and fan out output to each `transform`. Deferred due to complexity around argument equivalence and cache invalidation during fix-then-revalidate loops - - Implement default transform (pass-through for output already containing `passed` field) - -#### Estimated Effort: 2 weeks - -### 9) Connect LangGraph to Plan Lifecycle - -#### Problem -The reactive/LangGraph infrastructure exists but isn't connected to the main plan workflow. - -#### Implementation Steps - -1. **Create Unified Plan Graph** - 

-   class PlanLifecycleGraph:
-       def strategize_subgraph(self) -> StateGraph
-       def execute_subgraph(self) -> StateGraph
-       def apply_subgraph(self) -> StateGraph
-
- -2. **Wire Phase Transitions** - - `use` command triggers strategize graph - - `execute` command triggers execute graph - - `apply` command triggers apply graph - -3. **Remove Linear Pipeline** - - Replace tell/build/apply with graph execution - - Maintain backward compatibility at CLI level - -#### Estimated Effort: 1 week - -### Implementation Roadmap - -#### Phase 1: Foundation (8-10 weeks) -1. Tool-based resource modification (critical foundation) -2. Sandbox infrastructure -3. Decision tree system - -#### Phase 2: Core Systems (10-12 weeks) -4. Advanced Context Management System (ACMS) — UKO, CRP, strategies, fusion, budget -5. Actor system with behavioral definitions -6. Resource system, resource types, and resource registry -7. MCP integration, tool system, skill system, and tool-resource bindings - -#### Phase 3: Validation and Integration (3 weeks) -8. Validation and semantic error prevention -9. Connect LangGraph to plan lifecycle - -#### Phase 4: Production Hardening (4 weeks) -10. Checkpointing and rollback -11. Cost controls and rate limiting -12. Security fixes (remove eval, fix async) - -### Total Estimated Timeline: 6-7 months - -### Key Success Metrics - -1. **Multi-file generation**: Can generate a REST API with routes/, models/, tests/ -2. **Safe execution**: All changes happen in sandbox, reviewed before apply -3. **Decision correction**: Can correct a decision and recompute only affected work -4. **Scale to large codebases**: ACMS with UKO, pluggable strategies, and fusion can work with 10K+ file projects efficiently -5. **Semantic safety**: Catches breaking changes before they're applied - -### Migration Strategy - -The implementation can proceed incrementally: - -1. Start with resource abstraction (enables everything else) -2. Add sandboxing to existing execute phase -3. Gradually replace text parsing with tool invocations -4. Build decision tree alongside existing flow -5. Enhance context as indexing comes online - -This allows the system to remain functional during development while progressively adding the architectural improvements described in the specification. - -## CleverAgents Architecture FAQ - -### Q: How does CleverAgents handle persistent repository knowledge beyond ephemeral context windows? - -**What exists today architecturally**: The specification defines the Advanced Context Management System (ACMS) — a sophisticated, pluggable, strategy-driven framework that goes far beyond ephemeral context windows. At its core are three innovations: (1) the Universal Knowledge Ontology (UKO), an inheritance-based RDF ontology that represents any resource at multiple abstraction levels with full provenance and temporal versioning; (2) the Context Request Protocol (CRP), through which actors dynamically declare what information they need at what detail depth; and (3) a Context Assembly Pipeline — a 10-component pluggable pipeline that orchestrates strategy selection, budget allocation, parallel execution, fragment deduplication, depth resolution, scoring, budget packing, ordering, preamble generation, and skeleton compression. Each component is backed by a replaceable Protocol implementation configurable at global, project, or plan scope. Underlying this is the Decision Tree structure which provides a durable, queryable record of every choice made during planning. - -**How the persistent model works in practice**: - -When a strategy actor analyzes a codebase during the Strategize phase, the ACMS assembles context dynamically using the CRP. The actor issues `request_context` calls specifying focus nodes (UKO URIs), breadth (hop count), depth (an integer or named level like `SIGNATURES`, `FULL_SOURCE`), and optional depth gradients. The Context Assembly Pipeline then orchestrates the full assembly: the StrategySelector chooses applicable strategies (e.g., `breadth-depth-navigator` for graph traversal, `semantic-embedding` for similarity search, `plan-decision-context` for historical decisions), the BudgetAllocator distributes the token budget, the StrategyExecutor runs them in parallel, and the fusion phase (FragmentDeduplicator, DetailDepthResolver, FragmentScorer, BudgetPacker, FragmentOrderer) deduplicates, resolves detail conflicts, scores, and packs the results into the available budget. - -Each decision creates a comprehensive Decision record that includes: - -

-context_snapshot:
-  hot_context_hash: str # Cryptographic hash of the exact context
-  hot_context_ref: str # Pointer to the full stored snapshot
-  relevant_resources: list[ResourceRef] # Every file/symbol that influenced this decision
-  actor_state_ref: str # Complete LangGraph checkpoint
-  uko_focus_nodes: list[UKO_URI] # UKO nodes that were the focus of this context assembly
-  strategies_used: list[str] # Which ACMS strategies contributed to this context
-
- -This means when the system decides "refactor the authentication module to use async patterns," it permanently records: -- Which UKO nodes (files, classes, functions) were the focus of context assembly -- Which strategies retrieved context and at what detail depths -- What symbols and dependencies were traced via UKO graph traversal -- The exact code state that was analyzed (with temporal versioning via `validFrom`/`validUntil`) -- The reasoning chain that led to this choice -- Alternative approaches that were considered but rejected - -**The three-tier memory architecture enables scale**: - -1. **Hot tier**: Immediate working context assembled by the ACMS — the Context Assembly Pipeline's output loaded into the LLM context window, dynamically computed based on the actor's model context window minus response reserve and tool definitions -2. **Warm tier**: Recent decisions and their contexts from this plan tree — quickly accessible, retained for `context.tiers.warm.retention-hours` (default: 24h), queryable via the `plan-decision-context` strategy -3. **Cold tier**: Historical decisions from past plans on this codebase — queryable via `temporal-archaeology` strategy, retained for `context.tiers.cold.retention-days` (default: 90 days) - -When working on a 50,000 file codebase, the system doesn't need to hold all files in memory. Instead: -- Hot context focuses on the immediate task via demand-driven CRP requests (e.g., `request_context` with focus on 10-20 UKO nodes, breadth=2, depth_gradient={0:9, 1:4, 2:0}) -- Warm context maintains the decision chain via the `plan-decision-context` strategy, with skeleton compression propagating parent plan context to child plans -- Cold context provides historical patterns via the `temporal-archaeology` strategy ("last time we refactored auth, we also had to update these services") - -**Why this scales to massive codebases**: - -The key insight is that software development is inherently local - even in huge codebases, individual changes typically touch a bounded set of files. The Decision Tree captures these localities. When converting Firefox to Rust (your example), the system would: - -1. Make high-level architectural decisions and enforce invariants (captured as root decision nodes) -2. Decompose into major subsystem conversions (each a `subplan_parallel_spawn` or `subplan_spawn` decision spawning child plans) -3. Each subsystem plan makes decisions about its modules, inheriting applicable invariants -4. Module plans make decisions about individual files - -At each level, only the relevant context is loaded. The persistent decision graph means we can always reconstruct why we're converting a particular module and what constraints apply from higher-level decisions. - -**Concrete example of persistence in action**: - -

-Plan: Convert Firefox Renderer to Rust
-├── [invariant_enforced] "Maintain API compatibility with existing C++ callers"
-├── [invariant_enforced] "All converted modules must pass existing C++ test suites"
-├── [strategy_choice] Architecture approach: Start with leaf modules, work inward
-│   Context: Analyzed module dependency graph, 2,847 modules total
-│   Resources: module_graph.json, architecture_docs.md
-│   
-├── [subplan_parallel_spawn] Phase 1: Convert utility libraries (no external deps)
-│   └── [subplan_spawn] Convert string_utils module
-│       └── Plan: 01KH29R8WPKPBHRY7Q0NA9XW86
-│           ├── [prompt_definition] "Convert string_utils module to Rust"
-│           ├── [invariant_enforced] "Maintain API compatibility with existing C++ callers"
-│           ├── [implementation_choice] Use Rust's String type, not custom implementation
-│           │   Context: Analyzed 47 string_utils.cpp functions
-│           │   Resources: string_utils.cpp, string_utils.h, 12 dependent files
-│           │   Rationale: Rust's String provides same guarantees with better ergonomics
-│           └── ...
-
- -Even months later, we can query: "Why did we use Rust's String type?" and get the exact context and reasoning, without reprocessing the entire codebase. - -### Q: How does the system compute task-specific dependency closures for large-scale operations? - -**What exists today architecturally**: The specification defines multiple mechanisms for computing and maintaining minimal dependency closures. The execution blueprint produced during the Strategize phase doesn't just list steps - it includes a complete dependency graph with explicit scoping for each operation. - -**How dependency closure computation works**: - -During the Strategize phase, the strategy actor employs several mechanisms to compute precise dependency closures: - -1. **Resource-aware analysis**: The actor uses specialized skills to trace dependencies: - 

-   # Pseudocode of what happens inside a strategy actor
-   def compute_closure_for_refactoring(target_module):
-       closure = ResourceClosure()
-       
-       # Direct file dependencies
-       closure.add_files(find_imports(target_module))
-       closure.add_files(find_includes(target_module))
-       
-       # Symbol dependencies
-       for symbol in extract_exported_symbols(target_module):
-           closure.add_files(find_symbol_usage(symbol, scope='project'))
-       
-       # Test dependencies
-       closure.add_files(find_tests_for_module(target_module))
-       
-       # Build system dependencies
-       closure.add_files(find_build_references(target_module))
-       
-       return closure
-
- -2. **Hierarchical scoping**: When spawning child plans (via `subplan_spawn` or `subplan_parallel_spawn`), each child plan receives: - - An explicit `relevant_resources` list - - A `sandbox_strategy` appropriate for those resources - - Clear boundaries of what it can and cannot modify - - The parent plan's effective invariant view (already reconciled from action, project, and global scopes) - -3. **Decision-based tracking**: Each `subplan_spawn` decision records the child plan it creates, and `subplan_parallel_spawn` decisions group parallel child plans: - 

-   # Individual child plan spawn
-   decision_type: subplan_spawn
-   chosen_option: "Refactor authentication module"
-   downstream_plan_ids: ["plan-auth-refactor-123"]
-   artifacts_produced: 
-     - auth_module_files: ["auth.rs", "auth_test.rs", "auth_types.rs"]
-     - api_updates: ["api/v2/login.rs", "api/v2/logout.rs"]
-   
-   # Parallel group of child plans
-   decision_type: subplan_parallel_spawn
-   chosen_option: "Convert utility libraries in parallel"
-   # Contains subplan_spawn children, each with their own downstream_plan_ids
-
- -**Concrete example - Converting a subsystem to Rust**: - -Let's trace how the system handles "Convert Firefox's Network Stack to Rust": - -

-STRATEGIZE PHASE:
-1. Analyze network stack structure
-   - Identifies 847 C++ files in netwerk/ directory
-   - Traces public API surface (237 exported functions)
-   - Maps internal dependencies (1,432 internal calls)
-   
-2. Compute minimal closure for Phase 1 (DNS resolver):
-   - Core files: dns_resolver.cpp, dns_cache.cpp, dns_config.cpp (3 files)
-   - Direct dependencies: 12 files in netwerk/base/
-   - Test files: 8 test files specific to DNS
-   - Build files: 2 moz.build files
-   - Total closure: 25 files (not 847!)
-   
-3. Generate execution blueprint with child plans:
-   - [subplan_parallel_spawn] DNS module conversions:
-     - convert-dns-types: Closure of 5 files (type definitions)
-     - convert-dns-cache: Closure of 8 files (cache + tests)  
-     - convert-dns-resolver: Closure of 12 files (resolver + integration)
-
- -**Why this is tractable even for massive codebases**: - -The system leverages several key insights about real software: - -1. **Modular boundaries exist**: Even in legacy codebases, there are natural boundaries -2. **Changes are incremental**: We don't convert 50,000 files atomically -3. **Dependencies are sparse**: Most modules depend on a small fraction of the codebase -4. **Interfaces are narrow**: Public APIs are much smaller than implementations - -The Firefox example would decompose into ~1,000 bounded child plans (grouped via `subplan_parallel_spawn` decisions where independent), each touching 10-100 files. The parent plan tracks the overall architecture and enforces invariants, while each child plan maintains its focused closure. - -**How we prevent closure explosion**: - -- **Lazy expansion**: Dependencies are traced only as deep as needed for correctness -- **Interface-based boundaries**: When possible, work against stable interfaces -- **Incremental validation**: Each child plan validates its changes don't break dependents -- **Hierarchical merge strategies**: Parent plans resolve conflicts between child plan changes - -### Q: What mechanisms enforce global consistency during parallel execution across many files? - -**What exists today architecturally**: The sandbox model combined with hierarchical plan execution provides strong guarantees about consistency during parallel execution. This isn't just process isolation - it's semantic isolation with intelligent merge strategies. - -**How the coordination mechanism prevents compound errors**: - -1. **Complete isolation during execution**: - Each plan executes in its own sandbox, which means: - 

-   Plan A (refactoring auth module):
-   - Sandbox A1: Contains only auth/*.cpp, auth_tests/*.cpp
-   - Cannot see Plan B's intermediate states
-   - Cannot accidentally depend on Plan B's half-done work
-   
-   Plan B (updating API endpoints):
-   - Sandbox B1: Contains only api/*.cpp, api_tests/*.cpp  
-   - Makes changes assuming current auth interface
-   - Protected from Plan A's intermediate refactoring
-
- -2. **Resource-specific sandbox strategies provide natural coordination**: - 

-   Git repositories:
-     - Strategy: git worktrees
-     - Coordination: Git's three-way merge algorithm
-     - Conflict detection: Built into Git
-     - Rollback: git reset/checkout
-   
-   Databases:
-     - Strategy: Transaction isolation  
-     - Coordination: MVCC (multi-version concurrency control)
-     - Conflict detection: Serialization failures
-     - Rollback: Transaction abort
-   
-   Cloud Infrastructure:
-     - Strategy: Terraform workspaces
-     - Coordination: State locking
-     - Conflict detection: Resource conflicts in plan
-     - Rollback: Previous state restoration
-
- -3. **Hierarchical merge resolution**: - When child plans complete, the parent plan performs intelligent merging: - 

-   def merge_subplan_results(subplan_results):
-       # Group by resource type
-       by_resource = group_by_resource_type(subplan_results)
-       
-       # Apply resource-specific merge strategies
-        for resource_type, changes in by_resource:
-            if resource_type == 'git-checkout':
-                merge_git_changes(changes)  # Three-way merge
-            elif resource_type == 'fs-mount':
-                merge_fs_changes(changes)   # Copy-on-write reconciliation
-            elif resource_type.startswith('database'):
-                merge_db_changes(changes)   # Sequential application
-       
-       # Validate merged state
-       run_integration_tests()
-
- -**Concrete example - Preventing cascading failures**: - -Consider refactoring a shared authentication library used by 15 services: - -

-PARALLEL EXECUTION WITHOUT COORDINATION (what we prevent):
-- Service A refactors to async auth → breaks Service B
-- Service B compensates with workaround → breaks Service C  
-- Service C changes error handling → breaks Services D, E, F
-- Cascade of failures!
-
-CLEVERAGENTS COORDINATED EXECUTION:
-Parent Plan: Refactor auth library
-├── [invariant_enforced] "Auth library public API must remain backward compatible during transition"
-├── [subplan_spawn] Plan 1: Update auth library interface
-│   Sandbox: Only auth library files
-│   Output: New interface definition
-│   
-├── Barrier: Wait for Plan 1 completion
-│   
-├── [subplan_parallel_spawn] Plans 2-16: Update each service (in parallel)
-│   Each sandbox: Only that service's files
-│   Each uses: New interface from Plan 1
-│   No inter-service dependencies during execution
-│   
-└── Merge Phase:
-    - Collect all service updates
-    - Apply to main branch in order
-    - Run integration tests
-    - If conflicts: Parent plan resolves using semantic understanding
-
- -**Advanced coordination patterns**: - -1. **Optimistic concurrency with semantic conflict resolution**: - 

-   Two child plans both modify api/user.rs:
-   - Plan A: Adds async fn get_user_profile()
-   - Plan B: Adds fn validate_user_permissions()
-   
-   Merge strategy:
-   - Git merge succeeds (different functions)
-   - Semantic validation ensures both functions work together
-   - Parent plan adds integration glue if needed
-
- -2. **Checkpoint-based coordination**: - 

-   Execution timeline:
-   T1: Plan A creates checkpoint before major refactor
-   T2: Plan B creates checkpoint before API changes
-   T3: Plan A encounters error, rolls back to T1
-   T4: Plan B completes successfully
-   T5: Plan A retries with knowledge of B's success
-
- -3. **Resource locking for critical sections**: - 

-   When modifying shared schema files:
-   - Acquire exclusive lock on schema resources
-   - Make changes atomically
-   - Release lock with new version
-   - Other plans rebase on new schema
-
- -### Q: How does the system proactively prevent semantic errors before they propagate? - -**What exists today architecturally**: The specification defines multiple layers of proactive error prevention that go far beyond traditional testing. This is a comprehensive defense-in-depth approach that catches semantic errors before they can propagate. - -**Layer 1: Decision-time validation during Strategize**: - -Every decision includes semantic validation: -

-Decision: Refactor payment module to async
-alternatives_considered:
-  - "Convert to async/await patterns" (chosen)
-  - "Use thread pool with channels" (rejected: doesn't integrate with async ecosystem)
-  - "Keep synchronous with timeout" (rejected: doesn't solve core latency issue)
-confidence_score: 0.85
-validation_performed:
-  - Checked all payment API consumers can handle async
-  - Verified database driver supports async operations
-  - Confirmed no regulatory requirement for sync processing
-
- -**Layer 2: Execution-time semantic guards**: - -The execution actor uses a tool node that references the independently registered `local/validate-api-compat` tool: - -

-actors:
-  code_executor:
-    type: graph
-    skills:
-      - local/semantic-validators    # Contains validation tools for LLM tool-calling
-    nodes:
-      - name: semantic_validator
-        type: tool
-        tool: local/validate-api-compat # Named tool from Tool Registry
-
- -The `local/validate-api-compat` tool (independently registered via `agents tool add`) performs semantic validation — not just syntax checking. It extracts API signatures, finds breaking changes, checks affected consumers, and either auto-migrates or raises a `SemanticError` for manual review. - -**Layer 3: Invariant enforcement through the decision tree**: - -Invariants are attached at four scopes (global, project, action, plan) and managed via the unified `agents invariant` command or `--invariant` flags on creation commands. When a plan enters Strategize, the **Invariant Reconciliation Actor** (set via `--invariant-actor` on actions/plans/projects, or globally via `agents config set actor.default.invariant`) computes the effective invariant view by applying precedence rules (plan > project > global) to resolve conflicts. Each effective invariant is recorded as an `invariant_enforced` decision, making them visible, correctable, and auditable: - -

-# The system collects, reconciles, and enforces semantic invariants
-class InvariantEnforcer:
-    def compute_effective_invariants(self, plan):
-        """Compute the effective invariant view using the Invariant Reconciliation Actor."""
-        raw = self.collect_all_invariants(plan)
-        reconciler = (
-            self.get_plan_invariant_actor(plan)
-            or self.get_project_invariant_actor(plan)
-            or self.get_global_invariant_actor()
-        )
-        # Apply precedence: plan > project > global
-        return reconciler.reconcile(raw, precedence=['plan', 'project', 'global'])
-    
-    def collect_all_invariants(self, plan):
-        """Collect invariants from all scopes accessible to this plan."""
-        invariants = []
-        invariants.extend(self.get_global_invariants())
-        for project in plan.projects:
-            invariants.extend(self.get_project_invariants(project))
-        invariants.extend(self.get_action_invariants(plan.action))
-        invariants.extend(self.get_plan_invariants(plan))
-        return invariants
-    
-    # Example invariants at different scopes:
-    # Global:  "Payment processing must be idempotent"
-    # Project: "Database transactions must complete within 5 seconds"
-    # Action:  "Test files must not import production secrets"
-    # Plan:    "All API calls over TCP must be mocked"
-    
-    def check_invariant_preservation(self, changes, enforced_invariants):
-        for invariant in enforced_invariants:
-            if not self.verify_invariant(invariant, changes):
-                return InvariantViolation(invariant, changes)
-        return Success()
-
- -**Layer 4: Predictive error prevention through pattern matching**: - -The system learns from past failures: -

-Error Pattern Database:
-  - pattern: "Async conversion in payment module"
-    historical_failures:
-      - "Race condition in payment confirmation"
-      - "Timeout handling breaks idempotency"
-    preventive_checks:
-      - "Add explicit transaction boundaries"
-      - "Verify idempotency keys are preserved"
-      - "Check distributed lock acquisition"
-
- -**Concrete example - Preventing a subtle distributed systems bug**: - -Scenario: Refactoring a service to use event sourcing: - -

-PROACTIVE CONTAINMENT IN ACTION:
-
-1. Strategy Phase Semantic Analysis:
-   - Decision: "Convert order service to event sourcing"
-   - Semantic check: "Event sourcing requires eventual consistency"
-   - Identifies: 3 services assume immediate consistency
-   - Adds decision: "Update dependent services for eventual consistency"
-
-2. Execution Phase Invariant Checking:
-   - Detects: PaymentService.chargeCard() called after OrderCreated event
-   - Semantic issue: Payment before order confirmation violates business rules
-   - Automatic fix: Insert OrderConfirmed event requirement
-   
-3. Validation Node Catches Edge Case:
-   - Discovers: Audit service expects synchronous order numbers
-   - Impact: Async events break compliance reporting
-   - Resolution: Add audit event buffer with guaranteed ordering
-   
-4. Pre-Apply Semantic Verification:
-   - Simulates production event flow
-   - Detects: Under high load, events can arrive out of order
-   - Adds: Event ordering guarantees via vector clocks
-
- -**Why this prevents issues that traditional testing misses**: - -Traditional tests check "does the code work?" Our semantic containment asks: -- Does it preserve business invariants? -- Does it maintain architectural patterns? -- Does it respect distributed systems principles? -- Does it handle the edge cases we've seen before? - -**Integration with Definition of Done (DoD)**: - -Each plan's DoD includes semantic requirements: -

-definition_of_done:
-  must:
-    - "All API changes maintain backward compatibility"
-    - "No increase in p99 latency"
-    - "Audit trail remains complete"
-  should:
-    - "Improve code coverage by 10%"
-    - "Reduce cyclomatic complexity"
-  may:
-    - "Optimize for memory usage"
-
- -The validation nodes enforce these semantics, not just test passage. - -### Q: How does the system balance human supervision with autonomous operation? - -**What exists today architecturally**: The specification defines a comprehensive automation profile system with 10 confidence thresholds (0.0–1.0) and 3 boolean safety flags controlling every aspect of human-vs-automated operation. This isn't a binary human/AI split — it's a continuously adjustable matrix where each threshold specifies the minimum confidence level for automatic operation, tunable per task, per project, or per organization. - -**How the automation profiles work in practice**: - -

-manual Profile (all thresholds = 1.0):
-  - Every phase transition pauses for human action
-  - Every decision point pauses for human input
-  - User sees: Context, alternatives, recommendation, confidence score
-  - User provides: Explicit choice or custom guidance
-  - Use case: New users, critical production changes, first-time codebase exploration
-
-review Profile (phase thresholds = 0.0, decision thresholds = 1.0):
-  - Phases run automatically (Strategize, Execute proceed without pausing)
-  - Every decision within those phases pauses for human input
-  - Apply is manual; transient retries and child plans are automatic
-  - Use case: Teams that want to stay in the decision loop without triggering phases
-
-supervised Profile (strategize thresholds = 0.0, execute/apply = 1.0):
-  - Strategize runs automatically with autonomous decisions
-  - System pauses before Execute for human review of strategy
-  - Transient failures retried automatically
-  - Use case: Projects where you trust planning but want to review before execution
-
-cautious Profile (intermediate thresholds = 0.6-0.8):
-  - System proceeds automatically only when Semantic Escalation reports high confidence
-  - Escalates to human when uncertain (confidence below threshold)
-  - Higher bars (0.8) for risky operations, lower bars (0.6) for safer ones
-  - Use case: Gradual automation adoption, mixed-complexity projects
-
-trusted Profile (most thresholds = 0.0, apply = 1.0):
-  - Strategize and Execute run automatically with autonomous decisions
-  - Validation failures self-fixed within strategy bounds
-  - Child plans spawned automatically
-  - System pauses before Apply for human diff review
-  - Use case: Normal feature development, refactoring
-
-autonomous Profile (all thresholds = 0.0 except apply = 1.0):
-  - Everything automatic except Apply
-  - Can revise its own strategy if execution hits a wall
-  - Restores from checkpoints on failure
-  - Use case: Well-understood projects, batch operations
-
-ci Profile (all thresholds = 0.0, sandbox + checkpoints required):
-  - Complete end-to-end automation including Apply
-  - Sandbox and checkpoints remain mandatory for safety
-  - Unsafe tools blocked
-  - Use case: CI/CD pipelines, headless batch jobs, scheduled automation
-
-full-auto Profile (all thresholds = 0.0):
-  - Complete end-to-end automation including Apply
-  - No sandbox or checkpoint requirements
-  - Unsafe tools allowed
-  - Use case: Routine updates, environments with external safety mechanisms
-
-Custom profiles with intermediate thresholds (e.g., 0.3, 0.5, 0.7):
-  - Fine-grained control: auto-execute with confidence >= 0.5 but escalate below
-  - Different thresholds per flag: low bar for retries, high bar for strategy revision
-  - Enables "trust but verify" patterns without binary on/off
-  - Use case: Teams building confidence gradually, mixed-risk projects
-
- -**The decision correction mechanism enables progressive automation**: - -The `agents plan correct` command is crucial for building trust: - -

-# User observes AI made suboptimal choice
-agents plan tree <plan_id>
-# Sees: [Decision] "Use REST API for service communication"
-
-# User knows gRPC would be better for this use case
-agents plan correct <decision_id> --mode=revert \
-  --guidance "Use gRPC instead of REST. This service requires streaming 
-             updates and binary protocol efficiency. Set up protocol 
-             buffer definitions and generate client/server stubs."
-
-# System:
-# 1. Marks original decision as superseded
-# 2. Creates new decision with user guidance
-# 3. Recomputes ONLY affected downstream decisions
-# 4. Preserves all unrelated work
-
- -**Progressive trust building through automation profiles**: - -New users typically follow this progression: -1. Start with `manual` to understand system behavior -2. Move to `supervised` as confidence in the planning phase builds, or customize a profile with intermediate thresholds (e.g., 0.5) for gradual autonomy -3. Adopt `trusted` for routine development tasks -4. Enable `auto` for well-understood projects -5. Use `full-auto` for low-risk batch operations - -**Real autonomy through semantic understanding**: - -True autonomy isn't about removing humans - it's about the system understanding when it needs help: - -

-class AutonomyController:
-    def assess_decision_confidence(self, decision, context, profile):
-        factors = {
-            'past_success_rate': self.get_historical_success(decision.type),
-            'codebase_familiarity': self.get_familiarity_score(context.project),
-            'risk_assessment': self.evaluate_risk(decision),
-            'invariant_complexity': self.analyze_invariants(decision)
-        }
-        
-        confidence = self.compute_confidence(factors)  # Returns 0.0–1.0
-        threshold = profile.get_threshold(decision.flag)  # From profile
-        
-        if confidence >= threshold:
-            return ProceedAutonomously(decision, confidence)
-        else:
-            # Confidence below profile threshold — escalate to human
-            return RequestHumanGuidance(decision, confidence, threshold, factors)
-
- -**Concrete example - Autonomous handling of a complex refactoring**: - -

-Scenario: "Modernize legacy e-commerce system"
-
-INITIAL PLAN (Full Automation Mode):
-1. System analyzes 50,000 line codebase
-2. Identifies modernization opportunities
-3. Creates plan with 47 child plans
-
-AUTONOMOUS EXECUTION WITH SMART ESCALATION:
-
-[subplan_parallel_spawn] Plans 1-15: Update utility functions (executes autonomously)
-- Confidence: 0.95 (straightforward transformations)
-- Result: Success
-
-[subplan_spawn] Plan 16: Refactor payment processing
-- Confidence: 0.4 (critical business logic)
-- [invariant_enforced] "Preserve exact penny rounding behavior"
-- Action: ESCALATES to human
-- Human provides: "Preserve exact penny rounding behavior"
-- Continues autonomously with constraint
-
-[subplan_parallel_spawn] Plans 17-30: UI component updates (executes autonomously)
-- Confidence: 0.9 (isolated changes)
-- Result: Success
-
-[subplan_spawn] Plan 31: Database schema migration
-- Detects: Would require 6-hour downtime
-- Action: ESCALATES to human
-- Human provides: "Use online migration with feature flags"
-- Re-plans with zero-downtime approach
-
-[subplan_parallel_spawn] Plans 32-47: Complete autonomously
-
- -**The path to greater autonomy**: - -The system becomes more autonomous through: - -1. **Learning from corrections**: - - Every correction teaches the system about user preferences - - Patterns emerge: "This team always prefers gRPC for microservices" - - Future decisions incorporate these learnings - -2. **Building project-specific context**: - - Each successful plan adds to project knowledge - - System learns codebase patterns, team conventions, business rules - - Confidence increases with familiarity - -3. **Hierarchical delegation**: - - Proven child plan patterns become fully autonomous - - Human focuses on high-level decisions - - System handles implementation details - -4. **Semantic safety nets**: - - Comprehensive invariant checking reduces risk - - Rollback capabilities provide recovery path - - Humans can trust system won't cause catastrophic failures - -### Q: What's actually implemented today versus planned for the future? - -**Concrete implementations in the architecture**: - -1. **Decision Tree with Complete Context Capture** - - Full schema defined - - Storage model specified - - Correction mechanism detailed - - Query patterns established - -2. **Hierarchical Plan System** - - Spawning mechanism defined (sequential `subplan_spawn` and parallel `subplan_parallel_spawn`) - - Execution semantics specified (child plans are full Plans with their own decision trees) - - Invariant inheritance from parent plans, projects, and global scope - - Merge strategies documented - - Failure handling described - -3. **Resource-Aware Sandbox Isolation** - - Multiple strategies defined (git worktree, filesystem overlay, transactions) - - Lazy sandboxing for efficiency - - Resource-specific merge algorithms - - Cleanup behavior specified - -4. **Multi-Layer Error Prevention** - - Decision validation during planning - - Semantic validation nodes - - Invariant enforcement - - Definition of Done checking - -5. **Graduated Automation Controls** - - Three levels clearly defined - - Decision correction without full re-execution - - Confidence-based escalation - - Progressive trust building - -**Near-term implementations** (architecture complete, engineering straightforward): - -1. **Memory Tier Management** - - Hot/warm/cold distinction clear - - Context loading patterns defined - - Just needs LRU cache and storage backend - -2. **Cross-Plan Learning** - - Decision history provides training data - - Pattern extraction is standard ML - - Confidence scoring is well-understood - -3. **Cost/Risk Estimation** - - Dedicated estimation actor role defined - - Historical data provides baselines - - Standard prediction problem - -4. **Extended Validation Patterns** - - Pluggable validation architecture - - Project-specific rules as configuration - - Industry patterns can be packaged - -**Research territory** (requires innovation but architecture supports): - -1. **Optimal Context Selection for 100K+ file codebases** - - Current: Heuristic-based selection - - Research: ML-driven relevance ranking - - Architecture supports: Any selection algorithm can plug in - -2. **Automated Invariant Discovery** - - Current: User-defined invariants - - Research: Mining invariants from code patterns - - Architecture supports: Invariants are just validation rules - -3. **Cross-Project Knowledge Transfer** - - Current: Project-specific learning - - Research: Generalized pattern recognition - - Architecture supports: Cold tier can span projects - -4. **Fully Autonomous Recovery Strategies** - - Current: Rollback and retry with guidance - - Research: Automatic error understanding and fixing - - Architecture supports: Recovery is just another plan type - -**Why we can confidently handle Firefox-scale projects**: - -The architecture doesn't require magical AI breakthroughs. It requires: -- **Hierarchical decomposition**: ✓ Fully specified -- **Bounded context operations**: ✓ Dependency closure computation defined -- **Parallel execution with isolation**: ✓ Sandbox model complete -- **Semantic validation**: ✓ Multi-layer approach specified -- **Progressive automation**: ✓ Automation profiles and correction defined - -The difference between handling a 1,000 file project and a 100,000 file project is: -- More child plans (hierarchical decomposition handles this) -- Larger cold storage (standard database scaling) -- Better context selection (improves with use but works with heuristics) -- More validation patterns (accumulate over time) - -This isn't speculative architecture astronautics - it's applying proven distributed systems principles to AI agent coordination. The innovation is in the integration, not in requiring fundamental breakthroughs. - diff --git a/docs/work-remaining.md b/docs/work-remaining.md new file mode 100644 index 000000000..e666a63a5 --- /dev/null +++ b/docs/work-remaining.md @@ -0,0 +1,461 @@ +# Work Remaining + +This section describes what remains to be done to bring the current CleverAgents codebase up to the specification. + +**Last Updated:** February 6, 2026 + +### Current State Assessment + +#### What's Implemented + +Based on analysis of the current codebase: + +1. **Plan Lifecycle Foundation** - The 3-phase lifecycle (Strategize → Execute → Apply), instantiated from Action templates, is partially implemented in `plan_lifecycle_service.py` +2. **Database Models** - Models exist for projects, plans, contexts, changes, and actors +3. **LangGraph Integration** - Graph-based workflow support exists but is not fully connected to the plan workflow +4. **Reactive System** - A reactive system with stream routing is present +5. **Actor System** - Actor models and services exist but lack full behavioral definitions +6. **Basic Context Analysis** - Simple context loading and analysis capabilities +7. **Change Tracking** - Basic `Change` and `ChangeSet` models exist + +#### What's Missing + +Critical gaps between the specification and current implementation: + +1. **No Resource Abstraction** - The unified resource layer for files, databases, APIs is entirely missing +2. **No Sandboxing** - Execute phase writes directly to files without isolation +3. **No Decision Tree** - No decision tracking, storage, or correction mechanism +4. **No Skills/MCP Integration** - No skill registry, skill YAML parsing, tool adapters, or MCP protocol support +5. **Single-File Limitation** - Hard-coded to generate exactly one file per plan +6. **Text-Based Code Generation** - Still parsing LLM output instead of tool-based approach +7. **No Checkpointing** - No rollback or checkpoint capabilities +8. **No Code Intelligence / ACMS** - Missing ACMS implementation: UKO ontology analyzers, Context Assembly Pipeline (the 10-component pluggable pipeline for strategy orchestration, fragment fusion, and context finalization), CRP skill, context budget calculator, and backend indices (text, vector, graph). The full design is specified in the "Advanced Context Management System (ACMS)" section. +9. **No Context Tiers** - No hot/warm/cold memory architecture, no warm-to-cold demotion, no temporal versioning of UKO nodes +10. **Limited Validation** - Basic stub validation instead of semantic checks + +### Work Items by Priority + +### 1) Implement Tool-Based Resource Modification (Critical Foundation) + +#### Problem +The current system generates code as a single text blob that gets written to one file. The specification requires a modern tool-based approach where LLMs invoke discrete operations on resources. + +#### Implementation Steps + +1. **Create Resource Abstraction Layer** + 

+   # New modules needed:
+   src/cleveragents/domain/models/resources.py
+   src/cleveragents/domain/resources/handlers.py
+   src/cleveragents/domain/resources/sandbox.py
+
+ +2. **Implement Built-in Tools (via built-in skills)** + 

+   # Core file operations (provided by built-in skill groups)
+   read_file(path: str) -> str
+   write_file(path: str, content: str) -> None
+   edit_file(path: str, changes: list[Edit]) -> None
+   delete_file(path: str) -> None
+   move_file(src: str, dst: str) -> None
+   create_directory(path: str) -> None
+   list_files(pattern: str) -> list[str]
+   search_files(pattern: str, content_pattern: str) -> list[Match]
+
+ +3. **Connect Tools to ChangeSet** + - Each tool invocation that modifies resources creates a `Change` record + - ChangeSet accumulates these changes during execution + - No more parsing LLM text output for code + +#### Estimated Effort: 2-3 weeks + +### 2) Implement Sandbox Infrastructure (Critical for Safety) + +#### Problem +Execute phase currently writes directly to the project. The specification requires all changes happen in an isolated sandbox that can be reviewed before applying. + +#### Implementation Steps + +1. **Define Sandbox Interface** + 

+   class Sandbox(Protocol):
+       def create() -> SandboxRef
+       def read(path: str) -> Content
+       def write(path: str, content: Content) -> Change
+       def diff() -> DiffView
+       def commit() -> None
+       def rollback() -> None
+
+ +2. **Implement Sandbox Strategies** + - `GitWorktreeSandbox` - For git repositories (preferred) + - `FilesystemCopySandbox` - For non-git projects + - `TransactionSandbox` - For databases + - `NoOpSandbox` - For non-sandboxable resources + +3. **Lazy Sandbox Creation** + - Only create sandboxes when resources are accessed + - Each plan gets its own sandbox namespace + +#### Estimated Effort: 2 weeks + +### 3) Build Decision Tree System (Core Innovation) + +#### Problem +No decision tracking exists. The specification's key innovation is recording every decision with full context, enabling correction without full re-execution. + +#### Implementation Steps + +1. **Create Decision Models** + 

+   -- New tables needed
+   CREATE TABLE decisions (
+       decision_id TEXT PRIMARY KEY,  -- ULID
+       plan_id TEXT NOT NULL,
+       parent_decision_id TEXT,
+       decision_type TEXT NOT NULL,
+       question TEXT NOT NULL,
+       chosen_option TEXT NOT NULL,
+       alternatives_considered TEXT,  -- JSON array
+       context_snapshot TEXT NOT NULL,  -- JSON
+       created_at TEXT NOT NULL
+   );
+
+ +2. **Implement Decision Recording** + - Every choice during Strategize creates a Decision record + - Capture complete context snapshot with each decision + - Track downstream dependencies + +3. **Build Correction Mechanism** + 

+   agents plan correct <decision_id> --mode=revert --guidance "<new decision>"
+
+ - Mark decision as superseded + - Recompute only affected subtree + - Preserve unaffected work + +#### Estimated Effort: 3 weeks + +### 4) Implement Advanced Context Management System (ACMS) (Scalability Enabler) + +#### Problem +Current context is limited to a few hundred characters from a few files. Large codebases require intelligent, pluggable context discovery with multiple retrieval strategies, a universal knowledge representation, and demand-driven context assembly. The full ACMS design is specified in the "Advanced Context Management System (ACMS)" section of this document. + +#### Implementation Steps + +1. **Universal Knowledge Ontology (UKO) Foundation** + - Implement the four-layer UKO ontology hierarchy (Layer 0: universal foundation, Layer 1: domain specializations, Layer 2: paradigm/format specializations, Layer 3: technology-specific) + - Implement the DetailDepth / DetailLevelMap system with inheritance-based resolution (Layer 0 universal integer semantics, per-domain named level maps at Layer 1, paradigm refinements at Layer 2, language-specific insertions at Layer 3) + - Build source code analyzers for Python, TypeScript, Rust, Java (extract UKO nodes from source code via `uko-code:`, `uko-oo:`, `uko-func:`, `uko-py:`, etc.) + - Implement document analyzers for Markdown, reStructuredText, HTML (`uko-doc:`) with semantic awareness (topic extraction, implicit cross-reference inference via `uko-doc:discussesTopic`, citation resolution) + - Implement data schema analyzers for PostgreSQL, MySQL, SQLite (`uko-data:`) with schema introspection, foreign key graph construction, view dependency analysis, and stored procedure parsing + - Implement infrastructure analyzers for Docker Compose, Kubernetes (`uko-infra:`) with service dependency graph construction and config reference resolution + - Implement temporal versioning (`validFrom`, `validUntil`, `isCurrent`, `isRevisionOf`) + - Integrate eager indexing on resource add via the Resource Registry lifecycle hooks + +2. **Three-Index Backend Architecture** + - **Text Index**: Tantivy/SQLite FTS for exact keyword matches + - **Vector Index**: FAISS/Qdrant for semantic embedding search + - **Graph Store**: RDF store for UKO relationship traversal + - Implement `ScopedBackendView` to auto-filter all queries to plan project resources + +3. **Context Request Protocol (CRP)** + - Implement the `builtin/context` skill with tools: `request_context`, `query_history`, `get_context_budget` + - Implement `ContextRequest` and `ContextResponse` data structures + - Integrate CRP with actor execution loop + +4. **Context Assembly Pipeline (10-component pluggable pipeline)** + - Implement `ContextStrategy` base class and strategy registration + - Build six built-in strategies: `simple-keyword` (0.3), `semantic-embedding` (0.6), `breadth-depth-navigator` (0.85), `arce` (0.95), `temporal-archaeology` (0.5), `plan-decision-context` (0.7) + - **Phase 1 — Strategy Orchestration**: Implement `StrategySelectorProtocol` / `ConfidenceWeightedSelector`, `BudgetAllocatorProtocol` / `ProportionalBudgetAllocator`, `StrategyExecutorProtocol` / `ParallelStrategyExecutor` (with circuit breaker) + - **Phase 2 — Fragment Fusion**: Implement `FragmentDeduplicatorProtocol` / `ContentHashDeduplicator`, `DetailDepthResolverProtocol` / `MaxDepthResolver`, `FragmentScorerProtocol` / `WeightedCompositeScorer`, `BudgetPackerProtocol` / `GreedyKnapsackPacker`, `FragmentOrdererProtocol` / `RelevanceCoherenceOrderer` + - **Phase 3 — Context Finalization**: Implement `PreambleGeneratorProtocol` / `ProvenancePreambleGenerator`, `SkeletonCompressorProtocol` / `DepthReductionCompressor` + - Implement `ContextAssemblyPipeline` orchestrator and `PipelineConfigResolver` (scope chain: plan > project > global > built-in) + +6. **Context Budget Calculator & Refresh Manager** + - Dynamic budget computation from model context window, response reserve, tool definitions, system prompt + - Skeleton compression and plan context inheritance + - Automatic re-assembly on budget change exceeding refresh threshold + +7. **Tiered Context System** + - **Hot**: Current working set assembled by ACMS (in LLM context) + - **Warm**: Recent decisions and search results (warm retention window) + - **Cold**: Historical data, patterns, and archived UKO snapshots (queryable via retrieval) + +#### Estimated Effort: 8-10 weeks + +### 5) Complete Actor System with Behavioral Definitions + +#### Problem +Actors exist but don't define behavior beyond model selection. The specification requires actors to be composable graphs with tools, memory, and context policies. + +#### Implementation Steps + +1. **Extend Actor Configuration** + 

+   actors:
+     my_strategist:
+       type: graph
+       config:
+         provider: anthropic
+         model: claude-3-opus
+         memory_policy: per_plan
+         context_view: architect # High-level view
+       skills:
+         - local/file-ops             # provides read_file
+         - local/code-intelligence    # provides search_code, analyze_dependencies
+       routes:
+         strategize:
+           entry_point: analyze
+           nodes:
+             - name: analyze
+               type: llm
+             - name: plan
+               type: llm
+
+ +2. **Implement Tool Access Policies** + - Strategy actors: read-only tools + - Execute actors: read/write within sandbox + - Apply actors: commit tools + +3. **Actor-Specific Context Views** + - Strategist: Architecture, dependencies, patterns + - Executor: Implementation details, specific files + - Reviewer: Diffs, tests, risk analysis + +#### Estimated Effort: 2 weeks + +### 6) Resource System, Resource Types, and Resource Registry + +#### Problem +Resources are currently defined inline within projects. The specification requires resources to be independently registered first-class entities with a type system, DAG relationships (physical/virtual), auto-discovery, and content-identity tracking. Tools need resource bindings to declare and resolve their resource dependencies. + +#### Implementation Steps + +1. **Resource Type Registry and CLI** + - Implement 24 built-in resource types: 15 physical (`git-checkout`, `git`, `git-remote`, `git-branch`, `git-tag`, `git-commit`, `git-tree`, `git-tree-entry`, `git-stash`, `git-submodule`, `fs-mount`, `fs-directory`, `fs-file`, `fs-symlink`, `fs-hardlink`) and 9 virtual (`file`, `directory`, `symlink`, `commit`, `branch`, `tag`, `remote`, `submodule`, `tree`) + - Implement resource type YAML parsing and validation for custom types + - Implement `agents resource type add [--update]/remove/list/show` commands + - Implement dynamic CLI subcommand registration (custom types create new `agents resource add` subcommands) + +2. **Resource Registry and CLI** + - Implement `agents resource add /remove/list/show/tree` commands + - Implement Resource Registry persistence in database + - Implement DAG parent/child relationships with type constraints and cycle detection + - Implement `agents resource link-child`/`unlink-child` for manual DAG management + +3. **Auto-Discovery System** + - Implement handler-driven child resource discovery (git-checkout discovers git + fs-directory worktree root; git discovers remotes, branches, tags, commits, stashes, submodules; git-commit discovers root git-tree; git-tree discovers tree entries + subtrees; fs-mount discovers root fs-directory; fs-directory discovers subdirectories, files, symlinks, hardlinks) + - Implement resource reuse during discovery (link existing resources instead of duplicating) + - Implement refresh mechanism for keeping discovered children up to date + +4. **Physical/Virtual Resource Model** + - Implement content hashing for identity tracking + - Implement virtual resource linking (shared virtual parents for identical physical resources) + - Implement divergence detection (unlink when content changes) + +5. **Project-Resource Linking** + - Replace `agents project add-resource`/`remove-resource` with `link-resource`/`unlink-resource` + - Implement project-level overrides (read-only, alias) + - Migrate any existing inline resource data to Resource Registry + +6. **Resource Handlers** + - Implement `GitCheckoutHandler` with auto-discovery, sandbox creation, checkpoint support + - Implement `GitHandler` for git repository structure discovery (remotes, branches, tags, commits, stashes, submodules) + - Implement `GitObjectHandler` for git-commit, git-tree, git-tree-entry (read-only access to git objects) + - Implement `GitRefHandler` for git-branch, git-tag, git-stash (read/write ref manipulation) + - Implement `GitConfigHandler` for git-remote, git-submodule (read-only config access) + - Implement `FilesystemHandler` with auto-discovery for fs-mount, fs-directory, fs-file, fs-symlink, fs-hardlink + - Implement handler plugin system for custom resource types + +#### Estimated Effort: 5 weeks + +### 7) MCP Integration, Tool System, Skill System, and Tool-Resource Bindings + +#### Problem +No tool or skill abstraction exists. The specification defines tools as independently registered operations managed via `agents tool` CLI commands, and skills as namespaced collections of tools managed via `agents skill` CLI commands. Tools can be sourced from MCP servers, Agent Skills folders, built-ins, and custom code. Tools also need resource bindings to declare and resolve their dependencies on resources. + +#### Implementation Steps + +1. **Tool Registry and CLI** + - Implement tool YAML parsing and validation (including `resources` section for resource slots) + - Implement `agents tool add [--update]/remove/list/show` commands + - Implement Tool Registry persistence in database + - Implement anonymous tool support (inline definitions without registration) + +2. **Skill Registry and CLI** + - Implement skill YAML parsing and validation + - Implement `agents skill add [--update]/remove/list/show/tools` commands + - Implement hierarchical skill composition (includes) with per-tool metadata overrides + - Implement Skill Registry persistence in database + - Implement named tool references (resolving from Tool Registry) and anonymous inline tools + +3. **Tool-Resource Binding System** + - Implement resource slot parsing and validation in tool YAML + - Implement three binding modes: contextual, static, parameter + - Implement binding resolution at activation time (contextual/static) and invocation time (parameter) + - Implement resource type compatibility validation + - Implement built-in tool implicit resource bindings + - Inject bound resources into `ToolExecutionContext.resources` + +4. **MCP Tool Adapter** + 

+   class MCPToolAdapter:
+       def wrap_mcp_tool(self, tool) -> Tool:
+           # Add sandbox interception
+           # Add change tracking
+           # Add capability metadata
+           # Add resource binding support
+
+ +5. **Tool Capability Metadata** + 

+   class ToolCapability:
+       read_only: bool
+       write_scope: list[str]       # References resource slot names
+       checkpointable: bool
+       idempotent: bool
+       side_effects: list[str]
+
+ +6. **External MCP Server Support** + 

+   # In tool or skill YAML configuration
+   mcp_servers:
+     - name: github
+       command: "npx @anthropic/mcp-github"
+       env: {GITHUB_TOKEN: "${GITHUB_TOKEN}"}
+
+ +7. **Agent Skills Adapter** + - Implement SKILL.md frontmatter parsing for discovery + - Implement progressive disclosure (metadata → instructions → resources) + - Implement sandboxed script execution + +8. **Metadata Override System** + - Implement shallow-merge override logic for tool capability metadata + - Support overrides at skill-level (tool refs), include-level (tool_overrides), and actor graph node-level + - Built-in tool metadata is not overridable + +#### Estimated Effort: 5 weeks + +### 8) Implement Validation and Semantic Error Prevention + +#### Problem +Current validation is a stub. The specification requires multi-layer semantic validation. Validations are now a specialized subtype of Tool — they extend the Tool class with a `mode` (required/informational) and a structured JSON return format (`passed`, `message`, `data`). They can be attached to resources directly, through projects, or through plans. + +#### Implementation Steps + +1. **Decision-Time Validation** + - Validate choices during Strategize + - Check alternatives for feasibility + - Record validation in decision metadata + +2. **Execution-Time Guards** + 

+   # Actor references a skill containing validation tools
+   skills:
+     - local/semantic-validators   # validate_api_compatibility,
+                                   # check_invariants, verify_test_coverage
+
+ +3. **Validation as Tool Subtype** + 

+   # Register validations (tool subtypes)
+   agents validation add --config ./validations/run-tests.yaml --required local/run-tests
+   agents validation add --config ./validations/lint-check.yaml --required local/lint-check
+
+   # Attach to resource (directly or scoped to a project)
+   agents validation attach --project my-api my-api-repo local/run-tests
+   agents validation attach my-api-repo local/lint-check
+
+ +4. **Validation Class Inheritance** + - Implement Validation as a class extending Tool + - Reuse Tool Registry, binding resolution, and execution flow + - Add only: mode field, return format enforcement, attachment scoping + +5. **Tool Wrapping** + - Implement `wraps` field resolution: validate wrapped Tool exists at registration time, store reference + - Implement `transform` function: sandboxed Python execution that converts wrapped Tool output to `{ "passed", "message", "data" }` + - Implement property inheritance from wrapped Tool (`input_schema`, `resource_slots`, `timeout`) with override semantics + - Enforce read-only semantics on wrapped Tool invocation regardless of wrapped Tool's `writes` flag + - Emit registration-time warning when wrapping a Tool with `writes: true` + - Implement `argument_mapping` field: map Validation input arguments to wrapped Tool input arguments with support for renaming and fixed values + - (Deferred) Wrapped Tool deduplication: when multiple Validations wrap the same Tool with identical forwarded arguments, invoke it once and fan out output to each `transform`. Deferred due to complexity around argument equivalence and cache invalidation during fix-then-revalidate loops + - Implement default transform (pass-through for output already containing `passed` field) + +#### Estimated Effort: 2 weeks + +### 9) Connect LangGraph to Plan Lifecycle + +#### Problem +The reactive/LangGraph infrastructure exists but isn't connected to the main plan workflow. + +#### Implementation Steps + +1. **Create Unified Plan Graph** + 

+   class PlanLifecycleGraph:
+       def strategize_subgraph(self) -> StateGraph
+       def execute_subgraph(self) -> StateGraph
+       def apply_subgraph(self) -> StateGraph
+
+ +2. **Wire Phase Transitions** + - `use` command triggers strategize graph + - `execute` command triggers execute graph + - `apply` command triggers apply graph + +3. **Remove Linear Pipeline** + - Replace tell/build/apply with graph execution + - Maintain backward compatibility at CLI level + +#### Estimated Effort: 1 week + +### Implementation Roadmap + +#### Phase 1: Foundation (8-10 weeks) +1. Tool-based resource modification (critical foundation) +2. Sandbox infrastructure +3. Decision tree system + +#### Phase 2: Core Systems (10-12 weeks) +4. Advanced Context Management System (ACMS) — UKO, CRP, strategies, fusion, budget +5. Actor system with behavioral definitions +6. Resource system, resource types, and resource registry +7. MCP integration, tool system, skill system, and tool-resource bindings + +#### Phase 3: Validation and Integration (3 weeks) +8. Validation and semantic error prevention +9. Connect LangGraph to plan lifecycle + +#### Phase 4: Production Hardening (4 weeks) +10. Checkpointing and rollback +11. Cost controls and rate limiting +12. Security fixes (remove eval, fix async) + +### Total Estimated Timeline: 6-7 months + +### Key Success Metrics + +1. **Multi-file generation**: Can generate a REST API with routes/, models/, tests/ +2. **Safe execution**: All changes happen in sandbox, reviewed before apply +3. **Decision correction**: Can correct a decision and recompute only affected work +4. **Scale to large codebases**: ACMS with UKO, pluggable strategies, and fusion can work with 10K+ file projects efficiently +5. **Semantic safety**: Catches breaking changes before they're applied + +### Migration Strategy + +The implementation can proceed incrementally: + +1. Start with resource abstraction (enables everything else) +2. Add sandboxing to existing execute phase +3. Gradually replace text parsing with tool invocations +4. Build decision tree alongside existing flow +5. Enhance context as indexing comes online + +This allows the system to remain functional during development while progressively adding the architectural improvements described in the specification. diff --git a/mkdocs.yml b/mkdocs.yml index 268d1da78..f531b263f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,6 +10,8 @@ site_dir: build/site nav: - Specification: specification.md + - Work Remaining: work-remaining.md + - FAQ: faq.md - Development: - CI/CD Pipeline: development/ci-cd.md - Quality Automation: development/quality-automation.md