cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity
Files
master
cleveragents-core/docs/faq.md
T
HAL9000 f808abff86 chore(ci): fix pre-commit hook failures 
Fix JSON syntax errors in .devcontainer/devcontainer.json (removed
invalid JS-style // comments) and .devcontainer/opencode.json (removed
90+ trailing commas). Apply auto-fixes for end-of-file and trailing
whitespace issues across 100+ files. Fix SIM105 ruff violations in
benchmarks/core_circuit_breaker_bench.py (use contextlib.suppress).

Note: The security fix from issue #7478 (validate_path startswith bypass)
was already delivered to master in commit e18ac5f2. This PR as currently
structured is non-atomic (35 commits across 10+ issues) and needs
significant restructure before merge. This commit only addresses the
CI/pre-commit failures.

ISSUES CLOSED: #7478
2026-06-14 09:51:14 -04:00

723 lines
42 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CleverAgents Architecture FAQ
### Q: How does CleverAgents handle persistent repository knowledge beyond ephemeral context windows?
**What exists today architecturally**: The specification defines the Advanced Context Management System (ACMS) — a sophisticated, pluggable, strategy-driven framework that goes far beyond ephemeral context windows. At its core are three innovations: (1) the Universal Knowledge Ontology (UKO), an inheritance-based RDF ontology that represents any resource at multiple abstraction levels with full provenance and temporal versioning; (2) the Context Request Protocol (CRP), through which actors dynamically declare what information they need at what detail depth; and (3) a Context Assembly Pipeline — a 10-component pluggable pipeline that orchestrates strategy selection, budget allocation, parallel execution, fragment deduplication, depth resolution, scoring, budget packing, ordering, preamble generation, and skeleton compression. Each component is backed by a replaceable Protocol implementation configurable at global, project, or plan scope. Underlying this is the Decision Tree structure which provides a durable, queryable record of every choice made during planning.
**How the persistent model works in practice**:
When a strategy actor analyzes a codebase during the Strategize phase, the ACMS assembles context dynamically using the CRP. The actor issues `request_context` calls specifying focus nodes (UKO URIs), breadth (hop count), depth (an integer or named level like `SIGNATURES`, `FULL_SOURCE`), and optional depth gradients. The Context Assembly Pipeline then orchestrates the full assembly: the StrategySelector chooses applicable strategies (e.g., `breadth-depth-navigator` for graph traversal, `semantic-embedding` for similarity search, `plan-decision-context` for historical decisions), the BudgetAllocator distributes the token budget, the StrategyExecutor runs them in parallel, and the fusion phase (FragmentDeduplicator, DetailDepthResolver, FragmentScorer, BudgetPacker, FragmentOrderer) deduplicates, resolves detail conflicts, scores, and packs the results into the available budget.
Each decision creates a comprehensive Decision record that includes:
<div class="highlight"><pre><code>
<span style="color: cyan; font-weight: 600;">context_snapshot</span>:
<span style="color: cyan; font-weight: 600;">hot_context_hash</span>: str <span style="opacity: 0.7;"># Cryptographic hash of the exact context</span>
<span style="color: cyan; font-weight: 600;">hot_context_ref</span>: str <span style="opacity: 0.7;"># Pointer to the full stored snapshot</span>
<span style="color: cyan; font-weight: 600;">relevant_resources</span>: list[ResourceRef] <span style="opacity: 0.7;"># Every file/symbol that influenced this decision</span>
<span style="color: cyan; font-weight: 600;">actor_state_ref</span>: str <span style="opacity: 0.7;"># Complete LangGraph checkpoint</span>
<span style="color: cyan; font-weight: 600;">uko_focus_nodes</span>: list[UKO_URI] <span style="opacity: 0.7;"># UKO nodes that were the focus of this context assembly</span>
<span style="color: cyan; font-weight: 600;">strategies_used</span>: list[str] <span style="opacity: 0.7;"># Which ACMS strategies contributed to this context</span>
</code></pre></div>
This means when the system decides "refactor the authentication module to use async patterns," it permanently records:
- Which UKO nodes (files, classes, functions) were the focus of context assembly
- Which strategies retrieved context and at what detail depths
- What symbols and dependencies were traced via UKO graph traversal
- The exact code state that was analyzed (with temporal versioning via `validFrom`/`validUntil`)
- The reasoning chain that led to this choice
- Alternative approaches that were considered but rejected
**The three-tier memory architecture enables scale**:
1. **Hot tier**: Immediate working context assembled by the ACMS — the Context Assembly Pipeline's output loaded into the LLM context window, dynamically computed based on the actor's model context window minus response reserve and tool definitions
2. **Warm tier**: Recent decisions and their contexts from this plan tree — quickly accessible, retained for `context.tiers.warm.retention-hours` (default: 24h), queryable via the `plan-decision-context` strategy
3. **Cold tier**: Historical decisions from past plans on this codebase — queryable via `temporal-archaeology` strategy, retained for `context.tiers.cold.retention-days` (default: 90 days)
When working on a 50,000 file codebase, the system doesn't need to hold all files in memory. Instead:
- Hot context focuses on the immediate task via demand-driven CRP requests (e.g., `request_context` with focus on 10-20 UKO nodes, breadth=2, depth_gradient={0:9, 1:4, 2:0})
- Warm context maintains the decision chain via the `plan-decision-context` strategy, with skeleton compression propagating parent plan context to child plans
- Cold context provides historical patterns via the `temporal-archaeology` strategy ("last time we refactored auth, we also had to update these services")
**Why this scales to massive codebases**:
The key insight is that software development is inherently local - even in huge codebases, individual changes typically touch a bounded set of files. The Decision Tree captures these localities. When converting Firefox to Rust (your example), the system would:
1. Make high-level architectural decisions and enforce invariants (captured as root decision nodes)
2. Decompose into major subsystem conversions (each a `subplan_parallel_spawn` or `subplan_spawn` decision spawning child plans)
3. Each subsystem plan makes decisions about its modules, inheriting applicable invariants
4. Module plans make decisions about individual files
At each level, only the relevant context is loaded. The persistent decision graph means we can always reconstruct why we're converting a particular module and what constraints apply from higher-level decisions.
**Concrete example of persistence in action**:
<div class="highlight"><pre><code>
Plan: Convert Firefox Renderer to Rust
├── [<span style="color: magenta;">invariant_enforced</span>] &quot;Maintain API compatibility with existing C++ callers&quot;
├── [<span style="color: magenta;">invariant_enforced</span>] &quot;All converted modules must pass existing C++ test suites&quot;
├── [<span style="color: magenta;">strategy_choice</span>] Architecture approach: Start with leaf modules, work inward
│ Context: Analyzed module dependency graph, 2,847 modules total
│ Resources: module_graph.json, architecture_docs.md
│
├── [<span style="color: magenta;">subplan_parallel_spawn</span>] Phase 1: Convert utility libraries (no external deps)
│ └── [<span style="color: magenta;">subplan_spawn</span>] Convert string_utils module
│ └── Plan: 01KH29R8WPKPBHRY7Q0NA9XW86
│ ├── [<span style="color: magenta;">prompt_definition</span>] &quot;Convert string_utils module to Rust&quot;
│ ├── [<span style="color: magenta;">invariant_enforced</span>] &quot;Maintain API compatibility with existing C++ callers&quot;
│ ├── [<span style="color: magenta;">implementation_choice</span>] Use Rust&#x27;s String type, not custom implementation
│ │ Context: Analyzed 47 string_utils.cpp functions
│ │ Resources: string_utils.cpp, string_utils.h, 12 dependent files
│ │ Rationale: Rust&#x27;s String provides same guarantees with better ergonomics
│ └── ...
</code></pre></div>
Even months later, we can query: "Why did we use Rust's String type?" and get the exact context and reasoning, without reprocessing the entire codebase.
### Q: How does the system compute task-specific dependency closures for large-scale operations?
**What exists today architecturally**: The specification defines multiple mechanisms for computing and maintaining minimal dependency closures. The execution blueprint produced during the Strategize phase doesn't just list steps - it includes a complete dependency graph with explicit scoping for each operation.
**How dependency closure computation works**:
During the Strategize phase, the strategy actor employs several mechanisms to compute precise dependency closures:
1. **Resource-aware analysis**: The actor uses specialized skills to trace dependencies:
<div class="highlight"><pre><code>
<span style="opacity: 0.7;"># Pseudocode of what happens inside a strategy actor</span>
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">compute_closure_for_refactoring</span>(target_module):
closure = ResourceClosure()
<span style="opacity: 0.7;"># Direct file dependencies</span>
closure.add_files(find_imports(target_module))
closure.add_files(find_includes(target_module))
<span style="opacity: 0.7;"># Symbol dependencies</span>
<span style="color: magenta; font-weight: 600;">for</span> symbol <span style="color: magenta; font-weight: 600;">in</span> extract_exported_symbols(target_module):
closure.add_files(find_symbol_usage(symbol, scope=<span style="color: #66cc66;">&#x27;project&#x27;</span>))
<span style="opacity: 0.7;"># Test dependencies</span>
closure.add_files(find_tests_for_module(target_module))
<span style="opacity: 0.7;"># Build system dependencies</span>
closure.add_files(find_build_references(target_module))
<span style="color: magenta; font-weight: 600;">return</span> closure
</code></pre></div>
2. **Hierarchical scoping**: When spawning child plans (via `subplan_spawn` or `subplan_parallel_spawn`), each child plan receives:
- An explicit `relevant_resources` list
- A `sandbox_strategy` appropriate for those resources
- Clear boundaries of what it can and cannot modify
- The parent plan's effective invariant view (already reconciled from action, project, and global scopes)
3. **Decision-based tracking**: Each `subplan_spawn` decision records the child plan it creates, and `subplan_parallel_spawn` decisions group parallel child plans:
<div class="highlight"><pre><code>
<span style="opacity: 0.7;"># Individual child plan spawn</span>
<span style="color: cyan; font-weight: 600;">decision_type</span>: subplan_spawn
<span style="color: cyan; font-weight: 600;">chosen_option</span>: <span style="color: #66cc66;">&quot;Refactor authentication module&quot;</span>
<span style="color: cyan; font-weight: 600;">downstream_plan_ids</span>: [&quot;plan-auth-refactor-123&quot;]
<span style="color: cyan; font-weight: 600;">artifacts_produced</span>:
- <span style="color: cyan;">auth_module_files</span>: [&quot;auth.rs&quot;, &quot;auth_test.rs&quot;, &quot;auth_types.rs&quot;]
- <span style="color: cyan;">api_updates</span>: [&quot;api/v2/login.rs&quot;, &quot;api/v2/logout.rs&quot;]
<span style="opacity: 0.7;"># Parallel group of child plans</span>
<span style="color: cyan; font-weight: 600;">decision_type</span>: subplan_parallel_spawn
<span style="color: cyan; font-weight: 600;">chosen_option</span>: <span style="color: #66cc66;">&quot;Convert utility libraries in parallel&quot;</span>
<span style="opacity: 0.7;"># Contains subplan_spawn children, each with their own downstream_plan_ids</span>
</code></pre></div>
**Concrete example - Converting a subsystem to Rust**:
Let's trace how the system handles "Convert Firefox's Network Stack to Rust":
<div class="highlight"><pre><code>
STRATEGIZE PHASE:
1. Analyze network stack structure
- Identifies 847 C++ files in netwerk/ directory
- Traces public API surface (237 exported functions)
- Maps internal dependencies (1,432 internal calls)
2. Compute minimal closure for Phase 1 (DNS resolver):
- Core files: dns_resolver.cpp, dns_cache.cpp, dns_config.cpp (3 files)
- Direct dependencies: 12 files in netwerk/base/
- Test files: 8 test files specific to DNS
- Build files: 2 moz.build files
- Total closure: 25 files (not 847!)
3. Generate execution blueprint with child plans:
- [<span style="color: magenta;">subplan_parallel_spawn</span>] DNS module conversions:
- convert-dns-types: Closure of 5 files (type definitions)
- convert-dns-cache: Closure of 8 files (cache + tests)
- convert-dns-resolver: Closure of 12 files (resolver + integration)
</code></pre></div>
**Why this is tractable even for massive codebases**:
The system leverages several key insights about real software:
1. **Modular boundaries exist**: Even in legacy codebases, there are natural boundaries
2. **Changes are incremental**: We don't convert 50,000 files atomically
3. **Dependencies are sparse**: Most modules depend on a small fraction of the codebase
4. **Interfaces are narrow**: Public APIs are much smaller than implementations
The Firefox example would decompose into ~1,000 bounded child plans (grouped via `subplan_parallel_spawn` decisions where independent), each touching 10-100 files. The parent plan tracks the overall architecture and enforces invariants, while each child plan maintains its focused closure.
**How we prevent closure explosion**:
- **Lazy expansion**: Dependencies are traced only as deep as needed for correctness
- **Interface-based boundaries**: When possible, work against stable interfaces
- **Incremental validation**: Each child plan validates its changes don't break dependents
- **Hierarchical merge strategies**: Parent plans resolve conflicts between child plan changes
### Q: What mechanisms enforce global consistency during parallel execution across many files?
**What exists today architecturally**: The sandbox model combined with hierarchical plan execution provides strong guarantees about consistency during parallel execution. This isn't just process isolation - it's semantic isolation with intelligent merge strategies.
**How the coordination mechanism prevents compound errors**:
1. **Complete isolation during execution**:
Each plan executes in its own sandbox, which means:
<div class="highlight"><pre><code>
Plan A (refactoring auth module):
- Sandbox A1: Contains only auth/*.cpp, auth_tests/*.cpp
- Cannot see Plan B&#x27;s intermediate states
- Cannot accidentally depend on Plan B&#x27;s half-done work
Plan B (updating API endpoints):
- Sandbox B1: Contains only api/*.cpp, api_tests/*.cpp
- Makes changes assuming current auth interface
- Protected from Plan A&#x27;s intermediate refactoring
</code></pre></div>
2. **Resource-specific sandbox strategies provide natural coordination**:
<div class="highlight"><pre><code>
Git repositories:
- <span style="color: cyan;">Strategy</span>: git worktrees
- <span style="color: cyan;">Coordination</span>: Git&#x27;s three-way merge algorithm
- Conflict detection: Built into Git
- <span style="color: cyan;">Rollback</span>: git reset/checkout
<span style="color: cyan; font-weight: 600;">Databases</span>:
- <span style="color: cyan;">Strategy</span>: Transaction isolation
- <span style="color: cyan;">Coordination</span>: MVCC (multi-version concurrency control)
- Conflict detection: Serialization failures
- <span style="color: cyan;">Rollback</span>: Transaction abort
Cloud Infrastructure:
- <span style="color: cyan;">Strategy</span>: Terraform workspaces
- <span style="color: cyan;">Coordination</span>: State locking
- Conflict detection: Resource conflicts in plan
- <span style="color: cyan;">Rollback</span>: Previous state restoration
</code></pre></div>
3. **Hierarchical merge resolution**:
When child plans complete, the parent plan performs intelligent merging:
<div class="highlight"><pre><code>
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">merge_subplan_results</span>(subplan_results):
<span style="opacity: 0.7;"># Group by resource type</span>
by_resource = group_by_resource_type(subplan_results)
<span style="opacity: 0.7;"># Apply resource-specific merge strategies</span>
<span style="color: magenta; font-weight: 600;">for</span> resource_type, changes <span style="color: magenta; font-weight: 600;">in</span> by_resource:
<span style="color: magenta; font-weight: 600;">if</span> resource_type == <span style="color: #66cc66;">&#x27;git-checkout&#x27;</span>:
merge_git_changes(changes) <span style="opacity: 0.7;"># Three-way merge</span>
<span style="color: magenta; font-weight: 600;">elif</span> resource_type == <span style="color: #66cc66;">&#x27;fs-mount&#x27;</span>:
merge_fs_changes(changes) <span style="opacity: 0.7;"># Copy-on-write reconciliation</span>
<span style="color: magenta; font-weight: 600;">elif</span> resource_type.startswith(<span style="color: #66cc66;">&#x27;database&#x27;</span>):
merge_db_changes(changes) <span style="opacity: 0.7;"># Sequential application</span>
<span style="opacity: 0.7;"># Validate merged state</span>
run_integration_tests()
</code></pre></div>
**Concrete example - Preventing cascading failures**:
Consider refactoring a shared authentication library used by 15 services:
<div class="highlight"><pre><code>
PARALLEL EXECUTION WITHOUT COORDINATION (what we prevent):
- Service A refactors to async auth → breaks Service B
- Service B compensates with workaround → breaks Service C
- Service C changes error handling → breaks Services D, E, F
- Cascade of failures!
CLEVERAGENTS COORDINATED EXECUTION:
Parent Plan: Refactor auth library
├── [<span style="color: magenta;">invariant_enforced</span>] &quot;Auth library public API must remain backward compatible during transition&quot;
├── [<span style="color: magenta;">subplan_spawn</span>] Plan 1: Update auth library interface
│ Sandbox: Only auth library files
│ Output: New interface definition
│
├── Barrier: Wait for Plan 1 completion
│
├── [<span style="color: magenta;">subplan_parallel_spawn</span>] Plans 2-16: Update each service (in parallel)
│ Each sandbox: Only that service&#x27;s files
│ Each uses: New interface from Plan 1
│ No inter-service dependencies during execution
│
└── Merge Phase:
- Collect all service updates
- <span style="color: yellow; font-weight: 600;">Apply</span> to main branch in order
- Run integration tests
- If conflicts: Parent plan resolves using semantic understanding
</code></pre></div>
**Advanced coordination patterns**:
1. **Optimistic concurrency with semantic conflict resolution**:
<div class="highlight"><pre><code>
Two child plans both modify api/user.rs:
- Plan A: Adds async fn get_user_profile()
- Plan B: Adds fn validate_user_permissions()
Merge strategy:
- Git merge succeeds (different functions)
- Semantic validation ensures both functions work together
- Parent plan adds integration glue if needed
</code></pre></div>
2. **Checkpoint-based coordination**:
<div class="highlight"><pre><code>
Execution timeline:
T1: Plan A creates checkpoint before major refactor
T2: Plan B creates checkpoint before API changes
T3: Plan A encounters error, rolls back to T1
T4: Plan B completes successfully
T5: Plan A retries with knowledge of B&#x27;s success
</code></pre></div>
3. **Resource locking for critical sections**:
<div class="highlight"><pre><code>
When modifying shared schema files:
- Acquire exclusive lock on schema resources
- Make changes atomically
- Release lock with new version
- Other plans rebase on new schema
</code></pre></div>
### Q: How does the system proactively prevent semantic errors before they propagate?
**What exists today architecturally**: The specification defines multiple layers of proactive error prevention that go far beyond traditional testing. This is a comprehensive defense-in-depth approach that catches semantic errors before they can propagate.
**Layer 1: Decision-time validation during Strategize**:
Every decision includes semantic validation:
<div class="highlight"><pre><code>
<span style="color: cyan; font-weight: 600;">Decision</span>: Refactor payment module to async
<span style="color: cyan; font-weight: 600;">alternatives_considered</span>:
- &quot;Convert to async/await patterns&quot; (chosen)
- &quot;Use thread pool with channels&quot; (rejected: doesn&#x27;t integrate with async ecosystem)
- &quot;Keep synchronous with timeout&quot; (rejected: doesn&#x27;t solve core latency issue)
<span style="color: cyan; font-weight: 600;">confidence_score</span>: <span style="color: yellow;">0.85</span>
<span style="color: cyan; font-weight: 600;">validation_performed</span>:
- Checked all payment API consumers can handle async
- Verified database driver supports async operations
- Confirmed no regulatory requirement for sync processing
</code></pre></div>
**Layer 2: Execution-time semantic guards**:
The execution actor uses a tool node that references the independently registered `local/validate-api-compat` tool:
<div class="highlight"><pre><code>
<span style="color: cyan; font-weight: 600;">actors</span>:
<span style="color: cyan; font-weight: 600;">code_executor</span>:
<span style="color: cyan; font-weight: 600;">type</span>: graph
<span style="color: cyan; font-weight: 600;">skills</span>:
- local/semantic-validators # Contains validation tools for LLM tool-calling
<span style="color: cyan; font-weight: 600;">nodes</span>:
- <span style="color: cyan;">name</span>: semantic_validator
<span style="color: cyan; font-weight: 600;">type</span>: tool
<span style="color: cyan; font-weight: 600;">tool</span>: local/validate-api-compat <span style="opacity: 0.7;"># Named tool from Tool Registry</span>
</code></pre></div>
The `local/validate-api-compat` tool (independently registered via `agents tool add`) performs semantic validation — not just syntax checking. It extracts API signatures, finds breaking changes, checks affected consumers, and either auto-migrates or raises a `SemanticError` for manual review.
**Layer 3: Invariant enforcement through the decision tree**:
Invariants are attached at four scopes (global, project, action, plan) and managed via the unified `agents invariant` command or `--invariant` flags on creation commands. When a plan enters Strategize, the **Invariant Reconciliation Actor** (set via `--invariant-actor` on actions/plans/projects, or globally via `agents config set actor.default.invariant`) computes the effective invariant view by applying precedence rules (plan > project > global) to resolve conflicts. Each effective invariant is recorded as an `invariant_enforced` decision, making them visible, correctable, and auditable:
<div class="highlight"><pre><code>
<span style="opacity: 0.7;"># The system collects, reconciles, and enforces semantic invariants</span>
<span style="color: magenta; font-weight: 600;">class</span> <span style="color: cyan; font-weight: 600;">InvariantEnforcer</span>:
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">compute_effective_invariants</span>(self, plan):
<span style="color: #66cc66;">&quot;&quot;&quot;Compute the effective invariant view using the Invariant Reconciliation Actor.&quot;&quot;&quot;</span>
raw = self.collect_all_invariants(plan)
reconciler = (
self.get_plan_invariant_actor(plan)
<span style="color: magenta; font-weight: 600;">or</span> self.get_project_invariant_actor(plan)
<span style="color: magenta; font-weight: 600;">or</span> self.get_global_invariant_actor()
)
<span style="opacity: 0.7;"># Apply precedence: plan &gt; project &gt; global</span>
<span style="color: magenta; font-weight: 600;">return</span> reconciler.reconcile(raw, precedence=[<span style="color: #66cc66;">&#x27;plan&#x27;</span>, <span style="color: #66cc66;">&#x27;project&#x27;</span>, <span style="color: #66cc66;">&#x27;global&#x27;</span>])
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">collect_all_invariants</span>(self, plan):
<span style="color: #66cc66;">&quot;&quot;&quot;Collect invariants from all scopes accessible to this plan.&quot;&quot;&quot;</span>
invariants = []
invariants.extend(self.get_global_invariants())
<span style="color: magenta; font-weight: 600;">for</span> project <span style="color: magenta; font-weight: 600;">in</span> plan.projects:
invariants.extend(self.get_project_invariants(project))
invariants.extend(self.get_action_invariants(plan.action))
invariants.extend(self.get_plan_invariants(plan))
<span style="color: magenta; font-weight: 600;">return</span> invariants
<span style="opacity: 0.7;"># Example invariants at different scopes:</span>
<span style="opacity: 0.7;"># Global: &quot;Payment processing must be idempotent&quot;</span>
<span style="opacity: 0.7;"># Project: &quot;Database transactions must complete within 5 seconds&quot;</span>
<span style="opacity: 0.7;"># Action: &quot;Test files must not import production secrets&quot;</span>
<span style="opacity: 0.7;"># Plan: &quot;All API calls over TCP must be mocked&quot;</span>
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">check_invariant_preservation</span>(self, changes, enforced_invariants):
<span style="color: magenta; font-weight: 600;">for</span> invariant <span style="color: magenta; font-weight: 600;">in</span> enforced_invariants:
<span style="color: magenta; font-weight: 600;">if</span> <span style="color: magenta; font-weight: 600;">not</span> self.verify_invariant(invariant, changes):
<span style="color: magenta; font-weight: 600;">return</span> InvariantViolation(invariant, changes)
<span style="color: magenta; font-weight: 600;">return</span> Success()
</code></pre></div>
**Layer 4: Predictive error prevention through pattern matching**:
The system learns from past failures:
<div class="highlight"><pre><code>
Error Pattern Database:
- <span style="color: cyan;">pattern</span>: <span style="color: #66cc66;">&quot;Async conversion in payment module&quot;</span>
<span style="color: cyan; font-weight: 600;">historical_failures</span>:
- &quot;Race condition in payment confirmation&quot;
- &quot;Timeout handling breaks idempotency&quot;
<span style="color: cyan; font-weight: 600;">preventive_checks</span>:
- &quot;Add explicit transaction boundaries&quot;
- &quot;Verify idempotency keys are preserved&quot;
- &quot;Check distributed lock acquisition&quot;
</code></pre></div>
**Concrete example - Preventing a subtle distributed systems bug**:
Scenario: Refactoring a service to use event sourcing:
<div class="highlight"><pre><code>
PROACTIVE CONTAINMENT IN ACTION:
1. Strategy Phase Semantic Analysis:
- Decision: &quot;Convert order service to event sourcing&quot;
- Semantic check: &quot;Event sourcing requires eventual consistency&quot;
- Identifies: 3 services assume immediate consistency
- Adds decision: &quot;Update dependent services for eventual consistency&quot;
2. Execution Phase Invariant Checking:
- Detects: PaymentService.chargeCard() called after OrderCreated event
- Semantic issue: Payment before order confirmation violates business rules
- Automatic fix: Insert OrderConfirmed event requirement
3. Validation Node Catches Edge Case:
- Discovers: Audit service expects synchronous order numbers
- Impact: Async events break compliance reporting
- Resolution: Add audit event buffer with guaranteed ordering
4. Pre-<span style="color: yellow; font-weight: 600;">Apply</span> Semantic Verification:
- Simulates production event flow
- Detects: Under high load, events can arrive out of order
- Adds: Event ordering guarantees via vector clocks
</code></pre></div>
**Why this prevents issues that traditional testing misses**:
Traditional tests check "does the code work?" Our semantic containment asks:
- Does it preserve business invariants?
- Does it maintain architectural patterns?
- Does it respect distributed systems principles?
- Does it handle the edge cases we've seen before?
**Integration with Definition of Done (DoD)**:
Each plan's DoD includes semantic requirements:
<div class="highlight"><pre><code>
<span style="color: cyan; font-weight: 600;">definition_of_done</span>:
<span style="color: cyan; font-weight: 600;">must</span>:
- &quot;All API changes maintain backward compatibility&quot;
- &quot;No increase in p99 latency&quot;
- &quot;Audit trail remains complete&quot;
<span style="color: cyan; font-weight: 600;">should</span>:
- &quot;Improve code coverage by 10%&quot;
- &quot;Reduce cyclomatic complexity&quot;
<span style="color: cyan; font-weight: 600;">may</span>:
- &quot;Optimize for memory usage&quot;
</code></pre></div>
The validation nodes enforce these semantics, not just test passage.
### Q: How does the system balance human supervision with autonomous operation?
**What exists today architecturally**: The specification defines a comprehensive automation profile system with 10 confidence thresholds (0.0–1.0) and 3 boolean safety flags controlling every aspect of human-vs-automated operation. This isn't a binary human/AI split — it's a continuously adjustable matrix where each threshold specifies the minimum confidence level for automatic operation, tunable per task, per project, or per organization.
**How the automation profiles work in practice**:
<div class="highlight"><pre><code>
manual Profile (all thresholds = 1.0):
- Every phase transition pauses for human action
- Every decision point pauses for human input
- User sees: Context, alternatives, recommendation, confidence score
- User provides: Explicit choice or custom guidance
- Use case: New users, critical production changes, first-time codebase exploration
review Profile (phase thresholds = 0.0, decision thresholds = 1.0):
- Phases run automatically (Strategize, Execute proceed without pausing)
- Every decision within those phases pauses for human input
- Apply is manual; transient retries and child plans are automatic
- Use case: Teams that want to stay in the decision loop without triggering phases
supervised Profile (strategize thresholds = 0.0, execute/apply = 1.0):
- Strategize runs automatically with autonomous decisions
- System pauses before Execute for human review of strategy
- Transient failures retried automatically
- Use case: Projects where you trust planning but want to review before execution
cautious Profile (intermediate thresholds = 0.6-0.8):
- System proceeds automatically only when Semantic Escalation reports high confidence
- Escalates to human when uncertain (confidence below threshold)
- Higher bars (0.8) for risky operations, lower bars (0.6) for safer ones
- Use case: Gradual automation adoption, mixed-complexity projects
trusted Profile (most thresholds = 0.0, apply = 1.0):
- Strategize and Execute run automatically with autonomous decisions
- Validation failures self-fixed within strategy bounds
- Child plans spawned automatically
- System pauses before Apply for human diff review
- Use case: Normal feature development, refactoring
autonomous Profile (all thresholds = 0.0 except apply = 1.0):
- Everything automatic except Apply
- Can revise its own strategy if execution hits a wall
- Restores from checkpoints on failure
- Use case: Well-understood projects, batch operations
ci Profile (all thresholds = 0.0, sandbox + checkpoints required):
- Complete end-to-end automation including Apply
- Sandbox and checkpoints remain mandatory for safety
- Unsafe tools blocked
- Use case: CI/CD pipelines, headless batch jobs, scheduled automation
full-auto Profile (all thresholds = 0.0):
- Complete end-to-end automation including Apply
- No sandbox or checkpoint requirements
- Unsafe tools allowed
- Use case: Routine updates, environments with external safety mechanisms
Custom profiles with intermediate thresholds (e.g., 0.3, 0.5, 0.7):
- Fine-grained control: auto-execute with confidence >= 0.5 but escalate below
- Different thresholds per flag: low bar for retries, high bar for strategy revision
- Enables &quot;trust but verify&quot; patterns without binary on/off
- Use case: Teams building confidence gradually, mixed-risk projects
</code></pre></div>
**The decision correction mechanism enables progressive automation**:
The `agents plan correct` command is crucial for building trust:
<div class="highlight"><pre><code>
<span style="opacity: 0.7;"># User observes AI made suboptimal choice</span>
agents plan tree &lt;plan_id&gt;
<span style="opacity: 0.7;"># Sees: [Decision] &quot;Use REST API for service communication&quot;</span>
<span style="opacity: 0.7;"># User knows gRPC would be better for this use case</span>
agents plan correct &lt;decision_id&gt; <span style="color: cyan;">--mode</span>=revert <span style="opacity: 0.7;">\</span>
<span style="color: cyan;">--guidance</span> &quot;Use gRPC instead of REST. This service requires streaming
updates and binary protocol efficiency. Set up protocol
buffer definitions and generate client/server stubs.&quot;
<span style="opacity: 0.7;"># System:</span>
<span style="opacity: 0.7;"># 1. Marks original decision as superseded</span>
<span style="opacity: 0.7;"># 2. Creates new decision with user guidance</span>
<span style="opacity: 0.7;"># 3. Recomputes ONLY affected downstream decisions</span>
<span style="opacity: 0.7;"># 4. Preserves all unrelated work</span>
</code></pre></div>
**Progressive trust building through automation profiles**:
New users typically follow this progression:
1. Start with `manual` to understand system behavior
2. Move to `supervised` as confidence in the planning phase builds, or customize a profile with intermediate thresholds (e.g., 0.5) for gradual autonomy
3. Adopt `trusted` for routine development tasks
4. Enable `auto` for well-understood projects
5. Use `full-auto` for low-risk batch operations
**Real autonomy through semantic understanding**:
True autonomy isn't about removing humans - it's about the system understanding when it needs help:
<div class="highlight"><pre><code>
<span style="color: magenta; font-weight: 600;">class</span> <span style="color: cyan; font-weight: 600;">AutonomyController</span>:
<span style="color: magenta; font-weight: 600;">def</span> <span style="color: cyan; font-weight: 600;">assess_decision_confidence</span>(self, decision, context, profile):
factors = {
<span style="color: #66cc66;">&#x27;past_success_rate&#x27;</span>: self.get_historical_success(decision.type),
<span style="color: #66cc66;">&#x27;codebase_familiarity&#x27;</span>: self.get_familiarity_score(context.project),
<span style="color: #66cc66;">&#x27;risk_assessment&#x27;</span>: self.evaluate_risk(decision),
<span style="color: #66cc66;">&#x27;invariant_complexity&#x27;</span>: self.analyze_invariants(decision)
}
confidence = self.compute_confidence(factors) <span style="opacity: 0.7;"># Returns 0.0–1.0</span>
threshold = profile.get_threshold(decision.flag) <span style="opacity: 0.7;"># From profile</span>
<span style="color: magenta; font-weight: 600;">if</span> confidence &gt;= threshold:
<span style="color: magenta; font-weight: 600;">return</span> ProceedAutonomously(decision, confidence)
<span style="color: magenta; font-weight: 600;">else</span>:
<span style="opacity: 0.7;"># Confidence below profile threshold — escalate to human</span>
<span style="color: magenta; font-weight: 600;">return</span> RequestHumanGuidance(decision, confidence, threshold, factors)
</code></pre></div>
**Concrete example - Autonomous handling of a complex refactoring**:
<div class="highlight"><pre><code>
Scenario: &quot;Modernize legacy e-commerce system&quot;
INITIAL PLAN (Full Automation Mode):
1. System analyzes 50,000 line codebase
2. Identifies modernization opportunities
3. Creates plan with 47 child plans
AUTONOMOUS EXECUTION WITH SMART ESCALATION:
[<span style="color: magenta;">subplan_parallel_spawn</span>] Plans 1-15: Update utility functions (executes autonomously)
- Confidence: 0.95 (straightforward transformations)
- Result: Success
[<span style="color: magenta;">subplan_spawn</span>] Plan 16: Refactor payment processing
- Confidence: 0.4 (critical business logic)
- [<span style="color: magenta;">invariant_enforced</span>] &quot;Preserve exact penny rounding behavior&quot;
- <span style="color: yellow; font-weight: 600;">Action</span>: ESCALATES to human
- Human provides: &quot;Preserve exact penny rounding behavior&quot;
- Continues autonomously with constraint
[<span style="color: magenta;">subplan_parallel_spawn</span>] Plans 17-30: UI component updates (executes autonomously)
- Confidence: 0.9 (isolated changes)
- Result: Success
[<span style="color: magenta;">subplan_spawn</span>] Plan 31: Database schema migration
- Detects: Would require 6-hour downtime
- <span style="color: yellow; font-weight: 600;">Action</span>: ESCALATES to human
- Human provides: &quot;Use online migration with feature flags&quot;
- Re-plans with zero-downtime approach
[<span style="color: magenta;">subplan_parallel_spawn</span>] Plans 32-47: Complete autonomously
</code></pre></div>
**The path to greater autonomy**:
The system becomes more autonomous through:
1. **Learning from corrections**:
- Every correction teaches the system about user preferences
- Patterns emerge: "This team always prefers gRPC for microservices"
- Future decisions incorporate these learnings
2. **Building project-specific context**:
- Each successful plan adds to project knowledge
- System learns codebase patterns, team conventions, business rules
- Confidence increases with familiarity
3. **Hierarchical delegation**:
- Proven child plan patterns become fully autonomous
- Human focuses on high-level decisions
- System handles implementation details
4. **Semantic safety nets**:
- Comprehensive invariant checking reduces risk
- Rollback capabilities provide recovery path
- Humans can trust system won't cause catastrophic failures
### Q: What's actually implemented today versus planned for the future?
**Concrete implementations in the architecture**:
1. **Decision Tree with Complete Context Capture**
- Full schema defined
- Storage model specified
- Correction mechanism detailed
- Query patterns established
2. **Hierarchical Plan System**
- Spawning mechanism defined (sequential `subplan_spawn` and parallel `subplan_parallel_spawn`)
- Execution semantics specified (child plans are full Plans with their own decision trees)
- Invariant inheritance from parent plans, projects, and global scope
- Merge strategies documented
- Failure handling described
3. **Resource-Aware Sandbox Isolation**
- Multiple strategies defined (git worktree, filesystem overlay, transactions)
- Lazy sandboxing for efficiency
- Resource-specific merge algorithms
- Cleanup behavior specified
4. **Multi-Layer Error Prevention**
- Decision validation during planning
- Semantic validation nodes
- Invariant enforcement
- Definition of Done checking
5. **Graduated Automation Controls**
- Three levels clearly defined
- Decision correction without full re-execution
- Confidence-based escalation
- Progressive trust building
**Near-term implementations** (architecture complete, engineering straightforward):
1. **Memory Tier Management**
- Hot/warm/cold distinction clear
- Context loading patterns defined
- Just needs LRU cache and storage backend
2. **Cross-Plan Learning**
- Decision history provides training data
- Pattern extraction is standard ML
- Confidence scoring is well-understood
3. **Cost/Risk Estimation**
- Dedicated estimation actor role defined
- Historical data provides baselines
- Standard prediction problem
4. **Extended Validation Patterns**
- Pluggable validation architecture
- Project-specific rules as configuration
- Industry patterns can be packaged
**Research territory** (requires innovation but architecture supports):
1. **Optimal Context Selection for 100K+ file codebases**
- Current: Heuristic-based selection
- Research: ML-driven relevance ranking
- Architecture supports: Any selection algorithm can plug in
2. **Automated Invariant Discovery**
- Current: User-defined invariants
- Research: Mining invariants from code patterns
- Architecture supports: Invariants are just validation rules
3. **Cross-Project Knowledge Transfer**
- Current: Project-specific learning
- Research: Generalized pattern recognition
- Architecture supports: Cold tier can span projects
4. **Fully Autonomous Recovery Strategies**
- Current: Automatic retry for transient errors with structured
recovery hints and `agents plan errors` CLI (D1b)
- Research: Automatic error understanding and fixing
- Architecture supports: Recovery is just another plan type
**Why we can confidently handle Firefox-scale projects**:
The architecture doesn't require magical AI breakthroughs. It requires:
- **Hierarchical decomposition**: ✓ Fully specified
- **Bounded context operations**: ✓ Dependency closure computation defined
- **Parallel execution with isolation**: ✓ Sandbox model complete
- **Semantic validation**: ✓ Multi-layer approach specified
- **Progressive automation**: ✓ Automation profiles and correction defined
The difference between handling a 1,000 file project and a 100,000 file project is:
- More child plans (hierarchical decomposition handles this)
- Larger cold storage (standard database scaling)
- Better context selection (improves with use but works with heuristics)
- More validation patterns (accumulate over time)
This isn't speculative architecture astronautics - it's applying proven distributed systems principles to AI agent coordination. The innovation is in the integration, not in requiring fundamental breakthroughs.
Reference in New Issue View Git Blame Copy Permalink