From a3762a4cc28b24dad667a304329cbfa258785a55 Mon Sep 17 00:00:00 2001 From: CleverThis Date: Thu, 9 Apr 2026 09:22:43 +0000 Subject: [PATCH] docs: add invariant-reconciliation module guide, extend automation tracking docs, update mkdocs nav docs: add invariant-reconciliation module guide, extend automation tracking docs, update mkdocs nav - Add docs/modules/invariant-reconciliation.md: module guide for InvariantReconciliationActor (v3.8.0) - Extend docs/development/automation-tracking.md: all 16 supervisors, centralized tracking manager - Update mkdocs.yml: add Modules nav section, Custom Sandbox Strategy to Development ISSUES CLOSED: #5700 Co-authored-by: CleverThis Co-committed-by: CleverThis --- docs/development/automation-tracking.md | 102 +++++++++++- docs/modules/invariant-reconciliation.md | 202 +++++++++++++++++++++++ mkdocs.yml | 5 + 3 files changed, 305 insertions(+), 4 deletions(-) create mode 100644 docs/modules/invariant-reconciliation.md diff --git a/docs/development/automation-tracking.md b/docs/development/automation-tracking.md index 6711acc9f..4a1e52dec 100644 --- a/docs/development/automation-tracking.md +++ b/docs/development/automation-tracking.md @@ -44,12 +44,73 @@ For emergency announcements or important messages: | system-watchdog | `AUTO-WATCHDOG` | `[AUTO-WATCHDOG] System Health (Cycle 8)` | | backlog-groomer | `AUTO-GROOMER` | `[AUTO-GROOMER] Grooming Report (Cycle 23)` | | human-liaison | `AUTO-LIAISON` | `[AUTO-LIAISON] Status Update (Cycle 67)` | +| product-builder | `AUTO-PROD-BLDR` | `[AUTO-PROD-BLDR] Build Session (Cycle 5)` | +| architect | `AUTO-ARCH` | `[AUTO-ARCH] Architecture Report (Cycle 3)` | +| timeline-updater | `AUTO-TIME` | `[AUTO-TIME] Timeline Update (Cycle 12)` | +| docs-writer | `AUTO-DOCS` | `[AUTO-DOCS] Documentation Report (Cycle 7)` | +| architecture-guard | `AUTO-GUARD` | `[AUTO-GUARD] Guard Report (Cycle 9)` | +| continuous-pr-reviewer | `AUTO-REV-POOL` | `[AUTO-REV-POOL] Review Status (Cycle 4)` | +| uat-tester | `AUTO-UAT-POOL` | `[AUTO-UAT-POOL] UAT Status (Cycle 6)` | +| project-owner | `AUTO-PROJ-OWN` | `[AUTO-PROJ-OWN] Project Status (Cycle 11)` | +| agent-evolver | `AUTO-EVLV` | `[AUTO-EVLV] Agent Evolution Report (Cycle 10)` | +| bug-hunter | `AUTO-BUG-POOL` | `[AUTO-BUG-POOL] Bug Detection Pool Status (Cycle 60)` | +| spec-updater | `AUTO-SPEC` | `[AUTO-SPEC] Specification Update Report (Cycle 1)` | +| test-infra-improver | `AUTO-INF-POOL` | `[AUTO-INF-POOL] Infrastructure Analysis Report (Cycle 2)` | ## Required Labels All automation tracking issues MUST include: - **`Automation Tracking`** - Primary label for filtering -- Additional agent-specific labels as appropriate +- **`Type/Automation`** - Type label for automation issues +- **`State/In Progress`** - State label while agent is active +- **`Priority/Medium`** - Default priority (adjust as needed) + +## Centralized Tracking Manager + +The **`automation-tracking-manager`** subagent is the single source of truth for all +automation tracking operations. It was introduced to prevent cycle reuse issues where +agents would incorrectly comment on old tracking issues instead of creating new ones. + +### Why Centralized? + +Without centralization, agents experienced: +- **Cycle reuse**: Agents posting future cycle reports as comments on old issues +- **Duplicate issues**: Multiple tracking issues created for the same cycle +- **Missing cycles**: Agents skipping cycle numbers due to stale issue references +- **Inconsistent cleanup**: Previous cycle issues not always deleted before creating new ones + +### How It Works + +The `automation-tracking-manager` subagent: +1. Maintains a persistent cycle counter per agent prefix +2. Searches for and closes the previous cycle's tracking issue +3. Creates a new tracking issue with the correct sequential cycle number +4. Applies all required labels atomically +5. Returns the new issue number to the calling agent + +### Usage + +Agents that use the centralized manager delegate all tracking operations to it: + +```bash +# Example: implementation-orchestrator delegating to tracking manager +# (Inside agent definition — pseudocode) +TRACKING_RESULT=$(call_subagent automation-tracking-manager \ + --prefix "AUTO-IMP-POOL" \ + --type "Health Report" \ + --body "...") +NEW_ISSUE_NUMBER=$(echo "$TRACKING_RESULT" | jq -r '.issue_number') +``` + +### Migrated Agents + +The following agents have been migrated to use the centralized tracking manager: +- `system-watchdog` (most critical — was skipping cycles) +- `implementation-orchestrator` (was creating duplicate issues) +- `timeline-updater` +- `project-owner` +- `product-builder` +- `backlog-groomer` ## Interval Reporting and Health Monitoring @@ -74,6 +135,18 @@ This enables automated health monitoring by the system-watchdog to detect stalle | human-liaison | Status Update | Every 20 minutes (10 cycles) | Human activity monitoring | | system-watchdog | Health Report | Every 30 minutes (6 cycles) | System-wide health check | | session-persister | Checkpoint | Event-driven (variable) | After significant state changes | +| product-builder | Build Session | Every 10 cycles (~variable timing) | Full build session status | +| architect | Architecture Report | Every 10 cycles (~variable timing) | Architecture supervision status | +| timeline-updater | Timeline Update | Every 10 cycles (~variable timing) | Timeline maintenance status | +| docs-writer | Documentation Report | Every 10 cycles (~3.3 hours) | Documentation update status | +| architecture-guard | Guard Report | Every 10 cycles (~variable timing) | Codebase coherence check | +| continuous-pr-reviewer | Review Status | Every 10 cycles (~variable timing) | PR review pool status | +| uat-tester | UAT Status | Every 10 cycles (~variable timing) | UAT testing progress | +| project-owner | Project Status | Every 10 cycles (~variable timing) | Project ownership decisions | +| agent-evolver | Evolution Report | Every 10 cycles (~variable timing) | Agent improvement proposals | +| bug-hunter | Bug Detection Status | Every 10 cycles (~variable timing) | Bug detection pool health | +| spec-updater | Specification Update | Every 10 cycles (~variable timing) | Spec evolution status | +| test-infra-improver | Infrastructure Report | Every 10 cycles (~variable timing) | Test infrastructure analysis | ### Automated Health Monitoring @@ -196,7 +269,7 @@ Every agent that creates tracking issues must implement: 1. **Cleanup Function**: Delete previous cycle tracking issues before creating new ones 2. **Standardized Titles**: Use the exact format specified above -3. **Proper Labels**: Always include "Automation Tracking" label +3. **Proper Labels**: Always include "Automation Tracking", "Type/Automation", "State/In Progress", and "Priority/Medium" labels 4. **Structured Content**: Use consistent formatting for issue bodies ### Standard Tracking Functions @@ -214,7 +287,7 @@ Each agent should implement these functions: ### Create New Tracking Issue 1. Generate title with current cycle number -2. Include "Automation Tracking" label +2. Include "Automation Tracking", "Type/Automation", "State/In Progress", "Priority/Medium" labels 3. Add structured content with timestamp and status 4. Link to relevant repositories/PRs as needed ``` @@ -267,6 +340,18 @@ label:"Automation Tracking" [AUTO-IMP-POOL] in:title label:"Automation Tracking" [AUTO-WATCHDOG] in:title label:"Automation Tracking" [AUTO-GROOMER] in:title label:"Automation Tracking" [AUTO-LIAISON] in:title +label:"Automation Tracking" [AUTO-PROD-BLDR] in:title +label:"Automation Tracking" [AUTO-ARCH] in:title +label:"Automation Tracking" [AUTO-TIME] in:title +label:"Automation Tracking" [AUTO-DOCS] in:title +label:"Automation Tracking" [AUTO-GUARD] in:title +label:"Automation Tracking" [AUTO-REV-POOL] in:title +label:"Automation Tracking" [AUTO-UAT-POOL] in:title +label:"Automation Tracking" [AUTO-PROJ-OWN] in:title +label:"Automation Tracking" [AUTO-EVLV] in:title +label:"Automation Tracking" [AUTO-BUG-POOL] in:title +label:"Automation Tracking" [AUTO-SPEC] in:title +label:"Automation Tracking" [AUTO-INF-POOL] in:title ``` **Recent tracking issues (last 24 hours):** @@ -324,6 +409,11 @@ If tracking issues accumulate excessively: - Check if agent is actually running and creating issues - Confirm agent is using correct title format +**Agents posting to old tracking issues (cycle reuse):** +- Agent is not using the centralized `automation-tracking-manager` +- Migrate agent to use the centralized manager subagent +- Check that previous cycle cleanup runs before new issue creation + ### Debugging Steps 1. **Check agent status**: Verify agent is running and healthy @@ -346,8 +436,12 @@ The migration was completed by updating agent definitions in `.opencode/agents/` - Implement cleanup protocols - Apply standardized formatting +A second migration introduced the centralized `automation-tracking-manager` subagent +to prevent cycle reuse issues where agents were commenting on old tracking issues +instead of creating new ones. + ## Related Documentation - [System Watchdog](system-watchdog.md) - Specific documentation for watchdog agent - [Quality Automation](quality-automation.md) - Quality gate automation details -- [Ops Runbook](ops-runbook.md) - General operational procedures \ No newline at end of file +- [Ops Runbook](ops-runbook.md) - General operational procedures diff --git a/docs/modules/invariant-reconciliation.md b/docs/modules/invariant-reconciliation.md new file mode 100644 index 000000000..bb6271165 --- /dev/null +++ b/docs/modules/invariant-reconciliation.md @@ -0,0 +1,202 @@ +# Invariant Reconciliation Module + +**Package:** `cleveragents.actor.reconciliation` +**Introduced:** v3.8.0 + +The Invariant Reconciliation Actor is a built-in actor that automatically runs at +every plan phase transition to ensure all active invariants are consistent and +correctly prioritised before any strategy or execution work begins. + +For the invariant data model and scope hierarchy, see +[`docs/reference/invariants.md`](../reference/invariants.md). +For the `InvariantService` API, see +[`docs/api/core.md`](../api/core.md). + +--- + +## Purpose + +Invariants are natural-language constraints on plan execution that can be defined +at four scopes: global, project, action, and plan. When multiple invariants apply +to the same plan, conflicts can arise — for example, a project-level invariant may +contradict a plan-level one. The reconciliation actor resolves these conflicts +deterministically using a precedence chain and records each decision in the audit +trail. + +--- + +## Automatic Invocation + +`PlanLifecycleService` automatically invokes the reconciliation actor at the start +of three phase transitions: + +| Transition | Method | +|------------|--------| +| Strategize start | `start_strategize()` | +| Execute start | `execute_plan()` | +| Apply start | `apply_plan()` | + +If reconciliation fails, the phase transition is **blocked** with +`ReconciliationBlockedError` and an `INVARIANT_VIOLATED` event is emitted on the +event bus. The transition cannot proceed until the invariants are corrected. + +Post-correction reconciliation runs automatically via a `CORRECTION_APPLIED` event +subscription (best-effort; does not block correction completion). + +--- + +## Reconciliation Algorithm + +The actor follows a six-step algorithm: + +1. **Load** — fetch all invariants in scope for the plan (global → project → action → plan) +2. **Deduplicate** — remove exact-text duplicates, keeping the highest-scope copy +3. **Detect conflicts** — identify pairs where one invariant's constraint contradicts another's +4. **Resolve** — apply the precedence chain: `plan > action > project > global`; when scopes are equal, the invariant with the lower `position` index wins +5. **Record** — write each resolution decision to the audit trail with rationale +6. **Emit** — publish `INVARIANT_RECONCILED` event on the event bus + +--- + +## Key Classes + +### `InvariantReconciliationActor` + +```python +from cleveragents.actor.reconciliation import InvariantReconciliationActor + +actor = InvariantReconciliationActor(invariant_service, event_bus, audit_service) +result = await actor.reconcile(plan_id) +``` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `invariant_service` | `InvariantService` | Provides invariant CRUD and scope queries | +| `event_bus` | `EventBus` | Receives `INVARIANT_RECONCILED` / `INVARIANT_VIOLATED` events | +| `audit_service` | `AuditService` | Records reconciliation decisions | + +### `ReconciliationResult` + +```python +@dataclass +class ReconciliationResult: + plan_id: str + resolved_invariants: list[Invariant] + conflicts_detected: int + conflicts_resolved: int + decisions: list[ReconciliationDecision] + success: bool +``` + +### `ReconciliationDecision` + +```python +@dataclass +class ReconciliationDecision: + winner: Invariant + loser: Invariant + rationale: str # e.g. "plan scope > project scope" + timestamp: datetime +``` + +--- + +## Error Handling + +| Exception | When raised | +|-----------|-------------| +| `ReconciliationBlockedError` | Unresolvable conflict detected; phase transition blocked | +| `InvariantLoadError` | Database error loading invariants | +| `ReconciliationTimeoutError` | Reconciliation exceeded configured timeout | + +When `ReconciliationBlockedError` is raised: + +1. The phase transition method re-raises the exception to the caller +2. `PlanLifecycleService` emits `INVARIANT_VIOLATED` on the event bus +3. The plan remains in its current phase (e.g., `PENDING` before Strategize) +4. The operator must correct the conflicting invariants and retry the transition + +--- + +## DI Registration + +`InvariantService` is registered as a **Singleton** in the DI container: + +```python +# src/cleveragents/application/container.py (excerpt) +class Container(containers.DeclarativeContainer): + invariant_service = providers.Singleton( + InvariantService, + uow_factory=uow_factory, + event_bus=event_bus, + ) + invariant_reconciliation_actor = providers.Factory( + InvariantReconciliationActor, + invariant_service=invariant_service, + event_bus=event_bus, + audit_service=audit_service, + ) +``` + +--- + +## Usage Examples + +### Triggering Reconciliation Manually + +```python +from cleveragents.application.container import Container + +container = Container() +actor = container.invariant_reconciliation_actor() +result = await actor.reconcile(plan_id="plan-01JXYZ") + +if not result.success: + print(f"Reconciliation failed: {result.conflicts_detected} unresolvable conflicts") +else: + print(f"Reconciled {result.conflicts_resolved} conflicts") + for decision in result.decisions: + print(f" {decision.winner.text!r} won over {decision.loser.text!r}: {decision.rationale}") +``` + +### Listening for Reconciliation Events + +```python +from cleveragents.domain.events import INVARIANT_RECONCILED, INVARIANT_VIOLATED + +@event_bus.subscribe(INVARIANT_RECONCILED) +async def on_reconciled(event): + logger.info("Invariants reconciled", plan_id=event.plan_id, decisions=event.decision_count) + +@event_bus.subscribe(INVARIANT_VIOLATED) +async def on_violated(event): + logger.error("Invariant violation blocked phase transition", plan_id=event.plan_id) + # Notify operator, create alert, etc. +``` + +### Defining Invariants at Different Scopes + +```bash +# Global invariant (applies to all plans) +agents invariant create --scope global --text "Never delete production data without a backup" + +# Project-level invariant +agents invariant create --scope project --project my-project \ + --text "All database migrations must be reversible" + +# Plan-level invariant (highest precedence) +agents invariant create --scope plan --plan plan-01JXYZ \ + --text "This plan may drop the staging schema" +``` + +In this example, if the plan-level invariant conflicts with the project-level one, +the plan-level invariant wins (higher precedence). + +--- + +## Related Documentation + +- [ADR-016 Invariant System](../adr/ADR-016-invariant-system.md) — design rationale +- [ADR-007 Decision Tree & Correction](../adr/ADR-007-decision-tree-and-correction.md) — correction flow +- [Core API Reference](../api/core.md) — `InvariantService` API +- [Shell Safety Module](shell-safety.md) — another built-in safety actor diff --git a/mkdocs.yml b/mkdocs.yml index ca7740fea..af4eba30e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -23,6 +23,10 @@ nav: - Configuration: api/config.md - AI Providers: api/providers.md - TUI: api/tui.md + - Modules: + - Shell Safety: modules/shell-safety.md + - UKO Provenance Tracking: modules/uko-provenance.md + - Invariant Reconciliation: modules/invariant-reconciliation.md - Development: - CI/CD Pipeline: development/ci-cd.md - Quality Automation: development/quality-automation.md @@ -32,6 +36,7 @@ nav: - Ops Runbook: development/ops-runbook.md - System Watchdog: development/system-watchdog.md - Automation Tracking: development/automation-tracking.md + - Custom Sandbox Strategy: development/custom_sandbox_strategy.md - Implementation Timeline: timeline.md - FAQ: faq.md - Reference: reference/