forked from HAL9000/cleveragents-core
freemo
014033eed9
Add comprehensive automated health monitoring and recovery capabilities to the automation tracking system for proactive agent management. **Major Enhancements:** 1. **Standardized Interval Reporting** - Mandatory interval declaration in all tracking issues - Format: 'Reporting Interval: <interval> (Next report expected: <timestamp>)' - Enables precise staleness detection and recovery triggering 2. **Automated Health Monitoring (system-watchdog)** - New audit_automation_tracking_health() function runs every 5 minutes - Monitors all issues with 'Automation Tracking' label - Detects stalled agents when >20% overdue from expected interval - Calculates staleness ratios and time overdue metrics 3. **Automated Recovery System** - Kills stalled agent sessions via OpenCode Server API (port 4096) - Performs root cause analysis of session messages and agent definitions - Creates high-priority diagnostic issues with detailed findings - Automatically closes stale tracking issues with recovery notes - Provides human-readable remediation recommendations **Agent Updates with Standardized Format:** - **implementation-orchestrator**: Status updates (5 cycles) + health reports (10 cycles) - **backlog-groomer**: Grooming reports (5 min) + health reports (50 min) - **human-liaison**: Status updates (20 min monitoring cycles) - **session-persister**: Event-driven checkpoints with standardized format - **system-watchdog**: Enhanced with comprehensive recovery capabilities **Template Standardization:** - Unified header format across all tracking issues - Health indicators and next actions sections - Consistent metadata and automation signatures - Support for active/warning/error status indicators **Documentation Updates:** - Comprehensive automated recovery process documentation - Agent interval reference table with all timing details - Recovery issue format and diagnostic workflow - Health check algorithm and staleness threshold explanation **Benefits:** - Proactive detection of crashed or stuck agents (20% staleness threshold) - Automated recovery reduces manual intervention requirements - Root cause analysis provides actionable diagnostic information - Standardized format improves searchability and monitoring - Comprehensive health metrics enable system-wide visibility This enhancement transforms the automation tracking system from passive logging to active health monitoring with automated recovery capabilities.