feat(auto-agents): PR State Warmer substrate + supporting infra
Adds a long-lived sidecar (pr_state_warmer.py) that polls Forgejo's
/pulls endpoint every 30s and writes the full PR snapshot to a
shared SQLite store, eliminating the dispatcher's per-cycle
cold-cache stalls (24-30s rebuilds on flaky cycles) and the silent
50-PR pagination cap on the legacy single-page fetch.
Substrate
- tools/_pr_state_cache.py — SQLite store with (owner, repo) PK,
WAL mode, additive v2→v3 migration (comments_refreshed_updated_at),
bounded fcntl.flock migration lock, threading.Lock for per-process
init, @_with_reheal decorator (catches OperationalError no-such-
table + DatabaseError corruption with file quarantine), atomic
TEMP-table chunking for >32k seen-set, _normalize_updated_at to
canonicalize Forgejo tz-marker drift
- tools/pr_state_warmer.py — poll/upsert/vanish/comments-refresh
loop with fcntl.flock singleton (rejects second warmer), bounded
comments-refresh cap, persistent deferral via SQL pending query,
PermissionError-tolerant lock setup, cold-start log suppression
- tools/_pr_classification_cache.py — three-layer fall-through
(warmer cache → list cache → live fetch) with staleness gate
(PR_STATE_WARMER_STALE_AFTER_S floored at 30s in prod)
Comments cache hardening
- Bot-filter at write time drops bot status/claim/release/sentinel
while preserving **Implementation Attempt** markers (94.6%
reduction on bot-heavy PRs like #30's 19k-comment thread)
- _normalize_since_cursor strips microsecond precision before
building ?since= query (fixes the live-observed Forgejo HTTP 422
bug on PRs #25 + #28); handles uppercase Z, lowercase z, ±HH:MM
offsets (including non-zero like +05:30), naive ISO
- Lazy migration of legacy null-key by_author entries on _read_cache
- _newest_cursor walks tail-back skipping malformed entries
Supporting infrastructure (cumulative dmpipeline-v2 work)
- Telemetry server: SSE live tail, run-sessions enumeration,
cost/token tracking, app.js UI rewrite with collapsible sections
- MCP servers (mcp_ci_server, mcp_forgejo_server, mcp_git_server,
mcp_handoff_server, mcp_graphify_server) for opencode worker
context access
- Live log writer (tools/live_log_writer.py) — SSE-streaming
dispatcher event log
- Tier-dispatcher escalation flow with prompts trimmed for budget
- Shared bot-logins resolver (tools/_bot_logins.py) replacing two
drift-prone copies
- token_usage_audit.py for opencode cost analysis
Tests
- 2259 passing across 65 changed/new files
- New suites: test_pr_state_cache, test_pr_state_warmer,
test_pr_state_warmer_integration, test_pr_classification_cache,
test_pr_list_cache_backoff, test_mcp_* (5 servers),
test_live_log_writer_sse, test_telemetry_run_sessions,
test_review_post_ready_label
- Test_pr_comments_cache expanded with bot-filter coverage,
cursor-normalization regression pins, format-drift, atomicity,
failed-comments-not-stamped (silent-data-loss class)
- Parametrized @_with_reheal coverage across 7 wrapped APIs
- Real fault-inject atomicity test for chunked mark_vanished path
via Connection wrapper class
- Subprocess-based singleton flock test (cross-process contract)
- Event-driven SIGTERM-mid-poll test (no fixed-sleep flake)
Architecture notes
- Schema v3 migration is additive (ALTER ADD COLUMN); v0/v1 still
need destructive rebuild because pre-v2 column shape lacks
owner/repo. Cross-process drop-table-ping-pong prevented by the
fcntl migration lock + per-process _initialized flag.
- Comments-refresh deferral is persistent via
comments_refreshed_updated_at column — survives warmer restart,
picks up next cycle even if PR didn't change again. Replaces
in-memory changed_numbers list.
- Rollback path: PR_STATE_WARMER_PREFER=0 bypasses the warmer
cache and reverts to live-fetch behavior. PR_STATE_CACHE_DISABLE=1
short-circuits the warmer process at startup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
drew
2658deee94