a21466add232d59cdec1604e09d58ca05659a623
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 drew
|
0bc734c020 |
style: ruff format the controller-state-machine branch (288 files)
Applies `ruff format` to the accumulated formatting debt on this branch. Formatting-only — no behavioral changes. Required for CI/lint's format gate (`nox -s format -- --check`), which the branch was failing on 288 tracked files that drifted from ruff's canonical style. In-progress WIP files are intentionally excluded so this commit stays a clean formatting-only diff. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
drew
|
b80f5f7c3a |
fix(auto-agents): reviewer data-handling — five live-observed regressions
Five independent bug fixes around the reviewer's data pipeline, all live-observed on 2026-05-17 run-1 (PR #35 / PR-shaped traffic generally). Bundled because they share the same data-correctness intent and one fixture-update covers two of them. 1) **PR diff: distinguish "fetch failed" from "PR has no changes."** ``tools/_pr_diff.py``. The old code conflated both cases into ``unavailable=True``, which forced ``data_complete=False`` and blocked APPROVED verdicts. Live-observed on PR #35 created by the new_issue worker without any code changes — Forgejo returns HTTP 200 + empty body for a head==base PR, and the reviewer was incorrectly told the diff was unavailable. 2) **PR-state cache: refresh body + labels on the unchanged- ``updated_at`` branch.** ``tools/_pr_state_cache.py``. Forgejo label add/remove mutations do NOT bump ``updated_at``, so the warmer's cached PR object would carry stale labels for as long as the PR sat idle. Downstream consumers (cycle-cap, claim sweeps, filter exclusions) would never see them. Cheap fix — same row, two extra columns refreshed. 3) **Comments cache: URL-encode the ``since=`` cursor.** ``tools/_pr_comments_cache.py`` + matching test update. The ``+`` in ``+00:00`` decodes to a space on Forgejo's query- string parser, producing 422 errors. Live-observed on PR #35 run-1: 6 consecutive 422s on the same clean ``+00:00`` cursor before the cache backed off for 30 min. ``urllib.parse.quote`` with ``safe=''`` quotes every non-alphanumeric so ``+`` → ``%2B``, ``:`` → ``%3A``. Test updated to ``unquote`` the captured path before substring-matching. 4) **Reviewer prompt: surface the clone fallback.** ``tools/_review_prompt.py``. Adds an inline note in the pre-fetched diff section explaining the two diff sources (inline-truncated vs pre-cloned worktree) and the ``REVIEW_DISPATCHER_DIFF_MAX_BYTES`` cap. Closes a reviewer- side confusion where the model didn't know it could read source files from disk when the inline diff was truncated. 5) **Reviewer agent contract: truncated diff + clone IS data-complete.** ``.opencode/agents/pr-review-worker.md``. The prior wording said ``truncated=True`` forced ``data_complete= False`` and blocked APPROVED — but with the pre-cloned worktree available, the reviewer DOES have full code access and APPROVED should remain valid. Updated guidance now distinguishes "truncated but clone present" (APPROVED OK) from "truncated AND no clone" (COMMENT / REQUEST_CHANGES only). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
2658deee94 |
feat(auto-agents): PR State Warmer substrate + supporting infra
Adds a long-lived sidecar (pr_state_warmer.py) that polls Forgejo's /pulls endpoint every 30s and writes the full PR snapshot to a shared SQLite store, eliminating the dispatcher's per-cycle cold-cache stalls (24-30s rebuilds on flaky cycles) and the silent 50-PR pagination cap on the legacy single-page fetch. Substrate - tools/_pr_state_cache.py — SQLite store with (owner, repo) PK, WAL mode, additive v2→v3 migration (comments_refreshed_updated_at), bounded fcntl.flock migration lock, threading.Lock for per-process init, @_with_reheal decorator (catches OperationalError no-such- table + DatabaseError corruption with file quarantine), atomic TEMP-table chunking for >32k seen-set, _normalize_updated_at to canonicalize Forgejo tz-marker drift - tools/pr_state_warmer.py — poll/upsert/vanish/comments-refresh loop with fcntl.flock singleton (rejects second warmer), bounded comments-refresh cap, persistent deferral via SQL pending query, PermissionError-tolerant lock setup, cold-start log suppression - tools/_pr_classification_cache.py — three-layer fall-through (warmer cache → list cache → live fetch) with staleness gate (PR_STATE_WARMER_STALE_AFTER_S floored at 30s in prod) Comments cache hardening - Bot-filter at write time drops bot status/claim/release/sentinel while preserving **Implementation Attempt** markers (94.6% reduction on bot-heavy PRs like #30's 19k-comment thread) - _normalize_since_cursor strips microsecond precision before building ?since= query (fixes the live-observed Forgejo HTTP 422 bug on PRs #25 + #28); handles uppercase Z, lowercase z, ±HH:MM offsets (including non-zero like +05:30), naive ISO - Lazy migration of legacy null-key by_author entries on _read_cache - _newest_cursor walks tail-back skipping malformed entries Supporting infrastructure (cumulative dmpipeline-v2 work) - Telemetry server: SSE live tail, run-sessions enumeration, cost/token tracking, app.js UI rewrite with collapsible sections - MCP servers (mcp_ci_server, mcp_forgejo_server, mcp_git_server, mcp_handoff_server, mcp_graphify_server) for opencode worker context access - Live log writer (tools/live_log_writer.py) — SSE-streaming dispatcher event log - Tier-dispatcher escalation flow with prompts trimmed for budget - Shared bot-logins resolver (tools/_bot_logins.py) replacing two drift-prone copies - token_usage_audit.py for opencode cost analysis Tests - 2259 passing across 65 changed/new files - New suites: test_pr_state_cache, test_pr_state_warmer, test_pr_state_warmer_integration, test_pr_classification_cache, test_pr_list_cache_backoff, test_mcp_* (5 servers), test_live_log_writer_sse, test_telemetry_run_sessions, test_review_post_ready_label - Test_pr_comments_cache expanded with bot-filter coverage, cursor-normalization regression pins, format-drift, atomicity, failed-comments-not-stamped (silent-data-loss class) - Parametrized @_with_reheal coverage across 7 wrapped APIs - Real fault-inject atomicity test for chunked mark_vanished path via Connection wrapper class - Subprocess-based singleton flock test (cross-process contract) - Event-driven SIGTERM-mid-poll test (no fixed-sleep flake) Architecture notes - Schema v3 migration is additive (ALTER ADD COLUMN); v0/v1 still need destructive rebuild because pre-v2 column shape lacks owner/repo. Cross-process drop-table-ping-pong prevented by the fcntl migration lock + per-process _initialized flag. - Comments-refresh deferral is persistent via comments_refreshed_updated_at column — survives warmer restart, picks up next cycle even if PR didn't change again. Replaces in-memory changed_numbers list. - Rollback path: PR_STATE_WARMER_PREFER=0 bypasses the warmer cache and reverts to live-fetch behavior. PR_STATE_CACHE_DISABLE=1 short-circuits the warmer process at startup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
dc4e9368f2 |
feat(auto-agents): block-store substrate + DRY refactors
Externalise large prompt sections (PR diff, comments, CI failure
logs, reviews, linked issues) into a cross-process SQLite-backed
block store so the worker can recover original content when an
intermediate tier-* agent's summarisation strips inline sections.
New substrate
-------------
- ``tools/_block_store.py`` — SQLite WAL, per-row 1MB cap, 1h TTL,
janitor (startup + opportunistic per-hour in register()),
threading.Lock around the periodic-sweep gate.
- ``tools/_block_prompt.py`` — registration glue + ``## Available
blocks`` Markdown table renderer.
- ``tools/_prefetch_section.py`` — single-source-of-truth section
registry; collapses the duplicate ``_register_*_blocks`` helpers
the reviewer and implementer previously kept in lockstep.
- ``tools/mcp_block_store_server.py`` — FastMCP wrapper exposing
``block_fetch`` / ``block_list`` / ``block_register`` /
``block_invalidate`` to agents. Uses the expanded ``_mcp_common``
helpers.
- ``tools/_implementer_escalation_helpers.py`` — pure helpers
extracted from ``dispatch_implementer.py`` (~180 lines off the
3284-line file); takes ``claim_runtime`` as a kwarg for clean DI.
DRY refactors
-------------
- ``tools/_backoff.py`` — shared ``Backoff`` dataclass collapses the
three near-identical exponential-backoff state machines in
``_pr_comments_cache``, ``_ci_logs``, ``_pr_classification_cache``.
- ``tools/_mcp_common.py`` — expanded with ``bootstrap_loader``,
``error_envelope``, ``make_main`` so each MCP server's prelude is
three lines.
- ``tools/_pr_diff.build_diff_section_full`` — returns a 4-tuple
including the raw diff body so the reviewer's block-store
registration reuses the bytes instead of doing a second HTTP fetch.
Wiring
------
- ``_review_prompt.build_review_prompt`` builds a ``PrefetchSection``
registry via ``_review_sections``, registers them, and renders the
``## Available blocks`` table at the end of the prompt.
- ``_implementer_prefetch._fetch_pr_context`` /
``fetch_new_issue_context`` build the equivalent registry via
``_implementer_sections`` and stamp ``result.block_refs`` for the
prompt builder to read.
- ``_implementer_prompt`` builders include
``_build_available_blocks_section(result)`` in all three flows.
- ``dispatch_review.main`` + ``dispatch_implementer.main`` call
``_block_store.janitor()`` at startup; the per-call opportunistic
janitor in ``register()`` keeps the file bounded between restarts.
Agent contract updates
----------------------
- ``.opencode/agents/task-implementor.md`` +
``.opencode/agents/pr-review-worker.md``:
- ``block_store*`` permission
- new "Block-store substrate" paragraph explaining
``block_fetch`` / ``block_list`` as the summarisation recovery
path.
Tests
-----
- ``test_block_store.py`` — 48 tests pinning every public contract
(register/fetch/list/invalidate/janitor, key whitelist, TTL,
size cap, WAL durability).
- ``test_block_prompt`` — covered transitively via the e2e test.
- ``test_block_store_recovery_e2e.py`` — builds a real prompt via
``build_pr_fix_prompt``, applies a heading-bounded summariser stub
(``_summarise_inline_sections``), asserts inline content is stripped
yet block keys survive and ``block_fetch`` recovers original content.
Plus a ``block_list`` fallback test for the worst case where the
table itself was summarised away.
- ``test_mcp_block_store_server.py`` — 27 wrapper-contract tests.
- ``test_mcp_block_store_transport.py`` — spawns the actual server
subprocess via ``mcp.client.stdio`` and exercises the JSON-RPC
transport round-trip in ~1.5s. Catches FastMCP schema /
serialisation bugs the in-process tests miss.
- ``test_backoff.py`` — 14 behaviour-focused tests of the shared
``Backoff`` curve.
- ``test_pr_comments_cache.py`` + ``test_ci_logs.py`` — deleted the
now-redundant ``TestComputeNextAttemptAfter`` / ``TestBackoffActive``
/ ``TestBackoffHelpers`` classes; ``test_backoff`` covers them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
fedd99b9d4 |
fix(auto-agents): partial-delta branch persists merged view (not original cached)
On a partial live-delta the failing branch returned ``cached + delta`` merged to the caller, but persisted ONLY the original cached entries back to disk. The next cycle's backoff short-circuit then served the older, smaller view — a regression vs the failing branch's view. Live-test on PR #29 run-19 caught this: failing branch returned 144 comments, but the next short-circuit would have served 13. Persisting the merged view is safe because ``_merge_comments`` dedups by id — the write is strictly additive. ``any_partial_fetch`` stays True so the next clean delta still backfills toward completeness, and ``since_cursor`` advances to the newest merged entry so the next ``?since=`` picks up forward (not where we started). Coverage: 1 new test (``test_partial_delta_persists_merged_so_next_short_circuit_matches``) pins the contract. 28 cache tests passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
aa7c24fa18 |
feat(auto-agents): resilient comment cache — backoff + reviewer cutover
Two changes that together stop PR #29's per-cycle ``max_pages=20`` truncation WARN on its 2700+ comment history: 1. **Exponential backoff** in ``_pr_comments_cache.get_pr_comments``: on a failed live delta the cache stamps ``consecutive_failures`` and ``next_attempt_after`` (default base 60s × 2^(failures-1), capped at 30 min). The next cycle inside the backoff window short-circuits — serves the cached bulk stale with ``completed=False`` and does NOT hit the flaky endpoint again. First successful delta clears the counter, so a transient outage doesn't permanently throttle. Truncated cold seeds (page-cap hit on first walk) are also counted as failures so the every-cycle 30s pagination tax stops on PR-sized threads that genuinely exceed the cap. 2. **Reviewer prefetch cutover** in ``_review_prompt.py``: the reviewer's ``fetch_review_context`` now routes through ``_pr_comments_cache.get_pr_comments`` instead of the raw ``_review_pipeline.fetch_pr_comments``. Mirrors the Phase 2 feature-flag pattern: default ON, env off-switch ``REVIEW_DISPATCHER_USE_COMMENT_CACHE=0`` for rollback, WARN and fall back to the legacy paginator on any cache exception so a cache failure can never break a review cycle. Implementer dispatcher has used this cache directly since 2026-05-13 without incident. Coverage: 30 new tests (14 backoff + 16 cutover-wrapper). Touched-module suite: 43 passing. NOTE: the matching MCP wrapper (``forgejo_fetch_pr_comments_cached``) lives in ``tools/mcp_forgejo_server.py`` which is currently untracked; it'll ride with the Phase 1 commit that lands the MCP server file itself. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
19dad571bd |
feat(auto-agents): bounded comment view + digest, persistent cache, claim-sweep through cache
Three coordinated changes addressing run-8 finding R8-3 ("PR #30's
1440 comments poison the pipeline via prompt size × chain depth"):
Spec #5 — bounded view + deterministic digest. The prompt and
sentinel now embed at most DEFAULT_MAX_PROMPT_COMMENTS=50 verbatim
comments plus a one-paragraph rollup of the older bot attempt
comments (counts by tier / outcome / failing gates / last success).
_build_comments_section takes the most-recent N (comments[-N:]) not
the oldest — a long-standing bug where the worker on heavy PRs saw
ancient history and missed every recent attempt. Bot status / claim
/ sentinel comments are dropped from the view via author-based
classification (HAL9000 / HAL9001 defaults, FORGEJO_USERNAME /
FORGEJO_REVIEWER_USERNAME env overrides) — a content-only
classifier mis-counted them as "humans" and ballooned the view to
1252 items on the real test case (run-10 inspection).
Spec #5 (R8-5) — persistent comment cache fixes. Seed-on-truncation:
a page-cap-truncated fetch now seeds the cache (clipped but valid)
flagged any_partial_fetch=True. since_cursor replaces wall-clock
fetched_at as the ?since= delta cursor so backfill walks forward
from the newest cached comment instead of skipping the un-fetched
middle. _api_get_paginated gains an opt-in return_truncation=True
shape so the cache can distinguish "transient failure" (don't seed)
from "page cap hit" (seed and backfill next cycle).
Spec #6 — claim-sweep routes through the comment cache.
_claim_runtime._find_newest_claim_at used to paginate every page of
issue comments on every cycle (29 sequential round-trips for #30,
~10+ minutes when Forgejo was slow — see run-9 hang diagnosis). It
now reads from _pr_comments_cache.get_pr_comments and reverse-scans
for the marker with early-exit. get_pr_comments grew optional
owner/repo overrides so callers with a narrower RuntimeContext cfg
(no owner/repo attrs) can share the cache. Fail-safe on
completed=False: when the timeline is incomplete and no marker was
found, return datetime.now() so the sweep keeps the claim this
cycle rather than releasing on partial data.
Run-11 verification (PR #30 end-to-end):
- Dispatcher startup -> first cycle log: 12+ min hang -> 9 s
- pr_comments view len in sentinel: 1252 (run-10) -> 50 (run-11)
- pr_comments_digest populated with full tier/outcome/gates rollup
- data_complete=True; 4 implementer sessions ran cleanly
Tests green: 1603 passed / 3 skipped. New test files:
test_attempt_history.py, test_implementer_prefetch.py. New tests
added in test_pr_comments_cache.py, test_claim_runtime.py,
test_implementer_pr_context_cli.py, test_implementer_prompt_snapshot.py,
test_pr_context_sentinel.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
8ed4b96b1a |
feat(auto-agents): perf + observability + persistent comment cache
Folds B1-B4 + C2-C5 from the post-live-test plan into one commit:
B1 — npx tsx pre-warm in dispatchers-launcher.sh closes the cold-cache
30s AbortSignal timeout that killed both dispatchers' first cycle.
B2 — per-tier worker timeout
(IMPLEMENTER_DISPATCHER_WORKER_TIMEOUT_TIER_{N}_SECONDS) lets Tier 1
(qwen-large) and Tier 2 (kimi) get more wallclock than gpt-5-mini;
floor 60s.
B3 — _rebuild_prompt_from_cached_result skips the full prefetch on
tier transitions (worktree-reset puts everything back at the
prefetched head_sha, so the prefetch result + det_sections don't
change). Saves ~7 min per tier transition on comment-heavy PRs.
B4 — git-commit-util.md documents the FORBIDDEN naive recovery
pattern (git fetch && git reset --hard) that lost PR #30 attempt
3's real fix in the live test. Two correct paths now spelled out:
--force-with-lease=<branch>:<old-remote-sha> or stash+rebase+pop.
C2 — _pr_clone._refresh_mirror_with_retry adds one retry on git
fetch failure and force-reclones the bare mirror if both attempts
fail. Previously a single exit 128 logged WARN and continued with
stale data forever.
C3 — in-flight turn markers (asterisk suffix on input/output token
counts) in the per-turn log when completed=False. The archived
turn dict's completed field was already there; the log now surfaces
it. Sub-agent timeout archiving was already correct via
_archive_subagent_tree.
C4 — new module _recent_push_cache.py records per-PR push events
(head_sha + timestamp + cycle metadata). Prefetch surfaces in the
sentinel under recent_implementer_push (with --field accessor)
when the cached push matches the PR's current head_sha within
1h. Prevents the "dispatcher re-cycles right after pushing,
worker re-does the same compliance work" failure mode from
PR #28 cycle 2 in the live test.
C5 (replaces C1) — new module _pr_comments_cache.py wraps
_review_fetch.fetch_pr_comments with disk-backed delta-fetch
semantics. PR #30's 1340+ comment fetch (which previously took
~30s and hit the 20-page pagination cap) now becomes a 5-10 item
delta. Cache is per-PR, shared between reviewer + implementer
dispatchers, has 24h staleness bound, kill-switch via
IMPLEMENTER_DISPATCHER_COMMENT_CACHE_DISABLE=1.
Tests: 1484 passed, 3 skipped (+20 from
|