ae940f45644971d5937b755d2459eceffeb6500a
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 drew
|
ea4a96aad6 |
feat(auto-agents): worktree hygiene — startup janitor + retry-on-failure
Live evidence on PR #29 (runs 15-18): 4 implementer worktrees accumulated in ``/tmp/cleveragents-implementer-worktrees/``, one with a corrupted ``.git`` link (``fatal: not a git repository``). Root cause: every SIGTERM-induced dispatcher restart leaves the in-flight cycle's worktree orphaned, and the mirror's ``worktrees/<name>/`` bookkeeping survives without the directory. The next cycle's ``git worktree add`` against the same mirror can then fail with "already exists" or unhelpful path collisions. Two fixes that together close the loop: 1. **Startup janitor** (``_pr_clone.prune_orphan_worktrees``): scans the per-kind worktree base on dispatcher startup and removes any dir matching the canonical ``pr-{N}-{kind}-{hex-tag}`` shape that is EITHER older than the OpenCode worker ceiling (default 30 min — longer than any possible in-flight cycle) OR has a missing / zero-byte ``.git`` link (definitionally corrupted). Removes the dir AND the mirror's ``worktree`` bookkeeping. Idempotent. Skips operator scratch dirs that don't match the canonical name. Disable via ``DISPATCHER_WORKTREE_JANITOR_DISABLE=1``. Called once at the top of both ``dispatch_review.main`` and ``dispatch_implementer.main``. 2. **Retry-on-failure** in ``prepare_pr_worktree``: when ``git worktree add`` fails the first time, run ``git worktree prune`` to clear stale mirror bookkeeping, force- remove the target path if present, and retry exactly once. This rescues cycles whose janitor-min-age cushion missed a fresh orphan from a very-recent SIGTERM. Coverage: 13 new tests in ``test_pr_clone_janitor.py`` (recent vs stale removal, corruption detection regardless of age, non-canonical name safety, idempotency, disable env, ``git worktree remove`` call count). Full auto_agents suite: 2059 passing (+13 vs prior commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
8ed4b96b1a |
feat(auto-agents): perf + observability + persistent comment cache
Folds B1-B4 + C2-C5 from the post-live-test plan into one commit:
B1 — npx tsx pre-warm in dispatchers-launcher.sh closes the cold-cache
30s AbortSignal timeout that killed both dispatchers' first cycle.
B2 — per-tier worker timeout
(IMPLEMENTER_DISPATCHER_WORKER_TIMEOUT_TIER_{N}_SECONDS) lets Tier 1
(qwen-large) and Tier 2 (kimi) get more wallclock than gpt-5-mini;
floor 60s.
B3 — _rebuild_prompt_from_cached_result skips the full prefetch on
tier transitions (worktree-reset puts everything back at the
prefetched head_sha, so the prefetch result + det_sections don't
change). Saves ~7 min per tier transition on comment-heavy PRs.
B4 — git-commit-util.md documents the FORBIDDEN naive recovery
pattern (git fetch && git reset --hard) that lost PR #30 attempt
3's real fix in the live test. Two correct paths now spelled out:
--force-with-lease=<branch>:<old-remote-sha> or stash+rebase+pop.
C2 — _pr_clone._refresh_mirror_with_retry adds one retry on git
fetch failure and force-reclones the bare mirror if both attempts
fail. Previously a single exit 128 logged WARN and continued with
stale data forever.
C3 — in-flight turn markers (asterisk suffix on input/output token
counts) in the per-turn log when completed=False. The archived
turn dict's completed field was already there; the log now surfaces
it. Sub-agent timeout archiving was already correct via
_archive_subagent_tree.
C4 — new module _recent_push_cache.py records per-PR push events
(head_sha + timestamp + cycle metadata). Prefetch surfaces in the
sentinel under recent_implementer_push (with --field accessor)
when the cached push matches the PR's current head_sha within
1h. Prevents the "dispatcher re-cycles right after pushing,
worker re-does the same compliance work" failure mode from
PR #28 cycle 2 in the live test.
C5 (replaces C1) — new module _pr_comments_cache.py wraps
_review_fetch.fetch_pr_comments with disk-backed delta-fetch
semantics. PR #30's 1340+ comment fetch (which previously took
~30s and hit the 20-page pagination cap) now becomes a 5-10 item
delta. Cache is per-PR, shared between reviewer + implementer
dispatchers, has 24h staleness bound, kill-switch via
IMPLEMENTER_DISPATCHER_COMMENT_CACHE_DISABLE=1.
Tests: 1484 passed, 3 skipped (+20 from
|
||
|
drew
|
49a28b5cf5 |
feat(auto-agents): deterministic compliance short-circuit + push-substrate fixes
Three blocker classes surfaced by the 2026-05-13 4-hour live pipeline
test (PR #30 / #28 / #25), all fixed here:
1. `_pr_clone` left `remote.origin.mirror=true` on the bare mirror,
making every `git push --force-with-lease origin <refspec>` fail
with `fatal: --mirror can't be combined with refspecs`. Worktrees
inherited the bad config; worker had to discover + work around it
each cycle, and one cycle's fragile recovery LOST a real commit.
`_disable_mirror_push_semantics` clears the flag at clone time
and on every refresh.
2. Predicate treated `outcome=resolved + head_sha_advanced=False` as
transport-class (UNKNOWN bucket → wasted same-tier retry). It's
actually competence — the worker emitted a complete-looking JSON
while delivering nothing. Now escalates immediately.
3. The dispatcher's own deterministic sections (compliance_gaps +
gate_preflight) misled the worker into emitting `resolved`
whenever both were clean — regardless of real remote CI state.
New module `_implementer_compliance_apply` + dispatcher hook
`_maybe_short_circuit` move trivial compliance fixes (CONTRIBUTORS
line, CHANGELOG stub from PR title, ISSUES CLOSED footer from
prefetched linked_issues) onto the dispatcher's deterministic
side per the auto-agents.md policy: "deterministic Python owns
orchestration; the LLM only handles the actual creative work."
Two short-circuit paths:
- P0 (always-on): compliance clean + preflight clean + remote CI =
success → skip LLM, emit `no_changes_needed`. Eliminates the
hallucination class observed live.
- A (opt-in via IMPLEMENTER_DISPATCHER_AUTO_FIX_COMPLIANCE=1):
compliance has only fixable gaps + preflight clean + CI failing →
dispatcher applies fixes, pushes, emits `resolved`. Worker
reserved for code-bug PRs only.
Plus supporting changes:
- gate_preflight payload includes `remote_ci_state` + `diverges_from_remote_ci`
- Telemetry rows gain `outcome_disputed=True` when resolved+no-push
- `EscalationAction.SUCCESS` accepts `no_changes_needed` outcome
- task-implementor.md procedure re-ordered: read `--field ci` FIRST
- Runtime hook `_dispatch_runtime._maybe_read_short_circuit` consumes
the dispatcher's `_short_circuit_result` stash and synthesizes a
SessionResult instead of spawning the LLM session
- Cycle archive's post_session_result gains `auto_fix_report` +
`short_circuit` fields
Tier B (perf) and Tier C (observability) deferred to follow-up.
Tests: 1464 passed, 3 skipped (+23 from
|
||
|
drew
|
0b657cd0d9 |
fix(auto-agents): three-case contract, work_type dispatch, hardening for filesystem handoff
Post-commit review of
|
||
|
drew
|
d386ff4e8b |
feat(auto-agents): filesystem handoff for implementer pre-clone + pre-fetch
The 2026-05-10 default-ON flip of IMPLEMENTER_DISPATCHER_PREFETCH / IMPLEMENTER_DISPATCHER_PRECLONE put rich PR context and a pre-cloned worktree in the wrapper's prompt — but the deep `task` tool chain (implementation-worker → tier-dispatcher → tier-qwen-med → task-implementor) re-summarises the prompt at every level, so by the time task-implementor sees its input only BEGIN_PR_DIFF survives. The worker still called git-isolator-util (~82 s wasted) and re-issued Forgejo GETs for data the dispatcher had already fetched. This change introduces a filesystem-mediated handshake that is immune to prompt summarisation. The dispatcher writes two sentinel JSON files per cycle (workspace handoff next to the worktree; PR-context handoff in /tmp/cleveragents-implementer-handoff/) and the worker reads them via two new bash-allowed Python scripts. Missing / malformed / stale sentinels map to empty stdout, which is the worker's signal to fall through to the legacy GET / git-isolator-util. New modules: - tools/_pr_context_sentinel.py: dispatcher-side writer (atomic, with 200 KB per-field truncation and idempotent delete). - tools/implementer_workspace.py: worker-side reader CLI with `discover` and safety-checked `cleanup` subcommands. - tools/implementer_pr_context.py: worker-side reader CLI with `read --field <name>` for every prefetch field. Hooked into: - tools/_pr_clone.py: prepare_pr_worktree writes the workspace sentinel after worktree-add succeeds; WorktreeHandle.cleanup removes both worktree and sentinel; new _resolve_branch_for_sha populates the sentinel's branch field. - tools/dispatch_implementer.py: _prefetch_prompt writes the PR-context sentinel; _cleanup_clone_handle removes it. Two new skills (.opencode/skills/implementer-workspace, .opencode/skills/implementer-pr-context) and a rewrite of every "Pre-fetched section" wording in .opencode/agents/task-implementor.md to call the skills first and fall back to legacy GET only on empty stdout. Tests: 54 new (16 workspace CLI + 21 PR-context CLI + 7 sentinel writer + 10 _pr_clone integration). Full auto_agents suite: 1,123 passed, 3 skipped (was 1,069 before). Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
drew
|
00cc24acb5 |
fix(auto-agents): _pr_clone preclone gate also default-ON (Phase 4 follow-up)
The initial Phase 4 commit (
|
||
|
drew
|
6685c8e9a4 |
refactor(auto-agents): Phase 0.1 + 0.2 critique fold-in
Two consecutive critique rounds (architect / principal dev / test
engineer) of the Phase 0 commit and its first follow-on. End-state
fixes ride together because intermediate Phase 0.1 staging was never
committed.
Architecture
- Generalise the four shared env knobs to canonical DISPATCHER_*
names with one-shot deprecation warnings on REVIEW_DISPATCHER_*
fallbacks (back-compat preserved).
- Add IMPLEMENTER_DISPATCHER_PRECLONE Phase-3 feature flag with
explicit kill-switch precedence.
- Consolidate _kind_cfg lookups in prepare_pr_worktree end-to-end:
every consumer (gate predicates + _worktree_base) takes a
pre-resolved cfg via *_with_cfg twins, so a typo'd kind logs the
fall-through error exactly once per call. The local kind is also
normalised to "review" so path filenames and WorktreeHandle.kind
reflect the effective fall-back (no partial internal state).
- Raise _kind_cfg fall-through log from WARNING to ERROR.
- Rename _review_clone_creds.py to _pr_clone_creds.py.
- Type _KIND_CONFIG as TypedDict so typo'd keys are caught
statically.
Code hygiene
- Wire WorktreeHandle.kind into cleanup logging.
- Per-error-path warnings in commit_from_worktree (timeout / OSError
/ non-zero exit / empty stdout / sentinel parse failure).
- Switch emit_error.stream from Any to IO[str] | None.
- wrap_untrusted_section now filters None values from attrs.
- Worktree paths grow a kind segment: pr-{n}-{kind}-{tag}.
Tests
- 40 new tests in test_shared_substrate.py (54 total, up from 14).
- Parametrised env precedence + one-shot deprecation over all four
shared knobs.
- Direct unit test for _worktree_base_with_cfg with a hand-built
_KindConfig literal so future field additions fail loudly.
- Surface check covers Phase 0.1/0.2 callable additions; module-
private data structures intentionally excluded.
- Restructured the unknown-kind test to take the full success path
with explicit assertions on handle.kind, handle.path, and the
exactly-once ERROR fall-through, defending the consolidation
invariant + the full-fallback contract.
- Autouse fixture isolates _LEGACY_DEPRECATION_LOGGED per-test.
- Documented the substrate load-order trick in conftest.
No reviewer behaviour changes. IMPLEMENTER_DISPATCHER_PRECLONE and
kind="implementer" are forward-looking scaffolding only --
dispatch_implementer.py does not call prepare_pr_worktree yet
(Phase 3 wiring lands separately).
All 677 auto-agents tests pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
drew
|
f3b10a5e72 |
refactor(auto-agents): rename review clone/diff substrate for implementer parity
Phase 0 of the implementer-parity plan: rename the formerly review-only
pre-clone + diff-fetch helpers so the implementer dispatcher can share
them in upcoming phases without forking the substrate or hauling
review-specific naming into a different work-group registry.
Renames (git rename-detected):
- tools/_review_clone.py -> tools/_pr_clone.py
- tools/_review_diff.py -> tools/_pr_diff.py
Extractions (new shared modules):
- tools/_pr_prompt.py UNTRUSTED-CONTENT marker helpers
(fence_markers, redact_marker,
wrap_untrusted_section) for Phase 2
pre-fetch consumers.
- tools/_commit_lint.py lint_commit_message + CONVENTIONAL_TYPES
+ _bot_committer_email so both
self-validation CLIs reuse one lint.
- tools/_validate_cli_common.py DiffResult / DiffErrorKind /
diff_from_worktree /
commit_from_worktree / _excerpt_*
/ resolve_base_ref / emit_error so
implementer_validate (Phase 1) does
not duplicate ~280 lines of CLI
plumbing.
Slim:
- tools/_review_validate_helpers.py shrinks 499 -> 223 lines and now
owns only review-specific validate_position_in_diff +
draft_strict_checks. Re-exports the moved helpers so existing test
monkeypatches (helpers.diff_from_worktree, helpers.subprocess) keep
working without churn.
API surface change:
- prepare_pr_worktree(cfg, n, sha, *, kind="review") -- back-compat
default; implementer dispatcher passes kind="implementer" in Phase 3.
- _worktree_base / _is_preclone_disabled now consult per-kind env
vars (REVIEW_DISPATCHER_WORKTREE_BASE vs.
IMPLEMENTER_DISPATCHER_WORKTREE_BASE; matching DISABLE_PRECLONE
toggles). Mirror is shared per repo regardless of kind.
- WorktreeHandle gains a `kind` field (defaulted) so cleanup paths
can discriminate.
Bug fix discovered along the way:
- _validate_cli_common.emit_error froze stream=sys.stdout at
function-definition time, which made pytest's capsys invisible to
the JSON error output. Now resolves sys.stdout at call time.
Tests:
- New tests/auto_agents/test_shared_substrate.py (14 tests) covers:
every public symbol the legacy modules exposed, helper-re-export
identity preservation, per-kind worktree-base + disable-toggle
semantics, the kind back-compat default, and the new _pr_prompt
fence/redact helpers. Also guards against the deleted
_review_clone.py / _review_diff.py reappearing on disk.
- All 24 importing call-sites updated; full reviewer test suite
remains green (637 passed, 3 skipped).
- `git grep '_review_clone\|_review_diff' tools/` returns zero
matches, the plan's Phase 0 exit criterion.
Sets up Phase 1 (implementer-helpers skill), Phase 2 (pre-fetch
parity), and Phase 3 (pre-cloned implementer worktrees) with no
shared reviewer-only code paths.
ISSUES CLOSED: #N/A
Co-authored-by: Cursor <cursoragent@cursor.com>
|