a21466add232d59cdec1604e09d58ca05659a623
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 drew
|
0bc734c020 |
style: ruff format the controller-state-machine branch (288 files)
Applies `ruff format` to the accumulated formatting debt on this branch. Formatting-only — no behavioral changes. Required for CI/lint's format gate (`nox -s format -- --check`), which the branch was failing on 288 tracked files that drifted from ruff's canonical style. In-progress WIP files are intentionally excluded so this commit stays a clean formatting-only diff. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
drew
|
8ef2f949c2 |
fix(auto-agents): R3.5 — cap parity + attempt-label conclusive-outcome gate
Two audit findings from the post-R3.4 walk-through, both addressing gaps where one pool's stop-signal didn't propagate to the other. P0 — Universal triage-label exclusion at the candidate collector ------------------------------------------------------------------ Pre-R3.5 only the reviewer-side Python filter honoured ``auto/needs-human-triage`` (via ``_evaluate_filter``'s ``is_excluded`` check). The implementer (and any other dispatcher that uses ``collect_candidates``) would still pick up triage-labeled PRs because their filter scripts (``list_prs_ci_failing`` etc) had no knowledge of the label. Result: the cycle-cap's "all automation pauses on this PR" signal was half-effective — the reviewer stopped iterating but the implementer kept trying, defeating the cap's purpose. Fix: filter triage-labeled items inline in ``collect_candidates`` so the exclusion applies to every dispatcher routed through that path, regardless of which filter (Python or legacy TS) produced the item. Uses the existing ``_cycle_cap.labels_carry_triage`` helper which handles both Forgejo label shapes (dict-of-name and flat string). P1.1 — apply_attempt_label gated on conclusive outcomes ------------------------------------------------------- ``_post_session_action_with_escalation`` wrote ``auto/last-attempt-tier-N`` unconditionally after the first worker attempt, including on ``timeout`` / ``transport-error`` sessions that never produced a verdict. The next cycle's ``_read_start_tier_from_labels`` then bumped the start tier to N+1 — wasting a tier on a fix the lower one might have handled, AND feeding the estimator's step 2a constraint a false "tier N failed" signal. Fix: gate the label write on ``terminal_state == "completed"`` AND a parsed_json with an outcome field. Environmental failures (network blip, OpenCode 5xx) now leave the prior cycle's tier label intact instead of bumping. An ``INFO`` log fires on the skip path so an operator grepping the journal can correlate "no label written this cycle" with the underlying terminal_state. The in-cycle escalation respawn (site 2 at line 3162) is unchanged — it only fires when ``_implementer_escalation.decide()`` returns ESCALATE, which already accounts for the retry budget and is only reached for conclusive failures. P1.2 — Cross-pool parity test ----------------------------- Two new tests in ``test_dispatch_runtime.py``: - ``test_collect_candidates_excludes_triage_labeled_items`` pins the cross-pool invariant with both label shapes (dict-of-name and flat string). Regression-guard: a future dispatcher that bypasses the collector or weakens the label check here would re-enable the doom-loop pattern. - ``test_collect_candidates_accepts_missing_labels_field`` covers the defensive case — items returned from older list scripts may omit ``labels`` entirely; the filter must treat that as "no labels" and pass the item through, not crash. Tests: 2305 auto_agents passing (+2 new from this commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
88b9373fa9 |
feat(auto-agents): LLM activity scraper + cost dashboard wiring
Closes the cost-tracking instrumentation gap: the telemetry console's Cost tab read from an empty ``llm_activity`` table because nothing in production wrote to it. The scraper walks the OpenCode session archives that ``_opencode_worker`` already writes (including subagent trees via the BFS-walked ``parentID`` chain) and emits one row per assistant turn. Folded into the existing PR-State Warmer loop so it runs on the same 30s cadence without spinning a new sidecar. Schema (v6): - ``llm_activity`` grows ``session_id`` / ``message_id`` / ``provider`` / ``parent_session_id`` / ``subagent_depth`` columns - Partial UNIQUE INDEX on ``message_id`` makes re-scrapes idempotent - v5→v6 migration ALTER-gated on column existence (safe to re-run) Scraper (``tools/llm_activity_scraper.py``): - Reads ``.dispatcher-logs/sessions/*.json``, one row per assistant turn - Folds reasoning tokens into ``tokens_out`` and cache-write into ``tokens_in`` (preserves raw breakdown in ``raw`` JSON for future cost-calc refinements) - Normalises ``subagent_depth=0`` at top level so dashboards can filter ``WHERE subagent_depth > 0`` cleanly - Batch INSERT OR IGNORE via new ``PipelineCache.upsert_llm_activity_batch`` — one fsync per archive, not per turn Warmer integration: - First tick: full backfill of the archive directory - Subsequent ticks: 1h lookback via ``since=`` filter - Scraper failures are logged and swallowed — PR-state job stays load-bearing and unaffected - ``LLM_ACTIVITY_SCRAPER_DISABLE=1`` env kill switch Renames (mechanical, atomic): - ``tools/_forgejo_cache.py`` → ``tools/_pipeline_cache.py`` - ``ForgejoCache`` class → ``PipelineCache`` - Both reflect the module's broader scope (Forgejo data + pipeline telemetry tables); on-disk filename ``forgejo.sqlite`` and ``FORGEJO_*`` env vars are kept for compatibility Verified end-to-end on real archives: 435 archives → 3595 turns ingested (2873 from subagents) across 8 models / 5 providers / 9 PRs. Re-runs insert 0, dedup 3595. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
80d61de942 |
feat(auto-agents): R3 wrapper-chain retirement — direct task-implementor variants
Eliminates the remaining LLM wrapper chain (``tier-dispatcher`` +
``tier-{min,0,1,2}`` selectors) between the Python dispatcher and the
``task-implementor`` worker. Follows the R2 implementation-worker
retirement (
|
||
|
drew
|
2658deee94 |
feat(auto-agents): PR State Warmer substrate + supporting infra
Adds a long-lived sidecar (pr_state_warmer.py) that polls Forgejo's /pulls endpoint every 30s and writes the full PR snapshot to a shared SQLite store, eliminating the dispatcher's per-cycle cold-cache stalls (24-30s rebuilds on flaky cycles) and the silent 50-PR pagination cap on the legacy single-page fetch. Substrate - tools/_pr_state_cache.py — SQLite store with (owner, repo) PK, WAL mode, additive v2→v3 migration (comments_refreshed_updated_at), bounded fcntl.flock migration lock, threading.Lock for per-process init, @_with_reheal decorator (catches OperationalError no-such- table + DatabaseError corruption with file quarantine), atomic TEMP-table chunking for >32k seen-set, _normalize_updated_at to canonicalize Forgejo tz-marker drift - tools/pr_state_warmer.py — poll/upsert/vanish/comments-refresh loop with fcntl.flock singleton (rejects second warmer), bounded comments-refresh cap, persistent deferral via SQL pending query, PermissionError-tolerant lock setup, cold-start log suppression - tools/_pr_classification_cache.py — three-layer fall-through (warmer cache → list cache → live fetch) with staleness gate (PR_STATE_WARMER_STALE_AFTER_S floored at 30s in prod) Comments cache hardening - Bot-filter at write time drops bot status/claim/release/sentinel while preserving **Implementation Attempt** markers (94.6% reduction on bot-heavy PRs like #30's 19k-comment thread) - _normalize_since_cursor strips microsecond precision before building ?since= query (fixes the live-observed Forgejo HTTP 422 bug on PRs #25 + #28); handles uppercase Z, lowercase z, ±HH:MM offsets (including non-zero like +05:30), naive ISO - Lazy migration of legacy null-key by_author entries on _read_cache - _newest_cursor walks tail-back skipping malformed entries Supporting infrastructure (cumulative dmpipeline-v2 work) - Telemetry server: SSE live tail, run-sessions enumeration, cost/token tracking, app.js UI rewrite with collapsible sections - MCP servers (mcp_ci_server, mcp_forgejo_server, mcp_git_server, mcp_handoff_server, mcp_graphify_server) for opencode worker context access - Live log writer (tools/live_log_writer.py) — SSE-streaming dispatcher event log - Tier-dispatcher escalation flow with prompts trimmed for budget - Shared bot-logins resolver (tools/_bot_logins.py) replacing two drift-prone copies - token_usage_audit.py for opencode cost analysis Tests - 2259 passing across 65 changed/new files - New suites: test_pr_state_cache, test_pr_state_warmer, test_pr_state_warmer_integration, test_pr_classification_cache, test_pr_list_cache_backoff, test_mcp_* (5 servers), test_live_log_writer_sse, test_telemetry_run_sessions, test_review_post_ready_label - Test_pr_comments_cache expanded with bot-filter coverage, cursor-normalization regression pins, format-drift, atomicity, failed-comments-not-stamped (silent-data-loss class) - Parametrized @_with_reheal coverage across 7 wrapped APIs - Real fault-inject atomicity test for chunked mark_vanished path via Connection wrapper class - Subprocess-based singleton flock test (cross-process contract) - Event-driven SIGTERM-mid-poll test (no fixed-sleep flake) Architecture notes - Schema v3 migration is additive (ALTER ADD COLUMN); v0/v1 still need destructive rebuild because pre-v2 column shape lacks owner/repo. Cross-process drop-table-ping-pong prevented by the fcntl migration lock + per-process _initialized flag. - Comments-refresh deferral is persistent via comments_refreshed_updated_at column — survives warmer restart, picks up next cycle even if PR didn't change again. Replaces in-memory changed_numbers list. - Rollback path: PR_STATE_WARMER_PREFER=0 bypasses the warmer cache and reverts to live-fetch behavior. PR_STATE_CACHE_DISABLE=1 short-circuits the warmer process at startup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
dc4e9368f2 |
feat(auto-agents): block-store substrate + DRY refactors
Externalise large prompt sections (PR diff, comments, CI failure
logs, reviews, linked issues) into a cross-process SQLite-backed
block store so the worker can recover original content when an
intermediate tier-* agent's summarisation strips inline sections.
New substrate
-------------
- ``tools/_block_store.py`` — SQLite WAL, per-row 1MB cap, 1h TTL,
janitor (startup + opportunistic per-hour in register()),
threading.Lock around the periodic-sweep gate.
- ``tools/_block_prompt.py`` — registration glue + ``## Available
blocks`` Markdown table renderer.
- ``tools/_prefetch_section.py`` — single-source-of-truth section
registry; collapses the duplicate ``_register_*_blocks`` helpers
the reviewer and implementer previously kept in lockstep.
- ``tools/mcp_block_store_server.py`` — FastMCP wrapper exposing
``block_fetch`` / ``block_list`` / ``block_register`` /
``block_invalidate`` to agents. Uses the expanded ``_mcp_common``
helpers.
- ``tools/_implementer_escalation_helpers.py`` — pure helpers
extracted from ``dispatch_implementer.py`` (~180 lines off the
3284-line file); takes ``claim_runtime`` as a kwarg for clean DI.
DRY refactors
-------------
- ``tools/_backoff.py`` — shared ``Backoff`` dataclass collapses the
three near-identical exponential-backoff state machines in
``_pr_comments_cache``, ``_ci_logs``, ``_pr_classification_cache``.
- ``tools/_mcp_common.py`` — expanded with ``bootstrap_loader``,
``error_envelope``, ``make_main`` so each MCP server's prelude is
three lines.
- ``tools/_pr_diff.build_diff_section_full`` — returns a 4-tuple
including the raw diff body so the reviewer's block-store
registration reuses the bytes instead of doing a second HTTP fetch.
Wiring
------
- ``_review_prompt.build_review_prompt`` builds a ``PrefetchSection``
registry via ``_review_sections``, registers them, and renders the
``## Available blocks`` table at the end of the prompt.
- ``_implementer_prefetch._fetch_pr_context`` /
``fetch_new_issue_context`` build the equivalent registry via
``_implementer_sections`` and stamp ``result.block_refs`` for the
prompt builder to read.
- ``_implementer_prompt`` builders include
``_build_available_blocks_section(result)`` in all three flows.
- ``dispatch_review.main`` + ``dispatch_implementer.main`` call
``_block_store.janitor()`` at startup; the per-call opportunistic
janitor in ``register()`` keeps the file bounded between restarts.
Agent contract updates
----------------------
- ``.opencode/agents/task-implementor.md`` +
``.opencode/agents/pr-review-worker.md``:
- ``block_store*`` permission
- new "Block-store substrate" paragraph explaining
``block_fetch`` / ``block_list`` as the summarisation recovery
path.
Tests
-----
- ``test_block_store.py`` — 48 tests pinning every public contract
(register/fetch/list/invalidate/janitor, key whitelist, TTL,
size cap, WAL durability).
- ``test_block_prompt`` — covered transitively via the e2e test.
- ``test_block_store_recovery_e2e.py`` — builds a real prompt via
``build_pr_fix_prompt``, applies a heading-bounded summariser stub
(``_summarise_inline_sections``), asserts inline content is stripped
yet block keys survive and ``block_fetch`` recovers original content.
Plus a ``block_list`` fallback test for the worst case where the
table itself was summarised away.
- ``test_mcp_block_store_server.py`` — 27 wrapper-contract tests.
- ``test_mcp_block_store_transport.py`` — spawns the actual server
subprocess via ``mcp.client.stdio`` and exercises the JSON-RPC
transport round-trip in ~1.5s. Catches FastMCP schema /
serialisation bugs the in-process tests miss.
- ``test_backoff.py`` — 14 behaviour-focused tests of the shared
``Backoff`` curve.
- ``test_pr_comments_cache.py`` + ``test_ci_logs.py`` — deleted the
now-redundant ``TestComputeNextAttemptAfter`` / ``TestBackoffActive``
/ ``TestBackoffHelpers`` classes; ``test_backoff`` covers them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
cf13de9b5a |
feat(auto-agents): per-repo linked-issue policy (strict / informational-on-not-found / disabled)
Adds a per-repo policy knob for ``Closes #N`` / ``ISSUES CLOSED: #N``
resolution during dispatcher pre-fetch. Three modes:
- ``strict`` (default, unchanged) — every reference is resolved; a
``not-found`` (4xx) result is rendered as a quality concern the
reviewer is expected to surface; a ``fetch-error`` (5xx / runtime)
trips the dispatcher's defensive APPROVED→COMMENT downgrade.
- ``informational-on-not-found`` — a ``not-found`` result is shown
to the reviewer with an explicit policy note: "this signal is
informational, not blocking; MUST NOT factor into your verdict."
``fetch-error`` semantics are unchanged. Typical setting for forks
harvesting commits from an upstream with its own issue tracker.
- ``disabled`` — skip resolution entirely; section renders
"linked-issue resolution disabled by repo policy" preamble. The
worker is told it has no traceability check to perform.
Wired through ``DispatchConfig.linked_issue_policy`` (defaults to
``"strict"``) and ``{REVIEW,IMPLEMENTER}_DISPATCHER_LINKED_ISSUE_POLICY``
env vars. ``normalize_linked_issue_policy()`` validates the value at
``load_config`` time so a typo'd env var fails startup with a clear
error rather than silently defaulting.
Coverage: 15 new tests across ``test_review_fetch.py`` and
``test_review_views.py``. Touched-module suite: 250 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
c787c133df |
docs(auto-agents): document the WorkGroup wrapper seam (A7)
Adds an "Extensibility: the WorkGroup seam" section to the _dispatch_runtime.py module docstring naming the deliberate design choice: pipeline-specific behaviour is added by injecting WorkGroup instances (with prompt_factory + post_session_action) into the generic loop, rather than by replicating the cycle / claim / lock machinery in each driver. This is dmpipeline's Python-side expression of the same pattern agents/final-working put in markdown — three thin-wrapper supervisors (implementation-supervisor, pr-review-supervisor, pr-merge-supervisor) over a generic supervisor agent. The parallel was the cleanest idea from the LLM-supervised branch; this docstring makes it explicit so future pipelines reuse the seam instead of duplicating the boilerplate. Refs: docs/development/final-working-harvest-plan.md (A7). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
afda1ab5f8 |
feat(auto-agents): post-claim TOCTOU verification (W5, default-OFF)
Closes the cross-driver claim-collision race. Forgejo's label-add
endpoint is idempotent — two concurrent dispatchers (impl ↔ rev,
or cross-host instances) can both come back HTTP 200 even though
only one was actually first. The single-instance fcntl lock
prevents same-driver races but cross-driver / cross-host races
remain. conflict_drive.py has had an opt-in mitigation for this
since plan § 3.3.1; this commit ports the simpler "different
auto/claimed-* label appeared in the race window" version into
_dispatch_runtime.dispatch_one so the implementer and reviewer
drivers benefit from the same protection.
Flag-gated via DISPATCHER_VERIFY_CLAIM_AFTER_APPLY=1, default OFF
— preserves today's accept-the-race behaviour byte-for-byte. When
ON:
1. claim_work_item succeeds → label attached.
2. Re-GET /issues/{N}/labels. If a different auto/claimed-* label
is present alongside ours, release our claim (with detail
"post-claim-verify") and return terminal_state="claim-collision".
3. Otherwise register the in-flight claim and proceed normally.
Transient GET failures don't trigger collision (defensive: rather
miss one race than spuriously abandon a healthy claim — the
cycle-failure-budget catches persistent fetch issues).
5 new tests: flag default OFF, the detector's three behaviours
(foreign label → collision, own only → no collision, fetch
failure → no collision), end-to-end dispatch_one releasing on
collision without spawning the worker.
Refs: docs/development/final-working-harvest-plan.md (W5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
1e8ef793d2 |
feat(auto-agents): startup PAT validation hard-stop (G5)
Closes the dead-PAT-spins-idle-forever gap. Without this probe, a
rotated / revoked Forgejo PAT lets every dispatcher loop forever:
each work item's claim returns labels-fetch-failed (a "soft"
outcome that counts as a cycle success with zero dispatches), so
the consecutive_failures counter resets every iteration and the
cycle-failure-budget escape hatch never trips. Operators see
timed-out heartbeats but no SystemExit and no log alert.
_validate_pat_or_die() probes GET /user at run_outer_loop startup:
- 200: log INFO with the resolved login (auditable bot identity
at process start), return.
- 401 / 403: raise SystemExit(2) — matches the existing
cycle-failure-budget exit code so supervisor restart semantics
are uniform.
- Other (transient 5xx, network flap, malformed body): log
WARNING and return; the cycle-failure-budget covers persistent
cases.
Skippable via DISPATCHER_SKIP_PAT_VALIDATION=1 for tests / bisect.
The skip is logged loudly so a missing safety net in production
telemetry is obvious.
6 new tests cover the 5 response classes + the env-skip path.
Three existing run_outer_loop tests stub the validator (they test
loop semantics, not PAT validation, which has its own dedicated
tests).
Refs: docs/development/final-working-harvest-plan.md (G5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
drew
|
49a28b5cf5 |
feat(auto-agents): deterministic compliance short-circuit + push-substrate fixes
Three blocker classes surfaced by the 2026-05-13 4-hour live pipeline
test (PR #30 / #28 / #25), all fixed here:
1. `_pr_clone` left `remote.origin.mirror=true` on the bare mirror,
making every `git push --force-with-lease origin <refspec>` fail
with `fatal: --mirror can't be combined with refspecs`. Worktrees
inherited the bad config; worker had to discover + work around it
each cycle, and one cycle's fragile recovery LOST a real commit.
`_disable_mirror_push_semantics` clears the flag at clone time
and on every refresh.
2. Predicate treated `outcome=resolved + head_sha_advanced=False` as
transport-class (UNKNOWN bucket → wasted same-tier retry). It's
actually competence — the worker emitted a complete-looking JSON
while delivering nothing. Now escalates immediately.
3. The dispatcher's own deterministic sections (compliance_gaps +
gate_preflight) misled the worker into emitting `resolved`
whenever both were clean — regardless of real remote CI state.
New module `_implementer_compliance_apply` + dispatcher hook
`_maybe_short_circuit` move trivial compliance fixes (CONTRIBUTORS
line, CHANGELOG stub from PR title, ISSUES CLOSED footer from
prefetched linked_issues) onto the dispatcher's deterministic
side per the auto-agents.md policy: "deterministic Python owns
orchestration; the LLM only handles the actual creative work."
Two short-circuit paths:
- P0 (always-on): compliance clean + preflight clean + remote CI =
success → skip LLM, emit `no_changes_needed`. Eliminates the
hallucination class observed live.
- A (opt-in via IMPLEMENTER_DISPATCHER_AUTO_FIX_COMPLIANCE=1):
compliance has only fixable gaps + preflight clean + CI failing →
dispatcher applies fixes, pushes, emits `resolved`. Worker
reserved for code-bug PRs only.
Plus supporting changes:
- gate_preflight payload includes `remote_ci_state` + `diverges_from_remote_ci`
- Telemetry rows gain `outcome_disputed=True` when resolved+no-push
- `EscalationAction.SUCCESS` accepts `no_changes_needed` outcome
- task-implementor.md procedure re-ordered: read `--field ci` FIRST
- Runtime hook `_dispatch_runtime._maybe_read_short_circuit` consumes
the dispatcher's `_short_circuit_result` stash and synthesizes a
SessionResult instead of spawning the LLM session
- Cycle archive's post_session_result gains `auto_fix_report` +
`short_circuit` fields
Tier B (perf) and Tier C (observability) deferred to follow-up.
Tests: 1464 passed, 3 skipped (+23 from
|
||
|
drew
|
d3d66f3726 |
feat(auto-agents): in-cycle implementer tier escalation 0→1→2
Adds a flag-gated escalation loop to the implementer dispatcher (`IMPLEMENTER_ESCALATION_ENABLED=1`, default OFF). When the worker fails in a way the predicate determines escalation can help, the dispatcher holds the claim, resets the worktree to the prefetched head_sha, refreshes the TTL via _claim_runtime.claim_pr, applies the next-tier label, and re-runs the worker at the next tier — all within the same cycle. Bounded by per-failure-class budgets in _implementer_escalation.BUDGET_PER_FAILURE_CLASS. Tier 2 (tier-kimi) is default-ON with a kill-switch flag (IMPLEMENTER_ESCALATION_TIER2_ENABLED=0). Cross-cycle resumption: the dispatcher reads auto/last-attempt-tier-N at cycle start and seeds start_tier = min(N+1, max_tier) so crash recovery skips known-failed tiers. Worker holds release across the cycle via the new release_claim_on_exit: false directive — eliminates the inter-tier claim-absent race window. Behaviour preservation: flag=0 path is byte-equivalent to the pre-feature build (worker prompt unchanged, Phase 4 row schema unchanged, status-comment fingerprint unchanged). Issue work (new_issue work group) always takes the legacy path even with the flag on. Supersedes the cross-cycle-only Phase 5c scheme in auto-agents-tier-2-3-plan.md (now updated to point at the new plan doc and the dual-role label semantics). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
0150c4fbc0 |
feat(auto-agents): Tier 1+2 follow-ups from 2026-05-11 post-mortem
R1: bake 20-min bash timeout into quality-gates skill recipes so the first cold-cache --fast call no longer trips OpenCode's 120 s default (recovered ~2 min that was lost on the 2026-05-11 PR #30 cycle to timeout-and-retry). R2: plumb subagent_max_depth from _archive_subagent_tree's BFS walk through SessionResult -> SessionContext -> extract_phase4_ telemetry so the field stops landing as null on real multi-tier cycles. Distinguishes None (walk failed / unknown) from 0 (measured-flat). +7 behavioural tests, 4 existing tests updated to consume the new (paths, max_depth) tuple. R3: throttled operator-visible "worker still in-flight" log line every 120 s (configurable via DISPATCHER_HEARTBEAT_LOG_INTERVAL_SECONDS) so a 20-min worker turn emits ~10 status lines instead of going completely silent between session-start and session-end. +5 tests covering env-var override, defaults, garbage-rejection, end-to-end emission, and short-cycle suppression. R4: triaged 25 errored steps in fork-local unit_tests -- conclusive finding that they're caused by Rich Console defaulting to 80-col width in non-TTY CliRunner mode, truncating asserted column headers. Pre-existing on every branch, unrelated to auto-agents. Documented in the Tier 2/3 plan so future operators don't re-spend the diagnostic time. R5: short-circuit tier-dispatcher's estimator call on first attempts via new optional escalation_tier_hint parameter; implementation-worker now hard-codes hint=0. Saves ~30-60 s wall-clock per cycle on the common case (estimator's recommendation converged on Tier 0 in every observed cycle to date; sample too small for a confidence interval). Future-proof: the hint becomes dynamic when the auto/last-attempt-tier-N label scheme lands. +2 static lint tests pin the contract. Pre-commit polish (P0/P1/P2 from the consolidated critique): - Renumber tier-dispatcher CRITICAL rules 6,9,7,8 -> 6,7,8,9 - End-to-end pin tests for the R2 closure-mutation chain (subagent_max_depth=1 and =None paths) - Deterministic time.monotonic mock in heartbeat tests so they no longer depend on real-clock timing - Reject bool from extract_phase4_telemetry's int check (bool is subclass of int in Python -- would slip True/False through as 1/0) - Replace d.get("_subagent_depth") or 0 footgun with int(d.get(..., 0)) - Document DISPATCHER_HEARTBEAT_LOG_INTERVAL_SECONDS as a startup-only knob - Calibrate the "~95% Tier 0" claim to "Tier 0 in every observed cycle, n=1" - Document the reviewer-side R2 asymmetry (SessionContext carries the field; reviewer telemetry sink doesn't emit yet) - Add R5 rollback procedure - Trim duplicated R5 rationale prose in implementation-worker.md - Add reviewer back-compat test for SessionContext with the new subagent_max_depth field Tests: 1238 passed (+3 net-new), 3 skipped. Lint + typecheck clean. Validated via the local_ci_gate.sh wrapper. |
||
|
drew
|
f27cf1e017 |
feat(auto-agents): prefetch worker credentials in prompt + redact PAT in archives + implementer-cycle skill (P0-1, P0-3)
P0-1: the implementer dispatcher now embeds forgejo_pat / git_user_name / git_user_email directly into the worker prompt under a new "## Worker credentials (use these instead of env vars)" block. Live-test post-mortem of the 2026-05-10 implementer run showed the worker burning 245 s across 3 turns probing for env vars (printf denied -> printenv ... || true denied -> printenv ... succeeded); with values inline those turns disappear entirely. The dispatcher additionally passes redact_values=[cfg.token] to run_session_blocking so every occurrence of the PAT is replaced with <REDACTED> in the on-disk session archive (prompt body, tool input.command, any nested error string). Minimum redact-length floor of 12 chars prevents accidental archive mangling when a caller passes too-short credentials. Both reviewer and implementer pipelines benefit. P0-3: new .opencode/skills/implementer-cycle/SKILL.md is a 130-line cheat sheet that replaces the heavier auto-agents-system skill load on the implementation-worker's claim/dispatch/release path. The worker .md inlines the full `npx --yes tsx ... claim_pr.ts ...` one-liners so the skill load is informational, not load-bearing. 15 new tests: 10 cover redaction unit/integration paths (multi-occurrence, multi-secret, short-value warning, negative control, prompt+tool-input end-to-end); 5 cover the credentials section (presence/absence/partial/empty/canonical-order). 1043 auto_agents tests pass / 3 skipped. Forward-looking expectation: ~245 s/4 min saved per implementer cycle + measurable input-token reduction. Will be re-measured against the next live dispatch_implementer --once run. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
drew
|
355af84fb1 |
refactor(auto-agents): hard-switch supervisor decommission + implementer parity
Combines the 2026-05-09 hard-switch decommissioning of the LLM
implementation/pr-review supervisors with the Phase 2/3/4/5b
implementer parity work (prefetch + preclone + telemetry + operator-
status comments) and the third/fourth-round critique cleanup.
Removed
- .opencode/agents/implementation-supervisor.md (340 LoC)
- .opencode/agents/pr-review-supervisor.md (348 LoC)
- _dispatch_runtime.assert_no_legacy_supervisor +
detect_legacy_supervisor_sessions and the SUPERVISOR_TAGS /
SUPERVISOR_OVERRIDE_ENV plumbing in both dispatchers, along with
the five supervisor-coexistence tests in test_dispatch_runtime.py
- _watchdog_helpers.parse_truthy_env + watchdog_check.py
--check-env mode + their dedicated unit tests (the legacy
DISPATCHERS_RUNNING gate had no callers after the watchdog
rewrite became unconditional)
Added
- tools/_implementer_prefetch.py — pre-dispatch Forgejo fetches
(PR/issue body, diff, CI status, comments, reviews, linked
issues, Epic) per work group
- tools/_implementer_prompt.py — pure-function prompt assembly
with UNTRUSTED CONTENT fences and shared
PR_COMPLIANCE_CHECKLIST / OUTPUT_CONTRACT
- tools/_phase4_telemetry.py — extractor + JSONL sink for the
Phase 4 plan metrics
- tools/_status_comments.py — per-fingerprint operator-status
comment substrate, namespaced for reviewer + implementer
- _dispatch_runtime.SessionContext dataclass + SIGTERM/SIGINT
cooperative claim release with synchronous handler
- TestSupervisorAgentsDecommissioned and
TestAutoAgentsMdIsWatchdogOnly anti-regression lints (glob over
*supervisor*.md in .opencode/agents/, plus body keyword bans
and bash allow-list lint)
- pyproject.toml `slow` marker registration for the subprocess
SIGTERM smoke test
- tests/auto_agents/fixtures/{phase4-acceptance.yaml,
phase4-session-output-sample.txt}
Rewritten
- .opencode/agents/auto-agents.md from supervisor-fleet manager
(~545 LoC) to dispatcher heartbeat watchdog (~184 LoC); host
init system / process manager (systemd / runit / docker) is now
the explicit restart authority instead of "host-level process
supervisor"
- AGENTS.md production-launch story (Shells A-D) reflects the
deterministic-Python orchestration boundary; the bot-identity
fork-mode paragraph reads from FORGEJO_OWNER / FORGEJO_REPO
env vars instead of the deleted hard-coded supervisor flags
- tools/launch_fork.sh header documents three host-level entry
points (dispatchers-launcher.sh, opencode-builder.sh,
merge_drive.py)
- worker self-descriptions (implementation-worker.md,
pr-review-worker.md) refer to the dispatcher / merge driver
instead of the deleted supervisors; session-health-quick-util.md
and async-agent-util.md treat -SUP-suffixed sessions as
flag-and-escalate signals
Tests: 1006 passed, 3 skipped, 0 failed under tests/auto_agents/.
Lint: zero new ruff errors on touched files; three pre-existing
errors in tools/_pr_diff.py at lines blamed to 2026-05-07.
Operator note: the only in-process rollback knob for prefetch
issues is IMPLEMENTER_DISPATCHER_PREFETCH=0 (and the matching
IMPLEMENTER_DISPATCHER_PRECLONE=0). Anything beyond that is git
revert of this commit. Residual doc surface in the
auto-agents-system and supervised-workers skill READMEs is
documentation-only; the agent files those READMEs reference no
longer exist.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
drew
|
2f1be34d12 |
feat(auto-agents): implementer parity — verify-invariant verifiers, implementer-helpers skill, watchdog gate, _opencode_worker audit
Closes the four open items in `docs/development/auto-agents-tier-2-3-plan.md` § "Revised remaining scope (2026-05-08)" plus three rounds of fresh-eyes critique fold-in (rounds 3, 5, and post-round-5 polish). Highlights: - New continuous invariant verifiers on a shared `_verify_common.py` substrate: `verify_review_invariant.py` (R1: approval-without-CI) and `verify_implementer_invariant.py` (I1: head-commit fails commit-lint, I2: PR description missing Epic reference). Strictly additive cron-job- shaped scripts that open idempotent `auto/invariant-violation` issues; safe to run every 15 minutes in production. - New `implementer-helpers` skill at `.opencode/skills/implementer-helpers/SKILL.md` + CLI at `tools/implementer_validate.py` (4 subcommands: validate-commit-message, validate-pr-compliance, validate-file-budget, validate-changelog). Mirrors the reviewer side; `tools/_commit_lint.py` is shared so a future change to commit policy updates one place. - `auto-agents.md` watchdog gate: `DISPATCHERS_RUNNING=1` puts the primary orchestrator into watchdog-only mode. Heartbeat resolution + age computation factored into `tools/_watchdog_helpers.py` + the CLI `tools/watchdog_check.py` so the agent only needs `python3 tools/watchdog_check.py *` and `sleep *` bash permissions. The reader honours the env-var override first, then falls back to a freshest-mtime scan across `/var/run` / `$XDG_RUNTIME_DIR` / `/tmp` (deliberately diverging from the dispatcher's first-existing fallback to guard against stale heartbeats from previous root-owned sessions masking healthy user-mode heartbeats). - `_opencode_worker.py` audit: structured `error_kind` classification at every transport-error / timeout return site, plumbed through `_dispatch_runtime.py` into the cycle-log; new `_request_read` retry helper (3 × 0.5s linear backoff, transport-only) wrapping every idempotent read in a worker session so a single transient flap on a polling GET cannot trash a 10-minute worker session. - Static heredoc lint at `tests/auto_agents/test_prompt_heredoc_lint.py` glob-walks every agent prompt and skill recipe markdown, rejecting any heredoc bash recipe in a fenced code block (per `bash-commands.md` rule 2 — heredocs fail at OpenCode's permission-engine parse time). - `bash-commands.md` rule 2 + its fix-it advice both lead with apostrophe-safe `printf "%s" "<body>"` (double-quoted) form; single-quoted form documented as the fragile JSON-only fallback. - `CHANGELOG.md` carries the full multi-round narrative (round 3 CRITICAL/HIGH/MEDIUM/LOW fold-in, round 5 docstring drift + telemetry refactor + broader heredoc lint scope, post-round-5 doc-drift polish). Net delta: +911 passing tests / 3 skipped (was 825 / 3); ruff clean on every new file; pre-existing lint debt in `_dispatch_runtime.py`, `_opencode_worker.py`, `conftest.py`, `_commit_lint.py` unchanged and out of scope for this commit. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
drew
|
0eb79c881d |
fix(auto-agents): move review submission into dispatcher (#2 full + safe glob)
Restructures the reviewer pipeline so every Forgejo curl GET / POST that ``pr-review-worker`` used to issue is now handled deterministically by ``dispatch_review.py``. The worker becomes "read pre-fetched data, emit structured JSON, exit" — no HTTP, no claim_pr.ts, no skill loads beyond the contributing checklist. Why: the first end-to-end ``dispatch_review --once`` run against PR #30 spent the whole 30-min budget retrying permission-denied curl POSTs (multi-line ``-d`` continuations don't match the bash glob, nested ``/tmp`` writes for review bodies don't match ``/tmp/*``, ``python3 -c "..."`` heredocs don't match anything we allow). Loosening the bash permissions to fix this would expand the worker's blast radius into arbitrary code execution; moving the submission into the dispatcher removes the operations that triggered the denials in the first place. Changes: - New module ``tools/_review_pipeline.py`` (six fetchers + linked-issue parser + brace-balanced JSON parser + three posters + finalize_review orchestrator). All fetchers swallow their own failures so a transient Forgejo blip degrades to "Pre-fetched X unavailable" rather than burning the cycle-failure budget. - ``tools/dispatch_review.py`` rewired: ``_fetch_review_context`` runs every fetcher in one pre-dispatch pass; ``_review_prompt`` now embeds the diff + 5 new pre-fetched data fences + an ``Output contract`` block describing the JSON the worker must emit; each work group binds a ``_build_post_session_action`` closure so ``dispatch_one`` POSTs the worker's verdict on its behalf. - ``WorkGroup.post_session_action`` hook in ``_dispatch_runtime.py`` (optional, defaults to None — implementer / merge-driver dispatchers are unchanged). Exceptions in the action are caught and recorded so a buggy hook can't orphan the claim release. - Defensive Tier 1F enforcement: dispatcher re-fetches the REQUEST_CHANGES count after the session and overrides the worker's outcome to ``tier_1f_escalation`` when count >= 5, attaching ``auto/needs-implementer`` per the tier-dispatcher contract. - ``pr-review-worker.md`` rewritten to drop every curl GET / POST step, drop claim_pr.ts (dispatcher owns the claim lifecycle), drop the ``forgejo-api`` and ``auto-agents-system`` skills, switch ``webfetch`` from allow to deny, and document the JSON output contract. - ``/tmp/*`` -> ``/tmp/**`` glob fix on the worker's ``external_directory`` / ``edit`` / ``write`` permissions plus the matching bash patterns (``git -C /tmp/**``, ``mkdir /tmp/**``, ``rm -rf /tmp/**``). Strict superset of the previous surface; un-breaks nested writes the model organically tries. Tests: +49 unit tests in ``test_review_pipeline.py`` (every fetcher, the brace-balanced parser, every poster, the orchestrator's terminal- state gating + dispatcher RC override + parse-failure recovery + poster- exception swallowing), +5 integration tests in ``test_dispatch_runtime.py`` covering the new ``post_session_action`` hook (default None, fires on timeout, doesn't fire on already-claimed, exception is caught and release still runs). Existing prompt tests in ``test_dispatch_review.py`` silence the new fetchers via a stub helper so they remain network- isolated. Full-prompt snapshot test in ``test_dispatch_runtime.py`` switched from a 4 KB byte-equivalent comparison to structural assertions (header byte-equivalent + every section in documented order + every outcome named in the output contract + ASCII-only) — same regression coverage with far less maintenance cost. 435 passed, 1 skipped in tests/auto_agents/. Skipped from the parallel proposal: adding ``python3 *`` to the worker's bash allow. Python is Turing-complete and inherits the OpenCode server env, so allowing it would expand the worker's surface from "scoped writes" to "arbitrary code + outbound network + home-dir read". That is a containerisation problem, not a permission-glob fix. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
drew
|
2cbe62a70c |
fix(auto-agents): F1+F2+F3 telemetry liveness, reviewer perf (A+C), bash rules
Bundles a long-overdue set of fixes that surfaced while watching the single-PR pipeline test against PR #30 the morning of 2026-05-07. # F1 — long-worker liveness contract The dispatcher heartbeat was only refreshed *between* worker sessions. On a 30-minute review the heartbeat file went stale, and any heartbeat-watchdog (dispatchers-launcher.sh / cleveragents-dispatchers.service) would SIGTERM a perfectly-healthy worker mid-cycle, orphaning the OpenCode session and the auto/claimed-* lock. ``_opencode_worker.run_session_blocking`` now accepts an ``on_poll`` callback fired once per status-poll iteration; ``_dispatch_runtime.dispatch_one`` and ``conflict_drive.py`` wire it to ``write_heartbeat(cfg.heartbeat_path)``. Callback exceptions are logged and swallowed so a transient EROFS on the heartbeat path can never mask a successful worker completion. # F2 — in-flight cycle visibility (schema v5) The ``dispatch_*_cycles`` tables previously only recorded a row at cycle *end*. While a worker was running, the operator's only signal was the heartbeat file — and even that became stale (see F1). Schema bumped to v5: ``ended_at`` is now nullable and ``cycle_id`` carries a UNIQUE index. ``begin_cycle`` writes the in-flight row at start; ``finish_cycle`` updates it at end. ``run_one_cycle``'s try/finally guarantees ``finish_cycle`` runs even when ``collect_candidates`` / ``dispatch_one`` raises, so an orphan ``ended_at IS NULL`` can no longer be stuck forever after a crash. The v4→v5 migration is now defined in ONE place — a set of helpers in ``_forgejo_cache.py`` (``DISPATCH_CYCLE_TABLES``, ``_dispatch_cycle_create_sql``, ``_dispatch_cycle_index_sqls``, ``migrate_dispatch_cycle_table_to_v5``, ``ensure_dispatch_cycle_schema``). Both ``ForgejoCache._migrate_to_v5_in_flight_rows`` and ``_dispatch_runtime.ensure_cycle_table`` import from there, eliminating the drift risk of the previous duplicated DDL. Pre-existing rows are preserved verbatim across the migration. # F3 — telemetry surface for the new state ``/api/health`` now returns ``in_flight_cycle: {cycle_id, started_at, session_id, candidates_count, elapsed_s}`` per dispatcher daemon and a ``running_long_worker: bool`` flag (heartbeat older than 600s AND a matching pid alive — should never fire under healthy F1 operation, so when it does it points at a real bug). The Drivers and Overview tabs in ``.opencode/telemetry/{index.html,app.js,style.css}`` render in-flight rows with a tinted background + "in flight" pill, daemon tiles get a dashed border for the long-worker state, and each tile shows the running cycle's id + elapsed time inline. # Fix A — reasoningEffort high → medium for pr-review-worker On its own that change alone would not have been enough, but combined with Fix C below it dropped a representative cycle from "timed out at 30:00" to a target ~2-3min. Pure config change in ``.opencode/agents/pr-review-worker.md``; no code path touched. # Fix C — pre-fetch PR diff in dispatch_review and embed in prompt The reviewer used to spawn a ``git-isolator-util`` subagent, which shelled out to ``git clone``, ``git fetch``, and ``git diff master...HEAD``. That subagent burned 90+ seconds and several token budgets per cycle. ``dispatch_review.py`` now fetches the unified diff via the Forgejo ``/pulls/{n}.diff`` endpoint and embeds it into the worker prompt under an ``UNTRUSTED CONTENT`` fence with explicit BEGIN_PR_DIFF / END_PR_DIFF markers, head_sha pinning, character-count metadata, and END marker redaction to defeat patch-text injection. The worker is instructed to use the embedded diff and skip the isolator subagent entirely when it is present. Falls back to the old path on fetch failure or via the ``REVIEW_DISPATCHER_EMBED_DIFF=0`` env switch. # Cross-cutting bash rules The ``pr-review-worker``'s shell tool calls kept hitting ``permission denied`` because OpenCode's permission engine matches the *raw, unexpanded* command string against allow-globs. Chained commands (``&&``, ``||``, ``;``, ``|``), command substitution (``$(...)``), bare variable assignments, multi-line continuations (``\\\n``), heredocs, and inline ``python3 -c "..."`` strings all contain characters the permission glob cannot span, and were silently denied. Added ``.opencode/instructions/bash-commands.md`` (wired into ``opencode.json`` via the ``instructions`` array so it appends to EVERY agent's system prompt globally), with hard rules + recovery recipes (``printf > /tmp/file`` instead of heredocs; ``printf > /tmp/script.py`` + ``python3 /tmp/script.py`` instead of ``python3 -c``; ``curl -d @/tmp/body.json`` instead of multi-line ``-d '{...}'``). # Pre-commit polish (architect/dev/test review) Surfaced during a chief-architect / principal-developer / senior-test-engineer code review of the uncommitted change: - Schema DDL deduplication (described above under F2). - ``finish_cycle`` INSERT-fallback now preserves ``started_at`` / ``driver_name`` when caller provides them; otherwise stamps a ``synthetic_started_at: true`` flag in the raw blob so cycle-time analytics can exclude rows whose duration was synthesised. - ``bytes=`` → ``chars=`` in the embedded-diff header. The value is ``len(diff_text)`` after ``decode("utf-8")`` — a UTF-8 character count, not a byte count. Off-by-multibyte for non-ASCII patches. - ``scripts/opencode-builder.sh`` mode 644 → 755. - ``.gitignore`` entries for ``.parked-prs.json`` (runtime state for ``tools/park_other_prs.py --restore``) and ``.dispatcher-logs/`` (append-only local pipeline log directory). # Tests (381 passed, 1 skipped) - ``test_opencode_worker.py``: 3 new tests for ``on_poll`` cadence, error swallowing, and backwards-compatible default. - ``test_dispatch_runtime.py``: 7 new tests for ``begin_cycle`` / ``finish_cycle`` semantics, the v4→v5 migration with row preservation, the crash-safe try/finally path, the ``dispatch_one`` → ``run_session_blocking`` ``on_poll`` wiring, and the new ``started_at`` / ``driver_name`` plumbing through the INSERT-fallback branch. - ``test_telemetry_server.py``: 4 new tests for ``in_flight_cycle`` in ``/api/health``, the elapsed-seconds computation, and the ``running_long_worker`` flag. - ``test_telemetry_schema.py``: assertion bumped from v4 → v5 and a new test confirming the cycle tables now allow ``ended_at IS NULL``. - ``test_dispatch_review.py`` (new file): 14 tests for diff fetch (happy path, truncation, HTTP/URL errors, END_PR_DIFF redaction, Forgejo auth scheme), ``_build_diff_section`` (dry-run, env toggle, embedding, fallback), and end-to-end prompt embedding. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
drew
|
f89d275650 |
chore(dispatchers): apply pre-push critique fixes + ship production launcher
Layered polish on top of
|
||
|
drew
|
593d142f6f |
feat(auto-agents): Tier 2 deterministic review/implementer dispatchers
Replaces the long-running pr-review-supervisor / implementation-
supervisor LLM polling loops with host-level Python dispatchers that
own queueing, claim ownership, watchdogs, and SQLite telemetry. The
LLM workers retain sole responsibility for review judgment and code
generation; Python owns only orchestration. The hard merge invariant
is unaffected — these dispatchers do not touch master.
Driver surface
- _opencode_worker.run_session_blocking — outcome-agnostic OpenCode
session lifecycle (completed / timeout / transport-error). Used
directly by reviewer / implementer dispatchers whose workers do
not emit the conflict-driver JSON exit schema.
run_worker_blocking is now a thin wrapper that adds the
conflict-specific JSON-outcome classification.
- _dispatch_runtime — shared Python runtime (work-group polling,
claim helpers, dispatch loop, telemetry). Pre-checks issue labels
before claiming and refuses when any auto/claimed-* is already
present, distinguishing already-claimed vs labels-fetch-failed
vs claim-failed terminal states. Frozen DispatchConfig.
- dispatch_review.py / dispatch_implementer.py — per-pipeline work
groups, prompts, and CLIs. Each refuses startup when a competing
AUTO-REV-SUP / AUTO-IMP-SUP legacy supervisor is live on the
same OpenCode server (override:
{REVIEW,IMPLEMENTER}_DISPATCHER_ALLOW_SUPERVISOR_COEXIST=1).
- _loader.py — shared sibling-module loader; replaces the three
duplicated copies in the dispatcher entry points.
Operational hardening
- run_outer_loop tracks consecutive cycle exceptions against
cycle_failure_budget (default 5, env-tunable per driver) and
exits 2 on exhaustion for supervisor-driven restart.
- _sanitize_release_detail strips control bytes and neutralises
triple-backtick fences before quoting worker raw_response in
Forgejo claim-release comments.
- scripts/opencode-builder.sh: OPENCODE_BUILDER_SERVER_ONLY=1 keeps
only the OpenCode HTTP API up so the Python dispatchers own
queue orchestration without auto-agents running concurrently.
Telemetry
- _forgejo_cache.py schema v4: dispatch_review_cycles,
dispatch_implementer_cycles. One row per cycle with cycle_id,
driver, candidates_count, claims_acquired, swept_count,
processed_count, terminal_state, worker_outcome, session_id,
worker_wallclock_seconds, raw.
- .opencode/telemetry/server.py wires the new tables into
/api/cycles?driver=dispatch_review|dispatch_implementer and
surfaces a composite terminal_state/worker_outcome 24h breakdown
so dashboards can distinguish session-level vs work-level
outcomes.
Tests
- 31 new tests in tests/auto_agents/test_dispatch_runtime.py
covering: candidate priority/dedup, claim/release labels,
foreign-claim refusal, same-kind-claim refusal,
labels-fetch-failed terminal state, sanitization, supervisor
coexistence guard (pass/refuse/override/unreachable-server),
session timeout / transport-error propagation,
JSON-vs-no-JSON worker exits, cycle failure budget exit and
reset, heartbeat cadence, end-to-end --once --dry-run /
--status CLI smoke, and full prompt-snapshot tests for
_review_prompt and _implementation_prompt (PR-fix + issue-impl).
- test_telemetry_schema.py asserts schema v4 and the presence of
the two new dispatcher cycle tables.
338 auto_agents tests pass (was 322 before Tier 2). Conflict driver
regression suite unchanged. Dispatchers run cleanly under
--status / --once --dry-run with no Forgejo or OpenCode HTTP traffic.
Co-authored-by: Cursor <cursoragent@cursor.com>
|