ae940f45644971d5937b755d2459eceffeb6500a
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 drew
|
88b9373fa9 |
feat(auto-agents): LLM activity scraper + cost dashboard wiring
Closes the cost-tracking instrumentation gap: the telemetry console's Cost tab read from an empty ``llm_activity`` table because nothing in production wrote to it. The scraper walks the OpenCode session archives that ``_opencode_worker`` already writes (including subagent trees via the BFS-walked ``parentID`` chain) and emits one row per assistant turn. Folded into the existing PR-State Warmer loop so it runs on the same 30s cadence without spinning a new sidecar. Schema (v6): - ``llm_activity`` grows ``session_id`` / ``message_id`` / ``provider`` / ``parent_session_id`` / ``subagent_depth`` columns - Partial UNIQUE INDEX on ``message_id`` makes re-scrapes idempotent - v5→v6 migration ALTER-gated on column existence (safe to re-run) Scraper (``tools/llm_activity_scraper.py``): - Reads ``.dispatcher-logs/sessions/*.json``, one row per assistant turn - Folds reasoning tokens into ``tokens_out`` and cache-write into ``tokens_in`` (preserves raw breakdown in ``raw`` JSON for future cost-calc refinements) - Normalises ``subagent_depth=0`` at top level so dashboards can filter ``WHERE subagent_depth > 0`` cleanly - Batch INSERT OR IGNORE via new ``PipelineCache.upsert_llm_activity_batch`` — one fsync per archive, not per turn Warmer integration: - First tick: full backfill of the archive directory - Subsequent ticks: 1h lookback via ``since=`` filter - Scraper failures are logged and swallowed — PR-state job stays load-bearing and unaffected - ``LLM_ACTIVITY_SCRAPER_DISABLE=1`` env kill switch Renames (mechanical, atomic): - ``tools/_forgejo_cache.py`` → ``tools/_pipeline_cache.py`` - ``ForgejoCache`` class → ``PipelineCache`` - Both reflect the module's broader scope (Forgejo data + pipeline telemetry tables); on-disk filename ``forgejo.sqlite`` and ``FORGEJO_*`` env vars are kept for compatibility Verified end-to-end on real archives: 435 archives → 3595 turns ingested (2873 from subagents) across 8 models / 5 providers / 9 PRs. Re-runs insert 0, dedup 3595. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
drew
|
07013335b9 |
Auto-agents pipeline: enforce merge invariant with deterministic driver
Implement the improve_auto-agents_pipeline plan end-to-end and apply post-implementation hardening fixes so master can only advance via SHAs that passed CI against the exact current master.
Key changes:
- Add deterministic merge driver in `tools/merge_drive.py`:
- Single-instance lock (`fcntl.flock`) + heartbeat/status surfaces.
- Train-merge with bisect-on-failure and independent bisect/restart budgets.
- `head_commit_id` optimistic lock enforcement on merge endpoint.
- Single-PR strategy switched to `Do=merge` (sha-stable) to avoid unverified rewritten commits.
- Restart throttling/sleeps to prevent burning retry budget under master churn.
- Persistent clone management (`ensure_repo`: fetch/reset/clean with fresh-clone fallback on corruption).
- Cooperative SIGTERM/SIGINT stop propagation through long CI polling.
- Explicit claim lifecycle for `auto/claimed-merge`:
- claim/release comments with TTL,
- expired-claim sweep,
- operational labels on release (`auto/ci-timeout`, `auto/restart-throttled`,
`auto/needs-implementer`, `auto/needs-conflict-resolution`).
- Claim-marker interoperability: sweep recognizes both driver and `claim_pr.ts` markers.
- Hardened API semantics: split idempotent GET retries vs state-change semantics.
- Remove token-in-URL clone pattern; use git `http.extraheader` auth instead.
- Adopt structured module logging + env-configurable levels.
- Add/expand invariant auditor in `tools/verify_invariant.py`:
- Non-zero exit when violations exist (cron/alert correctness).
- Robust auto-close routine using forward patch applicability on current `origin/master`.
- Merge-bot commit validation now checks:
- required CI contexts passed,
- associated PR has non-dismissed APPROVED review.
- Improve close-path wording/docs to match forward-apply algorithm.
- Add logger-based output and verbosity controls.
- Add operational setup/audit tooling:
- `tools/forgejo_audit.py` (preconditions/audit report).
- `tools/setup_auto_labels.py` (idempotent `auto/*` label provisioning).
- `tools/setup_branch_protection.py` (direct-push allow-list enforcement).
- `tools/audit_branch_protection.py` (dismiss_stale_approvals audit/flip support).
- `tools/migrate_to_new_driver.py` (claim/schedule/train cleanup migration).
- `tools/flag_stale_prs.py` (idle PR triage flow).
- `tools/local_ci_gate.sh` canonical local gate runner with `--continue-on-fail`.
- Add claim orchestration support in skills scripts:
- New `claim_pr.ts` helper (claim/release + TTL comments).
- `list_prs.ts` gains `--exclude-claimed` filter.
- Update script reference docs accordingly.
- Telemetry/schema upgrades in `tools/_forgejo_cache.py`:
- Add `merge_cycle`, `ci_gate_events`, `llm_activity`.
- Add batched `ci_gate_events` insertion API with rollback semantics.
- Ensure `bisect_depth` default handling is safe.
- Surface merge-driver telemetry in velocity reporting pipeline.
- Agent prompt/behavior updates:
- Review supervisor idle loop tuned (300s -> 60s).
- Review worker cycle cap/escalation behavior refined.
- Task implementor guidance updated to use local CI gate wrapper.
- Documentation and operational guidance:
- Expand `AGENTS.md` with merge invariant runbook, label registry, tool links,
and full merge-driver env var catalog (including logging/restart/claim TTL knobs).
- Update `CHANGELOG.md` with implementation and hardening entries, plus deferred TS-test note.
- Repo hygiene:
- Correct `.gitignore` to stop blanket ignoring `tools/*`; keep only generated artifacts ignored.
Testing/validation:
- Add comprehensive unit suite under `tests/auto_agents/` covering:
- merge driver recursion/restarts/409 paths/signal handling/claim sweeps,
- verifier auto-close logic with real local git fixtures,
- schema migration + telemetry batch writes,
- branch protection and setup/audit helpers.
- Current result: `100 passed` in `tests/auto_agents/`.
|