07013335b9
Implement the improve_auto-agents_pipeline plan end-to-end and apply post-implementation hardening fixes so master can only advance via SHAs that passed CI against the exact current master.
Key changes:
- Add deterministic merge driver in `tools/merge_drive.py`:
- Single-instance lock (`fcntl.flock`) + heartbeat/status surfaces.
- Train-merge with bisect-on-failure and independent bisect/restart budgets.
- `head_commit_id` optimistic lock enforcement on merge endpoint.
- Single-PR strategy switched to `Do=merge` (sha-stable) to avoid unverified rewritten commits.
- Restart throttling/sleeps to prevent burning retry budget under master churn.
- Persistent clone management (`ensure_repo`: fetch/reset/clean with fresh-clone fallback on corruption).
- Cooperative SIGTERM/SIGINT stop propagation through long CI polling.
- Explicit claim lifecycle for `auto/claimed-merge`:
- claim/release comments with TTL,
- expired-claim sweep,
- operational labels on release (`auto/ci-timeout`, `auto/restart-throttled`,
`auto/needs-implementer`, `auto/needs-conflict-resolution`).
- Claim-marker interoperability: sweep recognizes both driver and `claim_pr.ts` markers.
- Hardened API semantics: split idempotent GET retries vs state-change semantics.
- Remove token-in-URL clone pattern; use git `http.extraheader` auth instead.
- Adopt structured module logging + env-configurable levels.
- Add/expand invariant auditor in `tools/verify_invariant.py`:
- Non-zero exit when violations exist (cron/alert correctness).
- Robust auto-close routine using forward patch applicability on current `origin/master`.
- Merge-bot commit validation now checks:
- required CI contexts passed,
- associated PR has non-dismissed APPROVED review.
- Improve close-path wording/docs to match forward-apply algorithm.
- Add logger-based output and verbosity controls.
- Add operational setup/audit tooling:
- `tools/forgejo_audit.py` (preconditions/audit report).
- `tools/setup_auto_labels.py` (idempotent `auto/*` label provisioning).
- `tools/setup_branch_protection.py` (direct-push allow-list enforcement).
- `tools/audit_branch_protection.py` (dismiss_stale_approvals audit/flip support).
- `tools/migrate_to_new_driver.py` (claim/schedule/train cleanup migration).
- `tools/flag_stale_prs.py` (idle PR triage flow).
- `tools/local_ci_gate.sh` canonical local gate runner with `--continue-on-fail`.
- Add claim orchestration support in skills scripts:
- New `claim_pr.ts` helper (claim/release + TTL comments).
- `list_prs.ts` gains `--exclude-claimed` filter.
- Update script reference docs accordingly.
- Telemetry/schema upgrades in `tools/_forgejo_cache.py`:
- Add `merge_cycle`, `ci_gate_events`, `llm_activity`.
- Add batched `ci_gate_events` insertion API with rollback semantics.
- Ensure `bisect_depth` default handling is safe.
- Surface merge-driver telemetry in velocity reporting pipeline.
- Agent prompt/behavior updates:
- Review supervisor idle loop tuned (300s -> 60s).
- Review worker cycle cap/escalation behavior refined.
- Task implementor guidance updated to use local CI gate wrapper.
- Documentation and operational guidance:
- Expand `AGENTS.md` with merge invariant runbook, label registry, tool links,
and full merge-driver env var catalog (including logging/restart/claim TTL knobs).
- Update `CHANGELOG.md` with implementation and hardening entries, plus deferred TS-test note.
- Repo hygiene:
- Correct `.gitignore` to stop blanket ignoring `tools/*`; keep only generated artifacts ignored.
Testing/validation:
- Add comprehensive unit suite under `tests/auto_agents/` covering:
- merge driver recursion/restarts/409 paths/signal handling/claim sweeps,
- verifier auto-close logic with real local git fixtures,
- schema migration + telemetry batch writes,
- branch protection and setup/audit helpers.
- Current result: `100 passed` in `tests/auto_agents/`.
231 lines
7.5 KiB
Bash
Executable File
231 lines
7.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# tools/reports.sh — interactive menu for refreshing canvas reports
|
|
# and exporting them to PDF.
|
|
#
|
|
# Top-level menu:
|
|
# 1) Refresh PR velocity canvas → once / every 15 min / hourly
|
|
# 2) Refresh milestone completion canvas → once / every 15 min / hourly
|
|
# 3) Print a report to PDF → sub-menu of discoverable canvases
|
|
# q) Quit
|
|
#
|
|
# Continuous-loop modes run until Ctrl+C, at which point the whole
|
|
# script exits. "Run once" actions return to the main menu so you can
|
|
# chain operations (refresh → export to PDF) in a single session.
|
|
#
|
|
# PDFs are written to ./tools/pdf_reports/ (gitignored). The menu prints
|
|
# the resolved path after each export.
|
|
|
|
set -uo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
cd "$REPO_ROOT"
|
|
|
|
PDF_OUT_DIR="$REPO_ROOT/tools/pdf_reports"
|
|
|
|
RENDER_PR_VELOCITY=(python3 tools/render-pr-velocity.py)
|
|
RENDER_MILESTONES=(python3 tools/render-milestones.py)
|
|
EXPORT_PDF=(python3 tools/export-canvas-pdf.py)
|
|
|
|
timestamp() {
|
|
date '+%Y-%m-%d %H:%M:%S %Z'
|
|
}
|
|
|
|
# ── refresh helpers ─────────────────────────────────────────────────────────
|
|
|
|
run_refresh() {
|
|
# Args: <label> <cmd...>
|
|
local label="$1"; shift
|
|
echo
|
|
echo "[$(timestamp)] refreshing $label canvas ..."
|
|
if "$@"; then
|
|
echo "[$(timestamp)] done."
|
|
return 0
|
|
fi
|
|
local rc=$?
|
|
echo "[$(timestamp)] FAILED (exit $rc)" >&2
|
|
return "$rc"
|
|
}
|
|
|
|
run_loop() {
|
|
# Args: <interval_sec> <interval_label> <canvas_label> <cmd...>
|
|
local interval_sec="$1"
|
|
local interval_label="$2"
|
|
local canvas_label="$3"
|
|
shift 3
|
|
echo
|
|
echo "Refreshing $canvas_label every $interval_label. Press Ctrl+C to stop."
|
|
# In loop mode, Ctrl+C tears down the whole script — that's the
|
|
# intuitive UX: the user wanted out, don't bounce them to a menu.
|
|
trap 'echo; echo "[$(timestamp)] stopped by user."; exit 0' INT
|
|
while true; do
|
|
run_refresh "$canvas_label" "$@" || true
|
|
echo "[$(timestamp)] sleeping ${interval_sec}s until next refresh ..."
|
|
sleep "$interval_sec"
|
|
done
|
|
}
|
|
|
|
refresh_submenu() {
|
|
# Args: <canvas_label> <cmd...>
|
|
local canvas_label="$1"; shift
|
|
local title_upper
|
|
title_upper="$(echo "$canvas_label" | tr '[:lower:]' '[:upper:]')"
|
|
while true; do
|
|
cat <<EOF
|
|
|
|
──────────────────────────────────────────────
|
|
Refresh $title_upper canvas
|
|
──────────────────────────────────────────────
|
|
1) Run once and exit to main menu
|
|
2) Run continuously every 15 minutes
|
|
3) Run continuously every hour
|
|
b) Back to main menu
|
|
──────────────────────────────────────────────
|
|
EOF
|
|
read -rp "Choice: " choice
|
|
case "$choice" in
|
|
1) run_refresh "$canvas_label" "$@" || true; return 0 ;;
|
|
2) run_loop 900 "15 minutes" "$canvas_label" "$@" ;;
|
|
3) run_loop 3600 "1 hour" "$canvas_label" "$@" ;;
|
|
b|B) return 0 ;;
|
|
"") continue ;;
|
|
*) echo "Invalid choice: $choice" ;;
|
|
esac
|
|
done
|
|
}
|
|
|
|
# ── PDF export sub-menu ─────────────────────────────────────────────────────
|
|
|
|
pdf_submenu() {
|
|
mkdir -p "$PDF_OUT_DIR"
|
|
|
|
local list_out
|
|
list_out="$("${EXPORT_PDF[@]}" --list 2>/dev/null || true)"
|
|
if [[ -z "$list_out" ]]; then
|
|
echo
|
|
echo "No canvases found. Check your Cursor canvas folder."
|
|
return 1
|
|
fi
|
|
|
|
# mapfile works on bash 4+; read line-by-line for portability.
|
|
local -a CANVASES=()
|
|
while IFS= read -r line; do
|
|
[[ -n "$line" ]] && CANVASES+=("$line")
|
|
done <<< "$list_out"
|
|
|
|
if [[ ${#CANVASES[@]} -eq 0 ]]; then
|
|
echo
|
|
echo "No canvases found."
|
|
return 1
|
|
fi
|
|
|
|
while true; do
|
|
echo
|
|
echo "──────────────────────────────────────────────"
|
|
echo " Print a report to PDF"
|
|
echo "──────────────────────────────────────────────"
|
|
local i path stem
|
|
for ((i=0; i<${#CANVASES[@]}; i++)); do
|
|
path="${CANVASES[$i]}"
|
|
stem="${path##*/}"
|
|
stem="${stem%.canvas.tsx}"
|
|
printf " %d) %s\n" "$((i+1))" "$stem"
|
|
done
|
|
echo " a) Export ALL canvases"
|
|
echo " b) Back to main menu"
|
|
echo "──────────────────────────────────────────────"
|
|
echo " Output directory: $PDF_OUT_DIR"
|
|
echo "──────────────────────────────────────────────"
|
|
read -rp "Choice: " choice
|
|
|
|
case "$choice" in
|
|
a|A) export_all; return 0 ;;
|
|
b|B) return 0 ;;
|
|
"") continue ;;
|
|
*)
|
|
if [[ "$choice" =~ ^[0-9]+$ ]] \
|
|
&& (( choice >= 1 )) \
|
|
&& (( choice <= ${#CANVASES[@]} )); then
|
|
path="${CANVASES[$((choice-1))]}"
|
|
stem="${path##*/}"
|
|
stem="${stem%.canvas.tsx}"
|
|
export_one "$stem"
|
|
return 0
|
|
fi
|
|
echo "Invalid choice: $choice"
|
|
;;
|
|
esac
|
|
done
|
|
}
|
|
|
|
export_one() {
|
|
# Args: <canvas stem>
|
|
local stem="$1"
|
|
echo
|
|
echo "[$(timestamp)] exporting $stem → $PDF_OUT_DIR/$stem.pdf ..."
|
|
if "${EXPORT_PDF[@]}" "$stem" --output "$PDF_OUT_DIR" --quiet; then
|
|
local pdf="$PDF_OUT_DIR/$stem.pdf"
|
|
echo "[$(timestamp)] done."
|
|
echo
|
|
echo " PDF written to:"
|
|
echo " $pdf"
|
|
if [[ -f "$pdf" ]]; then
|
|
local size
|
|
size="$(wc -c <"$pdf" 2>/dev/null || echo "?")"
|
|
echo " ($size bytes)"
|
|
fi
|
|
else
|
|
local rc=$?
|
|
echo "[$(timestamp)] FAILED (exit $rc)" >&2
|
|
return "$rc"
|
|
fi
|
|
}
|
|
|
|
export_all() {
|
|
echo
|
|
echo "[$(timestamp)] exporting all canvases → $PDF_OUT_DIR/ ..."
|
|
if "${EXPORT_PDF[@]}" --all --output "$PDF_OUT_DIR" --quiet; then
|
|
echo "[$(timestamp)] done."
|
|
echo
|
|
echo " PDFs written to:"
|
|
echo " $PDF_OUT_DIR"
|
|
if [[ -d "$PDF_OUT_DIR" ]]; then
|
|
while IFS= read -r f; do
|
|
echo " - ${f##*/}"
|
|
done < <(ls -1 "$PDF_OUT_DIR"/*.pdf 2>/dev/null)
|
|
fi
|
|
else
|
|
local rc=$?
|
|
echo "[$(timestamp)] FAILED (exit $rc)" >&2
|
|
return "$rc"
|
|
fi
|
|
}
|
|
|
|
# ── main menu ───────────────────────────────────────────────────────────────
|
|
|
|
main_menu() {
|
|
cat <<'EOF'
|
|
|
|
──────────────────────────────────────────────
|
|
CleverAgents Reports
|
|
──────────────────────────────────────────────
|
|
1) Refresh PR velocity canvas
|
|
2) Refresh milestone completion canvas
|
|
3) Print a report to PDF
|
|
q) Quit
|
|
──────────────────────────────────────────────
|
|
EOF
|
|
}
|
|
|
|
while true; do
|
|
main_menu
|
|
read -rp "Choice: " choice
|
|
case "$choice" in
|
|
1) refresh_submenu "PR velocity" "${RENDER_PR_VELOCITY[@]}" ;;
|
|
2) refresh_submenu "milestone completion" "${RENDER_MILESTONES[@]}" ;;
|
|
3) pdf_submenu ;;
|
|
q|Q) exit 0 ;;
|
|
"") continue ;;
|
|
*) echo "Invalid choice: $choice" ;;
|
|
esac
|
|
done
|