Files
cleveragents-core/tools/render-milestones.py
T
drew 816b93953d fix(auto-agents): pre-launch fork environment fixes from prelaunch audit
Resolves all blockers identified in the upstream/fork prelaunch audit
so the deterministic merge driver can be exercised end-to-end against
drew/cleveragents-core sentinels without silently hitting upstream or
hanging on perpetually-failing CI.

Code changes
------------

- tools/launch_fork.sh (new): source-able preamble that pins
  FORGEJO_OWNER, FORGEJO_REPO, FORGEJO_URL, FORGEJO_API_BASE,
  FORGEJO_DEFAULT_BRANCH, FORGEJO_PAT, FORGEJO_ORG and validates the
  fork target via GET /repos/<owner>/<repo> (must exist, be a fork,
  grant push). Refuses to export anything on validation failure so
  the parent shell isn't half-configured. Closes the silent
  auto-detect risk where launching auto-agents.md from the canonical
  clone (whose origin points at upstream) would otherwise pick up
  upstream as the target.

- tools/flag_stale_prs.py: DEFAULT_ASSIGNEE is now env-driven via
  FORGEJO_DEFAULT_ASSIGNEE (default "freemo" preserves canonical
  behaviour). Fork-mode runs can set FORGEJO_DEFAULT_ASSIGNEE=drew
  once instead of remembering --assignee on every invocation.

- tools/{_forgejo_cache,count-master-merges,pr-stats,render-milestones}.py:
  read FORGEJO_OWNER / FORGEJO_REPO / FORGEJO_API_BASE with defaults
  matching the rest of the pipeline. _forgejo_cache.py partitions
  its SQLite path per-(owner, repo) so a fork-mode reporting pass
  never clobbers the canonical cache. The default
  cleveragents/cleveragents-core keeps the historical filename
  forgejo.sqlite for backward compatibility; non-default targets
  land at forgejo.<safe-owner>.<safe-repo>.sqlite.

- .opencode/skills/supervised-workers/scripts/{submit_review,
  submit_comment,fetch_pr_stale,fetch_pr_not_stale}.ts: deleted.
  These were orphaned LLM-session snapshot scripts referenced by no
  agent or skill, and they contained a hard-coded leaked Forgejo
  PAT. NOTE: file removal does not invalidate the token; the
  operator must rotate it on Forgejo separately.

Environment changes (applied via API; not in this commit)
---------------------------------------------------------

- Fork .forgejo/workflows/{benchmark-scheduled,ci,master,release}.yml
  synced to upstream HEAD via 4 API content-PUTs. The fork was 2
  days behind upstream and used a ${{vars.docker_prefix}} template
  that resolved to a non-pullable image, so every CI run failed in
  30-40 s.

- Fork ci.yml subsequently patched to drop the push-validation
  job: it requires secrets.FORGEJO_TOKEN, but Forgejo blocks
  creation of any secret whose name begins with FORGEJO_ via the
  API on personal forks (HTTP 400 "invalid secret name"); the
  merge driver pushes from outside CI in fork-mode anyway, so
  this validation is moot here. status-check.needs and conditional
  updated to drop the dependency. Net effect: required-status-check
  contexts (build, coverage, docker, integration_tests, lint,
  quality, security, typecheck, unit_tests, e2e_tests) now post
  real terminal states instead of being cancelled when
  push-validation aborted the whole workflow at 92 s.

- Fork branch protection PATCHed apply_to_admins=true. The fork
  was inheriting Forgejo's default of false, which would have let
  the merge driver bypass branch protection (it authenticates as
  drew, who is the fork admin). Production fidelity restored.

- Fork collaborators: HAL9000 added with write permission. The
  reviewer-identity slot (FORGEJO_REVIEWER_PAT) needs a PAT
  belonging to a different account than the merge driver since
  Forgejo blocks self-approval. Verified write permission via
  /repos/.../collaborators/HAL9000/permission.

Validation
----------

- 180/180 auto_agents unit tests pass.
- launch_fork.sh tested for: success path (sourced), failure path
  (sourced — no env vars leak when validation fails), and
  executed-not-sourced path (warns appropriately).
- Fork CI confirmed structurally healthy after the workflow patches:
  fresh run posts real statuses, 6/10 required gates already
  passing on commit 38fa7765 with the rest progressing normally
  (no fast failures).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 22:11:59 -04:00

971 lines
34 KiB
Python
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
"""Render milestone-completion.canvas.tsx from the live Forgejo milestones API.
This is the canonical, LLM-free way to refresh the milestone-completion
canvas. It pulls milestones + per-milestone recent-closed issues + open PR
counts straight from Forgejo, computes a close velocity over a configurable
trailing window (default 14 days), projects completion dates deterministically
from open / velocity, and fills a template with narrative auto-commentary
(overrideable via a TOML notes file).
Usage
-----
# Refresh the canvas in place (14-day velocity window):
python3 tools/render-milestones.py
# Use a shorter window for a more current signal:
python3 tools/render-milestones.py --window-days 7
# Write to stdout instead of the canvas path:
python3 tools/render-milestones.py --output -
# Override intro / callout / findings via a TOML notes file:
python3 tools/render-milestones.py --notes tools/milestones-notes.toml
Notes file format (all keys optional)
-------------------------------------
# tools/milestones-notes.toml
[notes]
intro_body = "Custom intro paragraph..."
projections_callout_title = "Read before interpreting projections"
projections_callout_body = "Custom caveats paragraph..."
key_findings_jsx = \"\"\"... raw JSX ...\"\"\"
footer_tail = "Additional context line for the footer."
# Per-milestone description / theme overrides (by milestone id, e.g. "v3.2.0"):
[notes.descriptions."v3.2.0"]
theme = "One-liner override..."
description = "Paragraph override..."
Any slot not set in the TOML falls back to a deterministic auto-generated
sentence / paragraph / JSX block computed purely from the numbers.
Data sources
------------
- Milestones + descriptions: GET /repos/{owner}/{repo}/milestones?state=all
- Per-milestone closed issues within window: GET /repos/{owner}/{repo}/issues
?type=issues&state=closed&milestones=<title>&sort=newest&limit=50&page={n}
(paginates until closed_at < cutoff)
- Open PR counts per milestone: GET /repos/{owner}/{repo}/pulls?state=open
Nothing in this script contacts an LLM. The entire canvas, including
narrative commentary and per-milestone descriptions, is deterministic given
(a) the API state at run time and (b) the optional TOML notes file.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
import time
import urllib.request
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any
try:
import tomllib
except ModuleNotFoundError: # Python < 3.11
tomllib = None # type: ignore
TOOLS_DIR = Path(__file__).resolve().parent
REPO_ROOT = TOOLS_DIR.parent
TEMPLATE_PATH = TOOLS_DIR / "milestones.canvas.template.tsx"
DEFAULT_NOTES_PATH = TOOLS_DIR / "milestones-notes.toml"
DEFAULT_OUTPUT_PATH = (
Path.home()
/ ".cursor/projects/home-drew-repos-cleveragents-core/canvases/milestone-completion.canvas.tsx"
)
# Env-driven so fork-mode reporting (`FORGEJO_OWNER=drew` etc.) renders
# from the fork's milestones. Defaults preserve canonical-mode behaviour.
API_BASE = os.environ.get(
"FORGEJO_API_BASE", "https://git.cleverthis.com/api/v1"
).rstrip("/")
REPO_OWNER = os.environ.get("FORGEJO_OWNER", "cleveragents")
REPO_NAME = os.environ.get("FORGEJO_REPO", "cleveragents-core")
REPO = f"{REPO_OWNER}/{REPO_NAME}"
# ── defaults: curated one-liner "theme" + longer "description" per milestone ──
# Pulled from docs/specification.md and the milestone bodies in Forgejo.
# Override any slot via tools/milestones-notes.toml → [notes.descriptions."<id>"].
DEFAULT_DESCRIPTIONS: dict[str, dict[str, str]] = {
"v3.0.0": {
"theme": (
"Minimal local-mode flow: register actions, link git resources, "
"run plans end-to-end (use → execute → diff → apply) with a "
"sandboxed worktree."
),
"description": (
"Persisted action/plan records in SQLite with Alembic migrations, "
"actor-based LLM execution path, tool-driven ChangeSet, "
"git-worktree sandbox, Apply stage committing into the target "
"repo, and ≥97% test coverage."
),
},
"v3.1.0": {
"theme": (
"Actor YAML compiles into live LangGraph StateGraphs; tool "
"router, MCP adapter, and validation runner go fully operational."
),
"description": (
"Pydantic-validated actor YAML (llm / tool / graph), GRAPH-type "
"actors compiling into LangGraph with subgraph resolution and "
"cycle detection, MCP adapter discovering external tool servers, "
"validation runner enforcing required/informational modes, and "
"multi-file ChangeSet generation."
),
},
"v3.2.0": {
"theme": (
"Record decisions during Strategize/Execute, expose them through "
"plan tree/explain, manage invariants, and support plan "
"correction with selective subtree recomputation."
),
"description": (
"12 decision types with context snapshots, influence DAG for "
"selective recomputation, invariant precedence chain (plan > "
"action > project > global), revert/append correction engine "
"with BFS impact analysis, validation pipeline distinguishing "
"required (blocking) from informational modes, DoD gating on "
"plan apply."
),
},
"v3.3.0": {
"theme": (
"Plans spawn child plans, execute in parallel, merge results via "
"three-way merge; checkpointing enables rollback to prior plan "
"states."
),
"description": (
"subplan_spawn / subplan_parallel_spawn decisions with "
"configurable concurrency, sequential/parallel/dependency-"
"ordered execution, three-way/sequential/last-wins merge "
"strategies, revert/append corrections, automatic checkpoint "
"intervals, phase reversion (constrained apply → Strategize), "
"and plan.resume with state recovery."
),
},
"v3.4.0": {
"theme": (
"Advanced Context Management System v1: project-scale indexing, "
"demand-driven context request protocol, hot/warm/cold tiering, "
"and pluggable strategies."
),
"description": (
"Context assembly pipeline (10 pluggable components across "
"Strategy Orchestration, Fragment Fusion, Context Finalization), "
"repo indexing >10k files, UKO Layers 01, parallel strategies "
"(keyword, semantic, graph), skeleton compressor for child-plan "
"context, project-scoped config service, context CLI commands."
),
},
"v3.5.0": {
"theme": (
"System executes a large-scale autonomous task with 4+ level "
"subplan hierarchy, 10+ parallel subplans, and validation-gated "
"apply."
),
"description": (
"Hierarchical plan decomposition, parallel scaling, 8 built-in "
"automation profiles composing safety profiles (sandbox/"
"checkpoint/unsafe-tool gating, cost/retry limits), estimation "
"actor producing cost/risk forecasts, A2A local facade "
"(groundwork for server mode), and an LSP server stub with "
"JSON-RPC stdio transport."
),
},
"v3.6.0": {
"theme": (
"Advanced capabilities beyond the MVP core: extra LLM backends, "
"cloud/database/virtual resources, container tool execution, "
"ACP→A2A rename, and UKO Layers 23."
),
"description": (
"Advanced context strategies, cost/session budgets, pluggable "
"scope-chain resolution, devcontainer-based tool execution, E2E "
"workflow spec tests, wiring of all 38 domain events, FAISS/"
"Tantivy search backends. ADRs 042, 043, 047."
),
},
"v3.7.0": {
"theme": (
"Textual-based TUI with MainScreen, persona system, reference/"
"command input (@, /, !), multi-session tabs, theming, and "
"TuiMaterializer A2A bridge."
),
"description": (
"3-state sidebar (conversation/permissions/settings), YAML "
"personas with import/export, SQLite session persistence, "
"Dracula theme, first-run wizard, shell danger detection, "
"permission question widget, Prometheus metrics export, session "
"JSON+Markdown export, ACMS graph backend, PostgreSQL & Helm. "
"ADRs 044046."
),
},
"v3.8.0": {
"theme": (
"Server mode via A2A (Agent-to-Agent) protocol, LangGraph "
"Platform RemoteGraph execution, authn/authz, entity sync, and "
"container/K8s deploys."
),
"description": (
"A2A JSON-RPC 2.0 wire format with _cleveragents/ extension "
"methods, stdio (local) and HTTP (server) transports, FastAPI + "
"A2A SDK server, API-token + team RBAC, PostgreSQL backend, "
"Docker/Kubernetes/Helm. ADRs 047048."
),
},
"v3.9.0": {
"theme": (
"Lowest-priority cleanup milestone for documentation "
"improvements and incremental feature polish."
),
"description": (
"Documentation updates and non-blocking feature refinements "
"that accumulate outside the M1M9 roadmap."
),
},
}
# Milestone titles. Forgejo only stores the version ("v3.2.0") — the "M3 —
# <name>" labels are curated here.
DEFAULT_MILESTONE_TITLES: dict[str, str] = {
"v3.0.0": "M1 — Local Source-Code Workflow",
"v3.1.0": "M2 — Actor Compiler + LLM Integration",
"v3.2.0": "M3 — Decisions + Validations + Invariants",
"v3.3.0": "M4 — Corrections + Subplans + Checkpoints",
"v3.4.0": "M5 — ACMS v1 + Context Scaling",
"v3.5.0": "M6 — Autonomy Hardening",
"v3.6.0": "M7 — Advanced Concepts & Deferred Features",
"v3.7.0": "M8 — TUI Implementation",
"v3.8.0": "M9 — Server Implementation",
"v3.9.0": "Documentation + Feature Updates",
}
# ── auth & HTTP ─────────────────────────────────────────────────────────────
def _load_token() -> str:
env_path = REPO_ROOT / ".devcontainer" / ".env"
if env_path.exists():
for line in env_path.read_text().splitlines():
if line.startswith("GITEA_TOKEN"):
_, _, val = line.partition("=")
return val.strip().strip('"').strip("'")
import os
tok = os.environ.get("GITEA_TOKEN")
if not tok:
raise RuntimeError(
"GITEA_TOKEN not found in .devcontainer/.env or environment"
)
return tok
def _api(path: str, token: str, retries: int = 3) -> Any:
url = f"{API_BASE}{path}"
req = urllib.request.Request(url, headers={"Authorization": f"token {token}"})
last_err: Exception | None = None
for attempt in range(retries):
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read())
except Exception as e: # noqa: BLE001
last_err = e
if attempt < retries - 1:
time.sleep(2 ** attempt)
raise RuntimeError(f"API request failed: {url}: {last_err}")
def _paginate(path_no_page: str, token: str) -> list[Any]:
out: list[Any] = []
page = 1
while True:
sep = "&" if "?" in path_no_page else "?"
page_data = _api(f"{path_no_page}{sep}page={page}", token)
if not page_data:
break
out.extend(page_data)
if len(page_data) < 50:
break
page += 1
if page > 200: # hard safety cap
break
return out
# ── core computations ───────────────────────────────────────────────────────
def _parse_iso(s: str | None) -> datetime | None:
if not s:
return None
try:
dt = datetime.fromisoformat(s.replace("Z", "+00:00"))
except ValueError:
return None
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
def fetch_milestones(token: str) -> list[dict]:
return _paginate(f"/repos/{REPO}/milestones?state=all&limit=50", token)
def fetch_open_prs_by_milestone(token: str) -> tuple[int, dict[str, int]]:
"""Return (total_open_prs, {milestone_title: open_pr_count})."""
prs = _paginate(f"/repos/{REPO}/pulls?state=open&limit=50", token)
by_ms: dict[str, int] = {}
for pr in prs:
ms = pr.get("milestone") or {}
title = ms.get("title") or ""
if title:
by_ms[title] = by_ms.get(title, 0) + 1
return len(prs), by_ms
def closed_in_window(
token: str, milestone_title: str, cutoff: datetime
) -> int:
"""Count issues closed for this milestone whose closed_at >= cutoff.
Paginates sort=newest (by closed_at desc) and stops when it sees an issue
that closed before cutoff.
"""
page = 1
count = 0
while True:
# urlencode minimal: milestones title can contain "."
path = (
f"/repos/{REPO}/issues?type=issues&state=closed"
f"&milestones={urllib.request.quote(milestone_title)}"
f"&sort=newest&limit=50&page={page}"
)
data = _api(path, token)
if not data:
break
stop = False
for issue in data:
dt = _parse_iso(issue.get("closed_at"))
if dt is None:
continue
if dt < cutoff:
stop = True
continue
count += 1
if stop or len(data) < 50:
break
page += 1
if page > 50:
break
return count
# ── classification & projection ─────────────────────────────────────────────
def classify_risk(
*,
state: str,
days_overdue: int | None,
pct: float,
open_count: int,
is_docs_bucket: bool,
) -> str:
"""Deterministic risk classification from the numbers.
- done: Forgejo state = closed.
- low: open docs/feature-updates bucket (e.g. v3.9.0).
- critical: overdue by >= 45 days AND <40% complete.
- high: everything else that's open.
"""
if state == "closed":
return "done"
if is_docs_bucket:
return "low"
if days_overdue is not None and days_overdue >= 45 and pct < 40.0:
return "critical"
# Very-large-scope milestones with low progress are also critical even if
# not heavily overdue yet (catches big M3/M6-style buckets before the date
# slips far past).
if open_count >= 1000 and pct < 25.0:
return "critical"
return "high"
def project_completion(
now: datetime,
open_count: int,
velocity_per_day: float,
) -> tuple[str, int | None]:
"""Return (label, days_to_finish). Label is a YYYY-MM-DD or 'Stalled'."""
if velocity_per_day <= 0 or open_count <= 0:
if open_count == 0:
return ("Done", 0)
return ("Stalled", None)
days = open_count / velocity_per_day
eta = now + timedelta(days=days)
return (eta.date().isoformat(), int(round(days)))
def build_projection_note(
*,
state: str,
due_dt: datetime | None,
closed_in_window_count: int,
window_days: int,
open_count: int,
closed_at: datetime | None,
now: datetime,
) -> str | None:
"""Short italic note shown in the table/detail cards."""
if state == "closed":
if due_dt and closed_at:
late_days = (closed_at.date() - due_dt.date()).days
if late_days > 0:
return f"Closed {late_days} days late"
if late_days < 0:
return f"Closed {-late_days} days early"
return "Closed on schedule"
return None
parts: list[str] = []
if due_dt:
overdue_days = (now.date() - due_dt.date()).days
if overdue_days > 0:
parts.append(f"{overdue_days} days overdue")
if closed_in_window_count == 0 and open_count > 0:
parts.append(
f"0 closes in last {window_days}d — no forward motion"
)
else:
vel = closed_in_window_count / max(window_days, 1)
parts.append(
f"{closed_in_window_count} closes in last {window_days}d "
f"({vel:.2f}/day)"
)
return "; ".join(parts) if parts else None
# ── narrative auto-commentary ───────────────────────────────────────────────
def narrate(
milestones: list[dict],
totals: dict,
now: datetime,
window_days: int,
) -> dict[str, str]:
closed_ms = [m for m in milestones if m["risk"] == "done"]
active_ms = [m for m in milestones if m["risk"] != "done"]
critical_ms = [m for m in milestones if m["risk"] == "critical"]
stalled_ms = [
m for m in active_ms if m["projectedDate"] == "Stalled"
]
closed_in_window_total = sum(m["closedInWindow"] for m in milestones)
project_velocity = closed_in_window_total / max(window_days, 1)
if project_velocity > 0:
days_to_clear = totals["open_issues"] / project_velocity
years_to_clear = days_to_clear / 365.25
clear_phrase = (
f"{years_to_clear:.1f} years to close at the current pace"
)
else:
clear_phrase = "no forward motion at the current pace"
intro_body = (
f"As of {totals['as_of_date']}. Issue counts are live from Forgejo; "
f"projected completion dates are computed from each milestone's "
f"trailing {window_days}-day close velocity applied to its current "
f"open count. "
f"{len(closed_ms)} of {len(milestones)} milestones are closed; the "
f"remaining {len(active_ms)} are open, "
f"{len(critical_ms)} of them flagged critical."
+ (
f" {len(stalled_ms)} milestone"
f"{'s' if len(stalled_ms) != 1 else ''} "
f"closed zero issues in the last {window_days} days and "
f"{'are' if len(stalled_ms) != 1 else 'is'} labelled "
f"Stalled rather than given a calendar ETA."
if stalled_ms
else ""
)
)
projections_callout_title = "Read before interpreting projections"
projections_callout_body = (
f"Projected completion dates use each milestone's last-"
f"{window_days}-day close rate as a linear forecast against today's "
f"open count. Caveats: (1) a {window_days}-day window is a rolling "
f"average — milestones showing zero closes are idle right now, not "
f"necessarily abandoned; (2) scope is still expanding on the biggest "
f"milestones (the timeline doc logged +110 new open issues vs. +2 "
f"closed in a single 60-minute window on Day 103), so even the "
f"non-stalled ETAs are optimistic; (3) CI has not been running on "
f"master since 2026-03-14 (issue #8508), so closure quality is not "
f"being gated by the test suite."
)
# Key findings — build paragraphs from the data, emit as JSX.
by_id = {m["id"]: m for m in milestones}
findings: list[str] = []
findings.append(
f"Only {closed_in_window_total:,} issues closed across the entire "
f"project in the last {window_days} days. Against "
f"{totals['open_issues']:,} currently-open issues that is "
f"{clear_phrase}."
)
if stalled_ms:
names = ", ".join(
f"{m['id']} ({m['title'].split('',1)[-1].strip().split(' ')[0]})"
for m in stalled_ms
)
findings.append(
f"{len(stalled_ms)} active milestone"
f"{'s are' if len(stalled_ms) != 1 else ' is'} fully stalled: "
f"{names}. Zero closes in the window against a combined "
f"{sum(m['open'] for m in stalled_ms):,} open issues."
)
# Biggest-backlog call-out
largest = sorted(active_ms, key=lambda m: m["open"], reverse=True)[:2]
if largest:
seg = []
for m in largest:
seg.append(
f"{m['id']} ({m['pct']:.1f}% complete, "
f"{m['open']:,} open)"
)
findings.append(
"Scope is concentrated in a few milestones: "
+ " and ".join(seg)
+ ". These dominate the roadmap's remaining work and set the "
"floor on any reasonable completion date."
)
if closed_ms:
closed_lines = []
for m in closed_ms:
note = m.get("projectionNote") or ""
closed_lines.append(
f"{m['id']} ({m['title']}{note.lower()})"
)
findings.append(
"Closed milestones: "
+ "; ".join(closed_lines)
+ ". Every remaining milestone with a due date is overdue."
)
findings.append(
f"{totals['total_open_prs']} total open PRs across the repo "
f"({totals['open_prs_unassigned']} unassigned to any milestone). "
f"Merging these would move the top-level numbers fastest on the "
f"heaviest milestones."
)
findings.append(
"CI outage since 2026-03-14 (issue #8508) means closed issues are "
"landing without the regression safety net the spec's ≥97% coverage "
"criterion presumes. Restoring CI is a prerequisite for trusting any "
"closure numbers above."
)
key_findings_jsx = "\n".join(_finding_jsx(p) for p in findings)
footer_tail = (
f"Project-wide {window_days}-day velocity: "
f"{project_velocity:.2f} closes/day."
)
return {
"intro_body": _escape_jsx_text(intro_body),
"projections_callout_title": _escape_jsx_attr(projections_callout_title),
"projections_callout_body": _escape_jsx_text(projections_callout_body),
"key_findings_jsx": key_findings_jsx,
"footer_tail": _escape_jsx_text(footer_tail),
}
def _finding_jsx(paragraph: str) -> str:
"""Split a paragraph into bold-lead + body Text JSX.
The first sentence (up to the first period) is bolded; the rest follows
as the body. Safe-encoded for JSX.
"""
# Find first sentence boundary
m = re.match(r"^(.+?[.!?])\s+(.*)$", paragraph.strip(), re.DOTALL)
if m:
lead = m.group(1).strip()
body = m.group(2).strip()
else:
lead = paragraph.strip()
body = ""
lead_esc = _escape_jsx_text(lead)
if body:
body_esc = _escape_jsx_text(body)
return (
f" <Text>\n"
f" <Text as=\"span\" weight=\"semibold\">{lead_esc}</Text>"
f"{{\" \"}}{body_esc}\n"
f" </Text>"
)
return (
f" <Text>\n"
f" <Text as=\"span\" weight=\"semibold\">{lead_esc}</Text>\n"
f" </Text>"
)
def _escape_jsx_text(s: str) -> str:
"""Escape a string for safe interpolation inside JSX text content.
JSX text is sensitive to `{`, `}`, and HTML special chars. We replace:
- `>` with `&gt;` (to keep parser happy when people write "> 14d")
- `<` with `&lt;` (rare, but safe)
- `{` / `}` with their unicode escape forms so JSX won't interpret them
- `&` is left alone when followed by a safe entity shape; else encoded
"""
s = s.replace("&", "&amp;")
s = s.replace("<", "&lt;").replace(">", "&gt;")
s = s.replace("{", "\u007B").replace("}", "\u007D")
# Restore common named entities we just double-escaped (&amp;gt; etc.) — we
# didn't insert any, but be defensive:
s = s.replace("&amp;amp;", "&amp;")
return s
def _escape_jsx_attr(s: str) -> str:
return s.replace("\\", "\\\\").replace('"', '\\"')
# ── template rendering ──────────────────────────────────────────────────────
def render(data: dict, notes: dict[str, Any]) -> str:
template = TEMPLATE_PATH.read_text()
narrative = narrate(
data["milestones"], data["totals"], data["now"], data["window_days"]
)
# Apply simple slot overrides
for key in (
"intro_body",
"projections_callout_title",
"projections_callout_body",
"key_findings_jsx",
"footer_tail",
):
if key in notes and isinstance(notes[key], str):
narrative[key] = notes[key] if key == "key_findings_jsx" else (
_escape_jsx_text(notes[key])
if key != "projections_callout_title"
else _escape_jsx_attr(notes[key])
)
generator_note = (
f"rendered at "
f"{datetime.now(timezone.utc).isoformat(timespec='seconds')} UTC "
f"by tools/render-milestones.py (window_days={data['window_days']})"
)
# Build MILESTONES JSON — drop internal-only fields Python uses for
# narrative building.
export_keys = [
"id",
"title",
"theme",
"description",
"dueDate",
"projectedDate",
"projectionNote",
"closed",
"open",
"openPRs",
"pct",
"velocityPerDay",
"closedInWindow",
"status",
"risk",
]
export_rows = []
for m in data["milestones"]:
row = {k: m.get(k) for k in export_keys}
# Skip "projectionNote" when falsy, to leave it as optional
if row["projectionNote"] is None:
row.pop("projectionNote")
export_rows.append(row)
subs: dict[str, str] = {
"GENERATOR_NOTE": generator_note,
"UPDATED_AT": data["updated_at_label"],
"AS_OF_DATE": data["totals"]["as_of_date"],
"WINDOW_DAYS": str(data["window_days"]),
"MILESTONES_JSON": json.dumps(export_rows, indent=2),
"INTRO_BODY": narrative["intro_body"],
"PROJECTIONS_CALLOUT_TITLE": narrative["projections_callout_title"],
"PROJECTIONS_CALLOUT_BODY": narrative["projections_callout_body"],
"KEY_FINDINGS_JSX": narrative["key_findings_jsx"],
"FOOTER_TAIL": narrative["footer_tail"],
}
rendered = template
for k, v in subs.items():
rendered = rendered.replace(f"{{{{{k}}}}}", v)
leftover = re.findall(r"\{\{[A-Z_]+\}\}", rendered)
if leftover:
raise RuntimeError(
f"Unfilled template placeholders: {sorted(set(leftover))}"
)
return rendered
# ── notes loading ───────────────────────────────────────────────────────────
def load_notes(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
if tomllib is None:
print(f"# WARN: tomllib unavailable; ignoring {path}", file=sys.stderr)
return {}
with path.open("rb") as f:
raw = tomllib.load(f)
notes = raw.get("notes") or {}
return dict(notes)
# ── orchestration ───────────────────────────────────────────────────────────
def gather_data(
token: str,
*,
now: datetime,
window_days: int,
log,
) -> dict:
cutoff = now - timedelta(days=window_days)
log(f"fetching milestones ...")
ms_raw = fetch_milestones(token)
log(f" got {len(ms_raw)} milestones")
log(f"fetching open PRs (for per-milestone PR counts) ...")
total_open_prs, open_prs_by_ms = fetch_open_prs_by_milestone(token)
log(f" {total_open_prs} open PRs total")
milestones: list[dict] = []
for ms in ms_raw:
title_version = ms.get("title") or ""
if not title_version:
continue
state = ms.get("state") or ""
open_count = int(ms.get("open_issues") or 0)
closed_count = int(ms.get("closed_issues") or 0)
total = open_count + closed_count
pct = (100.0 * closed_count / total) if total else 0.0
due_dt = _parse_iso(ms.get("due_on"))
closed_at = _parse_iso(ms.get("closed_at"))
is_docs_bucket = title_version == "v3.9.0"
if state == "open":
log(
f" {title_version}: counting closes in last "
f"{window_days}d ..."
)
closed_window = closed_in_window(token, title_version, cutoff)
velocity_per_day = closed_window / max(window_days, 1)
else:
closed_window = 0
velocity_per_day = 0.0
projected, _ = project_completion(now, open_count, velocity_per_day)
# For closed milestones, show the actual closed_at date in the
# "Projected" slot (so the table reads consistently) — the table
# label for closed rows says "Actually closed".
if state == "closed" and closed_at:
projected = closed_at.date().isoformat()
days_overdue = (
(now.date() - due_dt.date()).days if (due_dt and state == "open") else None
)
risk = classify_risk(
state=state,
days_overdue=days_overdue,
pct=pct,
open_count=open_count,
is_docs_bucket=is_docs_bucket,
)
status = (
"Closed"
if state == "closed"
else (
f"Open — {days_overdue}d overdue"
if days_overdue and days_overdue > 0
else (
"Open — no deadline"
if due_dt is None
else f"Open — due {due_dt.date().isoformat()}"
)
)
)
projection_note = build_projection_note(
state=state,
due_dt=due_dt,
closed_in_window_count=closed_window,
window_days=window_days,
open_count=open_count,
closed_at=closed_at,
now=now,
)
desc = DEFAULT_DESCRIPTIONS.get(title_version, {})
milestones.append(
{
"id": title_version,
"title": DEFAULT_MILESTONE_TITLES.get(
title_version, title_version
),
"theme": desc.get(
"theme",
(ms.get("description") or "").splitlines()[0]
if ms.get("description")
else "",
),
"description": desc.get(
"description",
(ms.get("description") or "").strip(),
),
"dueDate": due_dt.date().isoformat() if due_dt else None,
"projectedDate": projected,
"projectionNote": projection_note,
"closed": closed_count,
"open": open_count,
"openPRs": open_prs_by_ms.get(title_version, 0),
"pct": round(pct, 1),
"velocityPerDay": round(velocity_per_day, 2),
"closedInWindow": closed_window,
"status": status,
"risk": risk,
}
)
# Sort milestones by id (v3.0.0 < v3.1.0 < ... < v3.9.0). Natural string
# sort works here because versions are all v3.<digit>.0.
milestones.sort(key=lambda m: m["id"])
# Totals
total_issues = sum(m["open"] + m["closed"] for m in milestones)
total_closed = sum(m["closed"] for m in milestones)
total_open = sum(m["open"] for m in milestones)
total_open_prs_in_ms = sum(m["openPRs"] for m in milestones)
unassigned_prs = max(total_open_prs - total_open_prs_in_ms, 0)
totals = {
"as_of_date": now.date().isoformat(),
"total_issues": total_issues,
"closed_issues": total_closed,
"open_issues": total_open,
"total_open_prs": total_open_prs,
"open_prs_unassigned": unassigned_prs,
}
updated_at_label = now.strftime("%Y-%m-%d %H:%M UTC")
return {
"now": now,
"window_days": window_days,
"milestones": milestones,
"totals": totals,
"updated_at_label": updated_at_label,
}
# ── CLI ─────────────────────────────────────────────────────────────────────
def parse_args() -> argparse.Namespace:
p = argparse.ArgumentParser(description=__doc__.split("\n\n", 1)[0])
p.add_argument(
"--output",
default=str(DEFAULT_OUTPUT_PATH),
help="Output path for the rendered .tsx. Use '-' for stdout.",
)
p.add_argument(
"--notes",
default=str(DEFAULT_NOTES_PATH),
help="Path to TOML notes file (optional).",
)
p.add_argument(
"--window-days",
type=int,
default=14,
help="Trailing window (in days) used to compute close velocity. "
"Default: 14.",
)
p.add_argument(
"--as-of",
default=None,
help="Override the reference timestamp as an ISO string "
"(e.g. 2026-04-26T12:00:00Z). Defaults to now (UTC).",
)
p.add_argument(
"--quiet", action="store_true", help="Suppress progress messages."
)
return p.parse_args()
def main() -> None:
args = parse_args()
def log(msg: str) -> None:
if not args.quiet:
print(f"# {msg}", file=sys.stderr)
token = _load_token()
if args.as_of:
now = _parse_iso(args.as_of)
if now is None:
raise SystemExit(f"Could not parse --as-of: {args.as_of}")
else:
now = datetime.now(timezone.utc)
log(f"reference time: {now.isoformat()}; window = {args.window_days} days")
data = gather_data(token, now=now, window_days=args.window_days, log=log)
notes = load_notes(Path(args.notes))
if notes:
log(f"loaded {len(notes)} note override(s) from {args.notes}")
output = render(data, notes)
if args.output == "-":
sys.stdout.write(output)
else:
out_path = Path(args.output)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(output)
log(f"wrote {len(output):,} chars to {out_path}")
if __name__ == "__main__":
main()