Files
cleveragents-core/tools/_mcp_common.py
T
drew dc4e9368f2 feat(auto-agents): block-store substrate + DRY refactors
Externalise large prompt sections (PR diff, comments, CI failure
logs, reviews, linked issues) into a cross-process SQLite-backed
block store so the worker can recover original content when an
intermediate tier-* agent's summarisation strips inline sections.

New substrate
-------------

- ``tools/_block_store.py`` — SQLite WAL, per-row 1MB cap, 1h TTL,
  janitor (startup + opportunistic per-hour in register()),
  threading.Lock around the periodic-sweep gate.
- ``tools/_block_prompt.py`` — registration glue + ``## Available
  blocks`` Markdown table renderer.
- ``tools/_prefetch_section.py`` — single-source-of-truth section
  registry; collapses the duplicate ``_register_*_blocks`` helpers
  the reviewer and implementer previously kept in lockstep.
- ``tools/mcp_block_store_server.py`` — FastMCP wrapper exposing
  ``block_fetch`` / ``block_list`` / ``block_register`` /
  ``block_invalidate`` to agents. Uses the expanded ``_mcp_common``
  helpers.
- ``tools/_implementer_escalation_helpers.py`` — pure helpers
  extracted from ``dispatch_implementer.py`` (~180 lines off the
  3284-line file); takes ``claim_runtime`` as a kwarg for clean DI.

DRY refactors
-------------

- ``tools/_backoff.py`` — shared ``Backoff`` dataclass collapses the
  three near-identical exponential-backoff state machines in
  ``_pr_comments_cache``, ``_ci_logs``, ``_pr_classification_cache``.
- ``tools/_mcp_common.py`` — expanded with ``bootstrap_loader``,
  ``error_envelope``, ``make_main`` so each MCP server's prelude is
  three lines.
- ``tools/_pr_diff.build_diff_section_full`` — returns a 4-tuple
  including the raw diff body so the reviewer's block-store
  registration reuses the bytes instead of doing a second HTTP fetch.

Wiring
------

- ``_review_prompt.build_review_prompt`` builds a ``PrefetchSection``
  registry via ``_review_sections``, registers them, and renders the
  ``## Available blocks`` table at the end of the prompt.
- ``_implementer_prefetch._fetch_pr_context`` /
  ``fetch_new_issue_context`` build the equivalent registry via
  ``_implementer_sections`` and stamp ``result.block_refs`` for the
  prompt builder to read.
- ``_implementer_prompt`` builders include
  ``_build_available_blocks_section(result)`` in all three flows.
- ``dispatch_review.main`` + ``dispatch_implementer.main`` call
  ``_block_store.janitor()`` at startup; the per-call opportunistic
  janitor in ``register()`` keeps the file bounded between restarts.

Agent contract updates
----------------------

- ``.opencode/agents/task-implementor.md`` +
  ``.opencode/agents/pr-review-worker.md``:
  - ``block_store*`` permission
  - new "Block-store substrate" paragraph explaining
    ``block_fetch`` / ``block_list`` as the summarisation recovery
    path.

Tests
-----

- ``test_block_store.py`` — 48 tests pinning every public contract
  (register/fetch/list/invalidate/janitor, key whitelist, TTL,
  size cap, WAL durability).
- ``test_block_prompt`` — covered transitively via the e2e test.
- ``test_block_store_recovery_e2e.py`` — builds a real prompt via
  ``build_pr_fix_prompt``, applies a heading-bounded summariser stub
  (``_summarise_inline_sections``), asserts inline content is stripped
  yet block keys survive and ``block_fetch`` recovers original content.
  Plus a ``block_list`` fallback test for the worst case where the
  table itself was summarised away.
- ``test_mcp_block_store_server.py`` — 27 wrapper-contract tests.
- ``test_mcp_block_store_transport.py`` — spawns the actual server
  subprocess via ``mcp.client.stdio`` and exercises the JSON-RPC
  transport round-trip in ~1.5s. Catches FastMCP schema /
  serialisation bugs the in-process tests miss.
- ``test_backoff.py`` — 14 behaviour-focused tests of the shared
  ``Backoff`` curve.
- ``test_pr_comments_cache.py`` + ``test_ci_logs.py`` — deleted the
  now-redundant ``TestComputeNextAttemptAfter`` / ``TestBackoffActive``
  / ``TestBackoffHelpers`` classes; ``test_backoff`` covers them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 22:55:52 -04:00

171 lines
6.9 KiB
Python

"""Shared helpers for the project's MCP servers.
Each ``tools/mcp_*_server.py`` is a per-server process (per the design
in ``docs/development/...`` — graphify, ci, forgejo, git, handoff,
block_store), but they all share:
1. **Forgejo runtime context.** The ``_claim_runtime`` helpers expect
a ``RuntimeContext`` protocol object with ``token``,
``request_timeout_s``, ``api_retries``, ``claim_ttl_seconds``,
``owner``, ``repo``. Three MCPs (``ci``, ``forgejo``, ``git``) all
need to build one from env. Factored out so a PAT rotation or
default-tweak lands in one place.
2. **Identity selection (HAL9000 vs HAL9001).** Most write operations
use the worker identity (``FORGEJO_PAT`` — HAL9000), but the
reviewer's formal review submissions and umbrella-PR approvals use
the reviewer identity (``FORGEJO_REVIEWER_PAT`` — HAL9001). Passing
``reviewer=True`` to :class:`ForgejoCfg` produces a context with
the HAL9001 token while leaving owner/repo/timeouts alone.
3. **Sibling-module loader bootstrap.** Every MCP server inserts
``tools/`` into ``sys.path`` and imports ``load_sibling`` to load
non-package siblings (``_claim_runtime``, ``_review_fetch``, etc.).
:func:`bootstrap_loader` does the path insert + import in one call.
4. **Error envelope shape.** Tools return ``{"error": str, **fields}``
so the agent's parser branches once on a single key. Centralised
here as :func:`error_envelope`.
5. **Process entrypoint.** Every server's ``main()`` is the same
try/run/KeyboardInterrupt/Exception wrapper. :func:`make_main`
builds one bound to the server name for stderr labelling.
Kept deliberately small. If something only one MCP needs, it lives in
that MCP's file — not here.
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from types import ModuleType
from typing import Any, Callable
# ─── Runtime tuning ────────────────────────────────────────────────
# Per-call HTTP timeout for Forgejo API requests issued from the MCP
# servers. The default matches what the dispatchers use for similar
# read paths; bump via env if a slow Forgejo causes spurious tool
# errors during a long worker cycle.
_FORGEJO_TIMEOUT_S = int(os.environ.get("MCP_FORGEJO_TIMEOUT_S", "15"))
_FORGEJO_RETRIES = int(os.environ.get("MCP_FORGEJO_RETRIES", "3"))
class ForgejoCfg:
"""Minimal RuntimeContext satisfying ``_claim_runtime``'s
protocol, plus the ``owner`` / ``repo`` fields the ``_review_*``
helpers read directly.
Constructed per-call (cheap — just env reads) so a PAT rotation
on the dispatcher side is picked up without restarting the MCP
server. The ``reviewer`` flag selects HAL9001's PAT instead of
HAL9000's — used by tools that act on behalf of the umbrella
approver identity (notably the reviewer worker's review-submit
path).
"""
def __init__(self, *, reviewer: bool = False) -> None:
if reviewer:
self.token = os.environ.get("FORGEJO_REVIEWER_PAT", "")
else:
self.token = os.environ.get("FORGEJO_PAT", "")
self.request_timeout_s = _FORGEJO_TIMEOUT_S
self.api_retries = _FORGEJO_RETRIES
# Unused by reads but required by the RuntimeContext protocol.
# Defaults match the implementer dispatcher's own claim_ttl.
self.claim_ttl_seconds = int(os.environ.get("MCP_CLAIM_TTL_S", "7200"))
# ``_review_*`` helpers read these directly off cfg rather
# than the module-level fallbacks in ``_claim_runtime``.
self.owner = os.environ.get("FORGEJO_OWNER", "cleveragents")
self.repo = os.environ.get("FORGEJO_REPO", "cleveragents-core")
def require_token(cfg: ForgejoCfg, identity_label: str) -> str | None:
"""Return ``None`` if ``cfg.token`` is set; otherwise an
error-message string the MCP tool can return verbatim to the
agent. Centralised so every tool prints the same actionable
message ("set X env var") instead of opaque HTTP-401 errors."""
if cfg.token:
return None
env_name = (
"FORGEJO_REVIEWER_PAT" if identity_label == "reviewer" else "FORGEJO_PAT"
)
return (
f"{env_name} not set in MCP environment "
f"(identity={identity_label}). The MCP server inherits env from "
"the OpenCode process that spawned it — restart OpenCode after "
"sourcing tools/launch_fork.sh."
)
# ─── Sibling-module loader ──────────────────────────────────────────
def bootstrap_loader() -> Callable[[str, str], ModuleType]:
"""Insert ``tools/`` into ``sys.path`` and return ``load_sibling``.
Every MCP server starts with the same 3-line dance:
sys.path.insert(0, str(Path(__file__).parent))
from _loader import load_sibling
Wrapped here so a server's prelude becomes one expression::
load_sibling = bootstrap_loader()
_claim_runtime = load_sibling("_claim_runtime", "_claim_runtime.py")
"""
tools_dir = str(Path(__file__).resolve().parent)
if tools_dir not in sys.path:
sys.path.insert(0, tools_dir)
from _loader import load_sibling # noqa: E402 — deliberately late
return load_sibling
# ─── Error envelope ────────────────────────────────────────────────
def error_envelope(error: str, **fields: Any) -> dict[str, Any]:
"""Uniform error envelope shape used by every MCP tool.
Tools return ``{"error": str, **fields}`` so the agent's parser
branches once on the ``error`` key. Tools that return collections
on success include the empty collection in ``fields`` so the
agent's parsing code stays the same on error and success.
"""
return {"error": error, **fields}
# ─── Process entrypoint ────────────────────────────────────────────
def make_main(server: Any, server_name: str) -> Callable[[], int]:
"""Build a ``main()`` for an MCP server bound to ``server``.
Returns a zero-arg function suitable for ``sys.exit(main())``.
Same shape every MCP used to inline: KeyboardInterrupt → 0,
any other Exception → log to stderr + return 1.
``server_name`` appears in the stderr fatal line; OpenCode logs
that to its server output so operators can tell which MCP died.
"""
def main() -> int:
try:
server.run()
except KeyboardInterrupt:
return 0
except Exception as exc: # noqa: BLE001 — top-of-process catch
print(f"{server_name}: fatal: {exc!r}", file=sys.stderr)
return 1
return 0
return main
__all__ = (
"ForgejoCfg",
"bootstrap_loader",
"error_envelope",
"make_main",
"require_token",
)