23a014e402
The production seam that connects the controller's runner to actual
LLM execution. Per-attempt MCP subprocess spawn + OpenCode session
drive + canonical JSON read + strict-parse → output dict.
tools/controller/worker/agent_runner.py:
- production_agent_runner(): the function tested production wires
into run_one_attempt's agent_runner param. Per-attempt lifecycle:
1. Map role → matching MCP module + V1 output contract.
2. mkstemp a canonical-output path; spawn `python -m {mcp_module}`
subprocess with CONTROLLER_CANONICAL_OUTPUT_PATH set in env.
3. Call injected run_opencode_session (tests use JSON-RPC stdin
driver; production wires _opencode_worker.run_session_blocking).
4. Wait for the MCP's finalize_and_emit to write the canonical
JSON to the tempfile.
5. Strict-parse against the role's V1 output model.
6. SIGTERM/grace/SIGKILL the MCP + unlink the tempfile in finally.
- ROLE_TO_MCP_MODULE / ROLE_TO_OUTPUT_MODEL: explicit per-role
dispatch tables. Covers all 5 worker roles.
- Error classification per the WorkerError outcome enum:
- unknown role → ValueError (master bug; not a worker outcome)
- session raises generic Exception → WorkerError(worker-internal-error)
- session raises WorkerLostLock → propagated unchanged
- lost_lock_check returns True after session → WorkerLostLock
- MCP didn't emit canonical output within timeout → WorkerError(
worker-internal-error)
- MCP emitted JSON that fails strict-parse → WorkerError(
contract-violation; per v9 status='failed' policy maps to
workflow STUCK)
tools/controller/mcp/_builder_base.py:
- finalize_and_emit() now writes to CONTROLLER_CANONICAL_OUTPUT_PATH
if set (production subprocess path) or stdout if not (direct-call
test path; capsys captures). Decouples canonical output from the
FastMCP JSON-RPC stdout transport — they would otherwise collide
in the subprocess (both writing to the same stream).
- Atomic file write: open + write + fsync + close.
- Existing test_mcp_builders.py (capsys-based) still passes with the
fallback path; new test_worker_agent_runner.py exercises the
file-based subprocess path with real MCPs.
10 new tests in test_worker_agent_runner.py:
- End-to-end with real MCP subprocesses (implementer-resolved,
reviewer-approve, estimator-tier-recommendation)
- Error paths: unknown role (ValueError), session raises (wrapped
as WorkerError), session raises WorkerLostLock (propagated),
lost_lock_check returns True post-session (raises WorkerLostLock),
no finalize → finalize_timeout → WorkerError
- Role-map sanity: every role has both an MCP module and output model
Total: 376 controller tests; full auto_agents suite 2738 pass.
This commit completes the v1 boundary — the controller now has the
complete stack from V1 contracts through MCP response builders, DB
schema, worker dequeue/heartbeat/runner/workspace, master state
machine + tick + reaper + pickup guard + scheduler + discovery +
Forgejo writes + MERGING + HTTP adapter, AND a production
agent_runner that wires it all to real OpenCode + MCP subprocess
spawning. Phase 1d-3+ enhancements (real OpenCode session wiring
in run_opencode_session) and Phase 2+ migration steps remain.
194 lines
6.7 KiB
Python
194 lines
6.7 KiB
Python
"""Shared base for per-role response-builder MCP servers.
|
|
|
|
Each builder MCP holds per-attempt state in a single module-level
|
|
``BuilderState`` instance (per-attempt subprocess model — see plan
|
|
v9). Tools mutate that state; ``finalize()`` validates and emits
|
|
canonical JSON.
|
|
|
|
This module is helper code, not an MCP server itself. The role-specific
|
|
modules import these helpers, declare their Pydantic output type, and
|
|
wire up tools via FastMCP.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import sys
|
|
import threading
|
|
from dataclasses import dataclass, field
|
|
from datetime import datetime, timezone
|
|
from typing import Any, Callable, TypeVar
|
|
|
|
from pydantic import BaseModel, ValidationError
|
|
|
|
T_Output = TypeVar("T_Output", bound=BaseModel)
|
|
|
|
|
|
@dataclass
|
|
class BuilderState:
|
|
"""Mutable accumulator for an in-flight response build.
|
|
|
|
Holds:
|
|
- ``started``: True after ``{role}_start`` was called (every
|
|
builder enforces ``start`` is the first tool).
|
|
- ``finalized``: True after a successful ``{role}_finalize``.
|
|
Further mutating calls are refused.
|
|
- ``identity``: workflow_id / attempt_id / pr_number — bound at
|
|
start, surfaced in every event log line so per-PR debugging
|
|
via ``grep "pr=N"`` finds builder activity.
|
|
- ``fields``: working dict of model-shaped data the role-specific
|
|
module fills in. Role modules know their model class; this
|
|
class is intentionally untyped here so it can be shared.
|
|
- ``audit``: ordered list of every tool call (name + args summary)
|
|
for ``controller-cli tail-events`` and operator forensics.
|
|
"""
|
|
|
|
started: bool = False
|
|
finalized: bool = False
|
|
identity: dict[str, Any] = field(default_factory=dict)
|
|
fields: dict[str, Any] = field(default_factory=dict)
|
|
audit: list[dict[str, Any]] = field(default_factory=list)
|
|
started_at: datetime | None = None
|
|
_lock: threading.Lock = field(default_factory=threading.Lock)
|
|
|
|
def require_started(self) -> None:
|
|
if not self.started:
|
|
raise BuilderError("call {role}_start(...) before any other tool")
|
|
|
|
def require_not_finalized(self) -> None:
|
|
if self.finalized:
|
|
raise BuilderError("response already finalized; no further mutations allowed")
|
|
|
|
def record(self, tool: str, args: dict[str, Any]) -> None:
|
|
"""Append an audit entry. Args are str-coerced to avoid
|
|
carrying large blobs (e.g. raw_log_excerpt) into the audit
|
|
log; the controller_events sink will summarize."""
|
|
with self._lock:
|
|
self.audit.append(
|
|
{
|
|
"tool": tool,
|
|
"ts": datetime.now(timezone.utc).isoformat(),
|
|
"args_summary": _summarize_args(args),
|
|
}
|
|
)
|
|
|
|
|
|
def _summarize_args(args: dict[str, Any]) -> dict[str, Any]:
|
|
"""Truncate any large string args so the audit log stays small.
|
|
Caps each str field at 200 chars; lists/dicts cap at counts only."""
|
|
out: dict[str, Any] = {}
|
|
for k, v in args.items():
|
|
if isinstance(v, str):
|
|
out[k] = v if len(v) <= 200 else f"{v[:200]}…({len(v)} chars)"
|
|
elif isinstance(v, (list, tuple)):
|
|
out[k] = f"<list len={len(v)}>"
|
|
elif isinstance(v, dict):
|
|
out[k] = f"<dict keys={len(v)}>"
|
|
else:
|
|
out[k] = v
|
|
return out
|
|
|
|
|
|
class BuilderError(ValueError):
|
|
"""Raised when a builder tool refuses a call due to an invariant
|
|
violation. The MCP wrapper converts this to the
|
|
``{"error": str}`` envelope the agent's tool-result parser sees."""
|
|
|
|
|
|
def ok(**fields: Any) -> dict[str, Any]:
|
|
"""Standard success envelope for builder tools."""
|
|
return {"status": "ok", **fields}
|
|
|
|
|
|
def err(msg: str, **fields: Any) -> dict[str, Any]:
|
|
"""Standard error envelope. Builder tools wrap their bodies in
|
|
a try/except BuilderError so the agent sees structured failures."""
|
|
return {"error": msg, **fields}
|
|
|
|
|
|
def call_with_invariants(
|
|
state: BuilderState,
|
|
tool: str,
|
|
fn: Callable[[], dict[str, Any]],
|
|
args: dict[str, Any],
|
|
) -> dict[str, Any]:
|
|
"""Run a tool body with the standard invariant guard rails.
|
|
|
|
Order of checks:
|
|
1. Audit-record the call.
|
|
2. Run ``fn()`` (which may raise ``BuilderError``).
|
|
3. On BuilderError → ``err(msg)``; on success → fn's return value
|
|
(caller already wrapped it in ``ok(...)``).
|
|
"""
|
|
state.record(tool, args)
|
|
try:
|
|
return fn()
|
|
except BuilderError as e:
|
|
return err(str(e), tool=tool)
|
|
|
|
|
|
def finalize_and_emit(
|
|
state: BuilderState,
|
|
model_class: type[T_Output],
|
|
*,
|
|
extra_required_check: Callable[[BuilderState], None] | None = None,
|
|
) -> dict[str, Any]:
|
|
"""Common finalize helper: validate state.fields against
|
|
``model_class``, mark state.finalized, emit canonical JSON to
|
|
EITHER ``$CONTROLLER_CANONICAL_OUTPUT_PATH`` (preferred, set by
|
|
the production worker agent_runner) OR stdout (fallback for
|
|
direct-call tests).
|
|
|
|
Why the env-var path: when the MCP runs as a subprocess of the
|
|
worker controller (production), its stdout is shared with the
|
|
JSON-RPC transport that FastMCP uses. Writing canonical JSON to
|
|
stdout would collide with the transport's framing. The env-var
|
|
path gives us a clean side channel: subprocess writes the
|
|
canonical JSON to a file; agent_runner reads it after the
|
|
subprocess exits.
|
|
|
|
For direct-call tests (no subprocess), the env var is unset and
|
|
we fall back to stdout (captured by capsys).
|
|
"""
|
|
import os
|
|
state.require_started()
|
|
state.require_not_finalized()
|
|
if extra_required_check is not None:
|
|
extra_required_check(state)
|
|
try:
|
|
model = model_class.model_validate(state.fields)
|
|
except ValidationError as exc:
|
|
return err(
|
|
f"finalize failed schema validation for {model_class.__name__}: {exc}",
|
|
validation_error=str(exc),
|
|
)
|
|
canonical = model.model_dump_json()
|
|
# Mark BEFORE emitting so duplicate finalize calls return error.
|
|
state.finalized = True
|
|
out_path = os.environ.get("CONTROLLER_CANONICAL_OUTPUT_PATH")
|
|
if out_path:
|
|
# Production / subprocess: file-based clean channel.
|
|
try:
|
|
with open(out_path, "w", encoding="utf-8") as f:
|
|
f.write(canonical)
|
|
f.write("\n")
|
|
f.flush()
|
|
os.fsync(f.fileno())
|
|
except OSError as exc:
|
|
return err(f"failed to write canonical output to {out_path}: {exc}")
|
|
else:
|
|
# Direct-call tests: emit to stdout (captured by capsys).
|
|
sys.stdout.write(canonical)
|
|
sys.stdout.write("\n")
|
|
sys.stdout.flush()
|
|
return ok(committed=True, output_bytes=len(canonical))
|
|
|
|
|
|
__all__ = [
|
|
"BuilderError",
|
|
"BuilderState",
|
|
"call_with_invariants",
|
|
"err",
|
|
"finalize_and_emit",
|
|
"ok",
|
|
]
|