Files
cleveragents-core/tools/controller/master/__init__.py
T
drew 016b348117 feat(controller): grooming gate (Phase 0 + Phase 1 worker-shape dispatch)
Phase 0 (foundation):
- Cause enum (controller_events.cause) for action attribution
- Schema: grooming_decisions audit table; workflows gains
  grooming_evaluated_at + deferred_reason + deferred_at +
  deferred_target_workflow_id; pulls gains touched_files
- audit_comments: CLOSE / DEFER templates + render_comment_template
- forgejo_writes: close_issue + defer_issue 5-step crash-safe protocol
  (fingerprint dedup, error matrix, dry-run)
- patch_pr_state callback in forgejo_http
- grooming_config: 22-env-var frozen-dataclass config + log_effective
- pulls.touched_files cache extension (_pipeline_cache.py schema v8)
- reaper.reap_grooming_decisions audit-retention sweep
- reconciliation RESUME guard (deferred_reason)

Phase 1 (worker-queue shape, 2026-05-25):
- New state: GROOMING. New events: grooming_started, groom_verdict_
  {proceed,defer,close}. 5 new transitions; all invariants still clean
- GroomingInputV1 + GroomingOutputV1 Pydantic contracts
- outcomes._map_grooming_outcome routes verdicts to state-machine events
- prefetch.build_grooming_stage_b_input + list_open_prs callback
- scheduler GROOMING -> grooming_stage_b role
- promote: cfg-gated DISCOVERED -> GROOMING when CONTROLLER_GROOMING_
  ENABLED=true; issues skip grooming
- forgejo_writes decomposed: close_act/defer_act (Forgejo writes only;
  state-machine already transitioned) + close_decide_and_act/
  defer_decide_and_act (Phase 0 callers); _apply_workflow_transition
  is underscore-private
- grooming.py library: tokenization, suspicion scoring (Jaccard +
  weighted overlap), deterministic checks, action -> verdict mapping
- mcp/grooming_builder.py: 14-tool FastMCP server emits GroomingOutputV1
- .opencode/agents/grooming-stage-b.md: duplicate-detection agent
  prompt (claude-haiku-4-5)
- grooming_side_effects.run_grooming_side_effects_tick: per-state tick
  performs Forgejo writes after groom_verdict_{defer,close} fires.
  Filters on event_type='transition' + payload.event (centralizes the
  convention pending Phase 2's latest_transition_event helper)
- GroomingCallbacks frozen dataclass; loop.py + __main__.py wired

Worker role registry (single source of truth):
- worker/roles.py: WORKER_ROLES + WorkerRoleSpec + default_roles_csv
  + output_filename_for. agent_runner.ROLE_TO_MCP_MODULE / ROLE_TO_
  OUTPUT_MODEL derive from it; opencode_session.agent_name_for reads
  it for flat cases; all 6 prompt builders use output_filename_for;
  worker --roles default = default_roles_csv(); launcher script
  derives --roles via shell substitution. Cross-site invariant test
  enforces alignment across 5 sites + opencode.json MCP registry.

Phase 0 silent-bug fix:
- reconciliation.py RESUME guard SELECT now includes deferred_reason
  (was missing since Phase 0; guard was a silent no-op). Tightened
  from getattr to attribute access to fail fast on future omissions.

Tests (1456 total, +91 grooming-specific):
- test_grooming_phase0.py: 34 tests (orchestrator matrix, crash
  recovery, idempotency, dry-run)
- test_grooming_phase1.py: 60 tests (library, contracts, state
  machine, outcomes, scheduler, promote, prefetch, act-variants
  with signature parity, side-effect tick incl. natural-idempotency
  + executed-flag-skip + verdict-mismatch + reconciliation RESUME)
- test_mcp_builders.py TestGroomingBuilder: 29 tests (happy paths
  + 22 validation rules + Pydantic round-trip + master-tick-read-
  path companion)
- test_worker_agent_runner.py TestRoleMaps: cross-role wiring
  alignment + agent-prompt-vs-worker-fallback filename contract +
  inspect.signature equality (close_act/defer_act vs
  close_issue/defer_issue)
- test_state_machine.py: transition count 51 -> 56 +
  events_from_grooming

Live-validated end-to-end on 4 staged sentinel PRs (#55-#58) in
dry_run: agent emits verdicts via MCP, state-machine transitions
fire, side-effect tick writes audit row, deferred_reason gates
reconciliation RESUME correctly.

Deferred refinements + Phase 2 prerequisite (latest_transition_event
helper) tracked in .drew/regressions-plan.md "Phase 1 follow-up
backlog".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:05:29 -04:00

231 lines
6.0 KiB
Python

"""Master controller — singleton orchestrator.
Per plan v9: the master process is a singleton per (owner, repo). It:
1. Runs discovery (Forgejo poll) — Phase 1d-3
2. Drives the state machine: for each non-terminal workflow, advance
one step based on the outcome of its latest completed attempt
3. Runs the reaper + pickup guard each tick
4. Performs Forgejo writes (status comments, labels, merges) —
Phase 1d-3+
5. Runs periodic reconciliation (DB↔Forgejo) — Phase 1d-3+
This package ships incrementally. Phase 1d-1 (already shipped) had
the state machine + reaper + pickup guard. This Phase 1d-2 adds:
- ``outcomes.py``: maps worker output_payload outcomes → state machine
events (e.g., implementer outcome='resolved''implementer_pushed';
outcome='rebase-failed''implementer_rebase_failed').
- ``tick.py``: one tick handler — scans recently-completed attempts,
applies events to their workflows, emits controller_events.
"""
from .outcomes import (
EventMapResult,
map_outcome_to_event,
)
from .tick import (
TickReport,
run_tick,
)
from .scheduler import (
MAX_TIER,
PrefetchCallback,
ScheduledAttempt,
SchedulerReport,
schedule_next_attempts,
)
from .discovery import (
DiscoveredEntity,
DiscoveryReport,
ListIssuesCallback,
ListPRsCallback,
run_discovery,
)
from .forgejo_writes import (
AddLabelCallback,
GetLabelsCallback,
LabelAdjustResult,
ListCommentsCallback,
PostCommentCallback,
RemoveLabelCallback,
StatusCommentResult,
adjust_labels,
build_marker,
comment_has_fingerprint,
compute_fingerprint,
post_status_comment,
)
from .merging import (
MAX_BACKOFF_S,
MAX_MERGE_RETRIES,
MergeCallback,
MergeResponse,
MergingHandlerReport,
run_merging_tick,
)
from .forgejo_http import (
ForgejoCallbacks,
build_callbacks,
)
from .backfill import (
BACKFILL_MARKER_EVENT_TYPE,
BackfillReport,
has_backfill_run,
run_startup_backfill,
)
from .ci_poll import (
CIPollExhaustionReport,
DEFAULT_AWAITING_CI_TIMEOUT_S,
run_ci_poll_exhaustion_tick,
)
from .ci_status_poll import (
CIStatusPollReport,
GetCIStatusCallback,
run_ci_status_poll_tick,
)
from .promote import (
PromoteDiscoveredReport,
run_promote_discovered_tick,
)
# Phase 1 (Path A) grooming dispatch was reverted 2026-05-25; the
# implementation didn't match the controller's worker-queue pattern
# and needs a re-design before re-implementation. grooming_config is
# kept (env-var loading is dispatch-shape-agnostic) and re-exported
# here so __main__.py's startup log line still works.
from .grooming_config import (
GroomingConfig,
get_grooming_config,
log_effective_config,
)
from .ci_summarize import (
LogFetcher,
summarize_ci_status,
)
from .label_gate import (
DEFAULT_OPT_IN_LABEL,
count_filtered,
filter_by_opt_in_label,
has_opt_in_label,
opt_in_label_name,
)
from .prefetch import (
GetPRDetailsCallback,
GetPRDiffCallback,
ListPRCommentsCallback,
ListPRReviewsCallback,
PrefetchDataCallbacks,
build_conflict_resolver_input,
build_estimator_input,
build_implementer_input,
build_reviewer_input,
make_prefetch_callback,
)
from .reconciliation import (
GetIssueStateCallback,
GetPRStateCallback,
ReconciliationAction,
ReconciliationReport,
run_reconciliation_tick,
)
from .loop import (
MasterConfig,
MasterTickReport,
master_main_loop,
run_master_iteration,
)
__all__ = [
# Discovery
"DiscoveredEntity",
"DiscoveryReport",
"ListIssuesCallback",
"ListPRsCallback",
"run_discovery",
# Outcome mapping
"EventMapResult",
"map_outcome_to_event",
# Scheduler
"MAX_TIER",
"PrefetchCallback",
"ScheduledAttempt",
"SchedulerReport",
"schedule_next_attempts",
# Tick
"TickReport",
"run_tick",
# Main loop
"MasterConfig",
"MasterTickReport",
"master_main_loop",
"run_master_iteration",
# Forgejo writes
"AddLabelCallback",
"GetLabelsCallback",
"LabelAdjustResult",
"ListCommentsCallback",
"PostCommentCallback",
"RemoveLabelCallback",
"StatusCommentResult",
"adjust_labels",
"build_marker",
"comment_has_fingerprint",
"compute_fingerprint",
"post_status_comment",
# MERGING handler
"MAX_BACKOFF_S",
"MAX_MERGE_RETRIES",
"MergeCallback",
"MergeResponse",
"MergingHandlerReport",
"run_merging_tick",
# HTTP adapter (Forgejo wiring)
"ForgejoCallbacks",
"build_callbacks",
# Backfill (master startup)
"BACKFILL_MARKER_EVENT_TYPE",
"BackfillReport",
"has_backfill_run",
"run_startup_backfill",
# Reconciliation (periodic DB ↔ Forgejo sync)
"GetIssueStateCallback",
"GetPRStateCallback",
"ReconciliationAction",
"ReconciliationReport",
"run_reconciliation_tick",
# Prefetch (Phase 1h)
"GetPRDetailsCallback",
"GetPRDiffCallback",
"ListPRCommentsCallback",
"ListPRReviewsCallback",
"PrefetchDataCallbacks",
"build_conflict_resolver_input",
"build_estimator_input",
"build_implementer_input",
"build_reviewer_input",
"make_prefetch_callback",
# CI summarizer (Phase 1j)
"LogFetcher",
"summarize_ci_status",
# CI poll-exhaustion (Phase 1k++)
"CIPollExhaustionReport",
"DEFAULT_AWAITING_CI_TIMEOUT_S",
"run_ci_poll_exhaustion_tick",
# CI status poll (Phase 1k++++)
"CIStatusPollReport",
"GetCIStatusCallback",
"run_ci_status_poll_tick",
# Promote DISCOVERED → ANALYZING (Phase 1k+++)
"PromoteDiscoveredReport",
"run_promote_discovered_tick",
# Phase 1 grooming plan: dispatch reverted 2026-05-25 (see __init__.py
# import block comment). Config is kept for the startup log line.
"GroomingConfig",
"get_grooming_config",
"log_effective_config",
# Opt-in label gate (Phase 1k)
"DEFAULT_OPT_IN_LABEL",
"count_filtered",
"filter_by_opt_in_label",
"has_opt_in_label",
"opt_in_label_name",
]