ae940f45644971d5937b755d2459eceffeb6500a
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 drew
|
ae940f4564 |
feat(controller): Phase 1d-3c-4 — HTTP adapter wiring callbacks to _claim_runtime
The production glue between the controller's callback protocols
(discovery / forgejo_writes / merging) and the existing Forgejo HTTP
client in _claim_runtime. Single build_callbacks(cfg) factory; tests
use the same module with a fake runtime stub.
tools/controller/master/forgejo_http.py:
- ForgejoCallbacks dataclass bundling every callback the controller
needs: list_prs, list_issues, list_comments, post_comment,
get_labels, add_label, remove_label, merge_pr.
- build_callbacks(cfg, runtime=None): wires each callback as a thin
closure over runtime.get/post/delete. Production omits ``runtime``
to use the real _claim_runtime module; tests inject a fake.
- Forgejo path conventions match the API:
GET /repos/{o}/{r}/pulls?state=open
GET /repos/{o}/{r}/issues?state=open&type=issues (excludes PRs)
GET/POST /repos/{o}/{r}/issues/{n}/comments
GET/POST /repos/{o}/{r}/issues/{n}/labels
DELETE /repos/{o}/{r}/issues/{n}/labels/{name} (URL-encoded)
POST /repos/{o}/{r}/pulls/{n}/merge (body: {"Do": "merge"})
- Robust response handling:
- list endpoints: non-200 → empty list; non-list body → empty;
non-dict items filtered out.
- post_comment: 200/201 ok; other → RuntimeError.
- add_label / remove_label: 200/201/204 → True; remove-404 → True
(label already gone = goal achieved); else False.
- merge_pr: returns normalized MergeResponse. On 404, fetches the
PR's actual state (merged=True → pr_state='merged'; state='closed'
→ 'closed'; PR fetch failure or non-200 → leave pr_state=None so
the merging handler defaults to ABANDONED conservatively).
- Any callback exception → synthetic 503 so the merging handler's
retry logic kicks in cleanly.
23 new tests in test_master_forgejo_http.py:
- list_prs (path format, non-200 → empty, non-list body → empty,
filters non-dict items)
- list_issues (type=issues filter)
- list_comments (path format)
- post_comment (201, 200, non-2xx raises, non-dict body)
- labels (get / add 201/500 / remove 204/404/URL-encoded)
- merge (200, 409, 500, callback-raises-as-503, 404+merged,
404+closed, 404+pull-fetch-failure)
Plus a bug fix surfaced by the test_404_with_pull_fetch_failure test:
the PR-state-fetch branch was returning 'open' on a 500 response;
now correctly checks status==200 before inspecting the body.
Total: 366 controller tests; full auto_agents suite 2728 pass.
|
||
|
drew
|
94821d2702 |
feat(controller): Phase 1d-3c-3 — MERGING handler (6-response-shape table)
The master's per-tick handler for workflows in MERGING. Per plan v6+v9
non-blocking retry: state stays MERGING across ticks, retry counter +
backoff tracked in workflows.merging_retry_count and
merging_retry_next_attempt_at.
tools/controller/master/merging.py:
- MergeResponse: normalized {status_code, pr_state, error_message}
the callback returns. status_code drives the 6-response table:
200 → MERGED
409 → IMPLEMENTING(tier_last_succeeded) with reason=
'post-approval-base-conflict'
403 → STUCK ('branch-protection')
422 → AWAITING_CI ('ci-required-status-missing')
5xx (any) → stay MERGING; bump retry_count + schedule
next_attempt_at = now + 2^retry_count seconds
(capped at 60s); STUCK at retry_count >= 5
404 + pr_state='merged' → MERGED ('externally-merged')
404 + pr_state='closed' → ABANDONED ('externally-closed')
404 + no pr_state → ABANDONED (conservative default)
- run_merging_tick(engine, merge, owner, repo) — sweeps workflows
WHERE current_state='MERGING' AND kind='pr' AND owner+repo match
AND (next_attempt_at IS NULL OR next_attempt_at <= now). Per row:
call merge → map response → transition (or schedule retry) + emit
controller_events row.
- Callback failure (callback raises) wrapped as a synthetic 503 so
the retry logic kicks in cleanly.
- Counter resets on non-retry responses (409 / 422) — keeps backoff
fresh for future retries.
- MAX_MERGE_RETRIES = 5; MAX_BACKOFF_S = 60.0.
14 new tests across 7 classes:
- 200 happy path + event row
- 409 → IMPLEMENTING(tier_last_succeeded)
- 403 → STUCK
- 422 → AWAITING_CI + retry counter reset
- 404 paths (merged + closed + no-state default)
- 5xx retry (counter bumped + backoff scheduled +
in-window-skipped + max-retries-STUCK + callback-raises-as-5xx)
- Owner/repo filter (other repo's MERGING untouched)
- kind='pr' filter (issues never processed even if mis-seeded)
Total: 343 controller tests; full auto_agents suite 2705 pass.
|
||
|
drew
|
3e3796d918 |
feat(controller): Phase 1d-3c-2 — Forgejo write helpers (status + labels)
Idempotent status comment posting via fingerprint markers + per-label
no-op-aware adjust. Production wires the callbacks to existing
_status_comments + _claim_runtime helpers; tests use fakes.
tools/controller/master/forgejo_writes.py:
- compute_fingerprint(workflow_id, event_kind, content_key) →
16-char SHA256 prefix. Deterministic; same inputs → same fingerprint.
Different (workflow_id OR event_kind OR content_key) → different fp.
- build_marker(fp) → HTML comment "<!-- controller:fingerprint:abc -->".
Searchable + invisible in Forgejo's rendered UI.
- comment_has_fingerprint(body, fp) → bool. Used by post_status_comment
for the dedup check before posting.
- post_status_comment(): full idempotency protocol —
1. compute fingerprint
2. list_comments callback → check existing for marker
3. if found → return duplicate (no post)
4. else post via post_comment callback
Failure modes:
- list_comments raises → fall through to post (conservative;
fingerprint match on next sweep catches the duplicate)
- post_comment raises → return failed; caller retries
- adjust_labels(add, remove): one-shot get_labels + per-label
add/remove. add-when-already-present → no-op; remove-when-absent
→ no-op. Per-label failure isolated. get_labels failure marks
all requested actions failed (caller retries).
19 new tests:
- fingerprint helpers (5: deterministic, distinct inputs, marker
format, body match, empty body no-match)
- post_status_comment (5: new post, duplicate skip, list-failure
fall-through, post failure, distinct event_kinds get distinct fps)
- adjust_labels (9: empty no-op, add-when-absent, add no-op,
remove-when-present, remove no-op, get failure, per-label
isolation with mixed failures, returning-False failure)
Total: 329 controller tests; full auto_agents suite 2691 pass.
Phase 1d-3c-3 (MERGING handler) + Phase 1c-3 (real OpenCode + MCP
spawn) remain.
|
||
|
drew
|
e8045d4b9b |
feat(controller): Phase 1d-3c-1 — discovery sweep + worker test deflake
Discovery polls Forgejo for open PRs/issues and inserts a fresh DISCOVERED workflow for any entity the controller hasn't seen yet. Same callback-injection pattern as the scheduler's prefetch: tests provide synthetic ListPRsCallback / ListIssuesCallback; production wires them to the existing _review_fetch helpers in a follow-up. tools/controller/master/discovery.py: - run_discovery(engine, owner, repo, list_prs, list_issues=None) — idempotent. Skips existing entities via a one-shot SELECT(kind, entity_number) WHERE owner+repo membership check. Inserts a DISCOVERED workflow + a controller_events 'discovered' row per new entity. Per-callback try/except: a PR-list failure doesn't block issue discovery, and vice versa. - Defensive coerce_int: rejects bools (True is an int subclass — bug class to avoid). Accepts string digits. - Returns DiscoveryReport(prs_seen, issues_seen, new_workflows, existing_skipped, new_entities[]). 12 new tests: - basics (empty, PRs only, issues only, both PRs+issues) - idempotency (2nd sweep skips; PR #42 + issue #42 coexist; other (owner, repo) isolated) - malformed input (non-int + bool skipped; string digit accepted) - callback failures (PR raises → still process issues; issues callback optional) - event row creation per new workflow Plus a worker test deflake: test_stolen_lock_between_agent_return_and_write was asserting the specific 'lost-lock-at-write' outcome, but under full-suite CPU contention the heartbeat thread can fire between the agent's return and _write_outcome — taking the 'lost-lock' (via lost_lock_event) path instead. Both are valid for this scenario; loosened the assertion to accept either. Total: 310 controller tests; full auto_agents suite 2672 pass. |
||
|
drew
|
337af855f3 |
feat(controller): Phase 1d-3b — per-workflow scheduler + static escalation
When the state machine transitions a workflow into a state that needs a worker (ANALYZING / IMPLEMENTING / REVIEWING / CONFLICT_RESOLVING), the scheduler enqueues a fresh pending workflow_attempts row with the input_payload prefetched. tools/controller/master/scheduler.py: - schedule_next_attempts(engine, prefetch=...) — finds workflows in worker-needing states without a matching pending/in_progress attempt; enqueues one fresh attempt per. Skips: - workflows already with pending/in_progress attempt for the role - ESCALATING at MAX_TIER → resolved to ABANDONED (no enqueue) - prefetch raised → per-workflow isolated failure - Static escalation policy resolved inline: ESCALATING + current_tier → IMPLEMENTING(min(tier+1, MAX_TIER)) OR ABANDONED. Both produce controller_events transition rows with the appropriate v9 event name (escalate_next_tier_available / escalate_max_tier_exhausted). - State→role table: ANALYZING → estimator, IMPLEMENTING → implementer, REVIEWING → reviewer, CONFLICT_RESOLVING → conflict_resolver. - PrefetchCallback is parameterized; production wires it to the Forgejo prefetch (Phase 1d-3c), tests inject a fake. - attempt_number monotonically increments via COALESCE(MAX, 0) + 1. - DEFERRED to Phase 1d-3c: actual Forgejo prefetch implementation (this commit ships the scheduler skeleton + the prefetch callback contract). 20 new tests: - state→role parametrization (4 states × matching role) - no-double-enqueue (3 cases: pending blocks, in_progress blocks, different-role doesn't block) - escalation (4 cases: tier 0→1, 1→2, MAX→ABANDONED, event row content) - prefetch raises → per-workflow skip - non-schedulable states parametrized (7 cases: DISCOVERED, AWAITING_CI, MERGING, MERGED, ABANDONED, STUCK, CREATED_PR) - attempt_number monotonicity (uses MAX+1) Total: 298 controller tests; full auto_agents suite 2660 pass. |
||
|
drew
|
130bbbf3c5 |
feat(controller): Phase 1d-3a — master main loop (composite tick)
Long-running master orchestrator composing the deterministic per-iteration work shipped in Phase 1d-1/1d-2. One iteration = state-machine tick + reaper + pickup guard, in that order (reasoning in module docstring). tools/controller/master/loop.py: - MasterConfig: tick_interval_s (default 30s), reaper_interval_s (default 60s), pickup_guard_max_pickups (default 3 per v6). - run_master_iteration(engine): single synchronous iteration — composable for tests + master_main_loop. - master_main_loop(engine): runs run_master_iteration on a loop until stop_event fires. Reaper runs less often than tick (every reaper_interval_s, not every tick_interval_s). on_iteration callback gets each MasterTickReport (used by tests + future structured logging). 7 new tests: - run_master_iteration: empty DB no-op; tick advances state (blocked → STUCK); reaper + pickup guard compose (stale in_progress at pickup limit → reaped → STUCK in one iteration). - master_main_loop: runs until stop; empty loop exits cleanly; on_iteration exception doesn't break loop; reaper runs less frequently than tick. Phase 1d-3+ remains for: discovery (Forgejo poll), per-workflow scheduling (enqueue next workflow_attempts after transition), Forgejo writes, MERGING state's merge call, reconciliation, backfill, operator CLI. Total: 278 controller tests; full auto_agents suite 2640 pass. |
||
|
drew
|
b519ebd98f |
feat(controller): Phase 1d-2 — outcome mapper + master tick handler
The master-side state machine driver. When a worker writes a complete
(or failed) attempt, the tick handler picks it up next pass, maps the
outcome to a state machine event, and applies the transition.
tools/controller/master/:
- outcomes.py: map_outcome_to_event() — pure function. Inputs:
role, current_state, output_payload, status, head_sha_advanced,
attempts_remaining_at_tier, conflict_count_at_current_tier.
Returns EventMapResult(event_name|None, reason).
- Implementer: resolved+pushed → implementer_pushed;
resolved+NO push → implementer_competence_failure (worker lied);
rebase-failed / competence-failure / blocked / noop all routed.
- Reviewer: approve → reviewer_approve; request-changes → retry vs
escalate based on attempts_remaining_at_tier; abstain →
reviewer_abstain; comment → no transition (operator review needed).
- Estimator: is_metadata_only → estimator_metadata_only; else
estimator_done.
- Conflict resolver: counts at current tier — 1st → first; 2nd →
second_same_tier (escalate); 3rd+ → three_plus (STUCK).
- Summarizer: doesn't drive state transitions.
- status='failed' policy: contract-violation → pickup_exhausted;
other failed outcomes (worker-internal-error / stale-input /
ttl-insufficient-for-retry / git-clone-failed / lost-lock / etc.)
→ no event (master re-enqueues per the v9 table).
- status='reaped' → no event (pickup guard handled).
- tick.py: run_tick() — one master tick. Queries
workflow_attempts WHERE status IN ('complete', 'failed') AND
finished_at > workflow.last_transition_at (heuristic for "not yet
processed"). Per row:
1. Validate current_state ∈ KNOWN_STATES (per v6 unknown-state
guard); if not, transition workflow → STUCK with reason='unknown-state'.
2. Map outcome → event via outcomes.map_outcome_to_event.
3. If no event, bump workflow.last_transition_at so we don't
re-process forever.
4. Apply event via state_machine.apply_event; IllegalTransition →
STUCK with reason='illegal-event'.
5. Commit transition (workflows.current_state + entered_state_at +
last_transition_at) AND insert controller_events row.
Returns TickReport with attempts_processed + transitions_applied +
transitions_log.
42 new tests:
- outcomes (22): status='failed' policy (3 paths) + each role's
happy paths + edge cases (unknown outcome / missing field /
None payload / unknown role).
- tick (20): implementer transitions (resolved+push → AWAITING_CI,
resolved-no-push → ESCALATING, rebase → CONFLICT_RESOLVING,
blocked → STUCK), reviewer transitions (approve → MERGING,
request-changes → IMPLEMENTING), event row content, no
reprocessing on second tick, unmapped attempts bump
last_transition_at, terminal workflows skipped (4 parametrized),
unknown current_state → STUCK.
Total: 271 controller tests; full auto_agents suite 2633 pass.
|
||
|
drew
|
976817fa22 |
feat(controller): Phase 1d-1 — state machine + reaper + pickup guard
The deterministic spine of the master controller. State machine is
pure data with 6 load-bearing invariants enforced via property tests.
Reaper resets stale-heartbeat workflow_attempts to pending. Pickup
guard transitions workflows to STUCK when an attempt has been
re-pended too many times without success.
tools/controller/state_machine.py:
- KNOWN_STATES = 12; TERMINAL_STATES = {MERGED, ABANDONED, STUCK,
CREATED_PR}. STUCK's only allowed exit is the operator-driven
operator_unstick event (back to DISCOVERED).
- 32 TRANSITIONS entries covering DISCOVERED → ANALYZING →
IMPLEMENTING ↔ AWAITING_CI / CONFLICT_RESOLVING / ESCALATING →
REVIEWING → MERGING → MERGED. Plus pickup_exhausted exits from
IMPLEMENTING/CONFLICT_RESOLVING/REVIEWING.
- 27 named events with descriptions. apply_event() lookup raises
IllegalTransitionError (lists legal events from current state)
or ValueError on unknown state (per v6 unknown-state guard).
- 6 LOAD-BEARING invariants for v1 (per v9 simplification):
1. no_path_implementing_to_reviewing_skips_ci (Hard Rule #1
constructional fix for the no-mans-land race)
2. terminal_states_have_no_exits (only STUCK→operator_unstick OK)
3. tier_monotonic_non_decreasing
4. every_pr_workflow_includes_reviewing
5. conflict_resolving_bounded (1st→IMPLEMENTING, 2nd→ESCALATING,
3rd→STUCK; structurally encoded)
6. escalation_deterministic
- reachable_from() honors cycles (DISCOVERED ∈ reachable(DISCOVERED)
via STUCK→operator_unstick path; AWAITING_CI self-loops via
ci_flake_retry).
tools/controller/reaper.py:
- reap_stale_attempts(): SELECT in_progress attempts whose
lock_heartbeat_at + lock_ttl_seconds < NOW (per-row TTL respects
per-role differences — estimator 180s, reviewer 720s, tier-2
implementer 2160s). UPDATEs status='pending', clears lock columns,
preserves pickup_count (the pickup guard handles that). Inserts
controller_events row with reason='lock-ttl-expired' per reap.
- Dialect-portable: Postgres uses interval arithmetic; SQLite uses
julianday(). Same logic either way.
tools/controller/pickup_guard.py:
- transition_exhausted_to_stuck(): finds attempts with status='pending'
AND pickup_count >= MAX_PICKUPS (default 3 per v6 blocker fix)
AND workflow not already terminal. Transitions workflow → STUCK,
marks attempt as 'reaped', inserts controller_events with
reason='attempt-pickup-exhausted' + pickup_count + max_pickups.
45 new tests:
- state_machine: basic shape (states partition, every transition uses
known states + defined events), apply_event success/error paths,
events_from + reachable_from helpers (including cycle awareness),
per-invariant zero-violations against the live table, per-invariant
monkeypatch-violations to prove the checks catch the bug class they
claim to, parametrised sanity check "every non-terminal can reach
some terminal".
- reaper: empty DB / fresh heartbeat / stale heartbeat reaped /
per-row TTL respected / event row created / only-in-progress
reaped / multiple stale attempts.
- pickup guard: empty DB / below limit / at limit / in-progress not
checked / terminal workflow skipped / event payload content /
default max_pickups matches v6.
Total: 229 controller tests; full auto_agents suite 2591 pass.
|
||
|
drew
|
30dfd92021 |
feat(controller): Phase 1c-2 — workspace + session sidecar + orphan janitor
Per-PR workspace umbrella manager (per plan v6) + the worker.session
sidecar that tracks live OpenCode session metadata for orphan cleanup
+ the startup janitor that sweeps orphaned workspaces from previous
worker crashes.
tools/controller/worker/:
- workspace.py: PerPRWorkspace + WorkspaceIdentity (frozen dataclass
yielding the canonical pr-{owner}-{repo}-{N} dir name). ensure_present
is idempotent. clone_if_absent runs `git clone --no-single-branch`
if .git/ missing; idempotent on re-call. fetch_and_validate runs
`git fetch origin --prune`, compares origin/<ref> to expected
head_sha, raises StaleInputError on mismatch (v6 stale-input fix),
then `git reset --hard <expected_sha>` + `git clean -fdx` to wipe
worktree residue from prior attempts. remove() rm -rf's the
workspace.
- session_sidecar.py: WorkerSession frozen dataclass +
write_sidecar (atomic via tmp+rename+fsync) + read_sidecar (tolerates
missing/empty/malformed/wrong-shape gracefully). Sidecar captures
opencode_server_url / session_id / subprocess_pid /
spawned_by_controller_pid / instance_id / spawned_at.
- janitor.py: sweep_orphans pre-queries the DB once for the set of
live instance_ids (workflow_attempts.status='in_progress'), then
scans workspace_root for pr-* dirs. For each:
- no sidecar → just delete (crashed pre-spawn)
- sidecar's instance_id is in live set → preserve (active worker)
- else → orphan. Try cancel_callback (production wires to OpenCode
cancel API); fall back to SIGTERM/grace/SIGKILL on the
subprocess_pid. Then delete sidecar + workspace. JanitorReport
summarises each sweep for structured logging.
Key v6 design points implemented:
- worker.session sidecar atomicity → orphan detection is robust
against partial writes (e.g., worker crashed mid-spawn).
- Cancel-callback-then-SIGKILL fallback → production prefers the
graceful OpenCode-side cancel; tests inject a fake.
- _kill_with_grace returns True after SIGKILL delivery; zombie
reaping is the parent's responsibility, not the janitor's.
27 new tests:
- WorkspaceIdentity (format + frozen)
- PerPRWorkspace paths + ensure_present idempotency
- clone_if_absent (creates worktree, idempotent, raises without URL)
- fetch_and_validate (matching sha resets clean; mismatched raises
StaleInputError; unknown ref raises)
- Sidecar I/O round-trip + atomicity + missing/empty/malformed handling
- Janitor: empty root / no-sidecar / active-lock / orphan-with-dead-pid
/ orphan-with-live-pid / cancel-callback (3 sub-paths: ok / fails /
raises) / mixed-workspaces / non-pr-skip
Total: 184 controller tests; full auto_agents suite 2546 pass.
|
||
|
drew
|
eab476e48e |
feat(controller): Phase 1c — worker controller skeleton
The dequeue+lock+heartbeat+runner+loop machinery. Production
OpenCode + MCP invocation slots in via the agent_runner callable
(Phase 1c-2). This commit is the structural foundation:
tools/controller/worker/:
- identity.py: build_instance_id() → "{hostname}/{pid}/{worker_uuid}"
per plan v9 (slash delimiter; IPv6-safe; uuid4 prefix for
per-instance uniqueness).
- heartbeat.py: Heartbeat thread that updates lock_heartbeat_at
every interval (default 30s). v9 simplified: TTL-only (no activity
tracking). UPDATE … WHERE locked_by_instance=us; rowcount=0 →
lost_lock_event.set() and thread exits, letting reaper handle it.
- runner.py: run_one_attempt() drives one attempt end-to-end.
Starts heartbeat → invokes agent_runner → on success writes
status='complete' + output_payload; on WorkerError writes
status='failed' with outcome label; on WorkerLostLock or detected
stolen-lock-at-write returns aborted (no DB write — reaper has
already re-pended). Defense-in-depth: even if agent returns
successfully, lost_lock_event.is_set() check skips the write.
- loop.py: worker_main_loop() polls the DB for pending attempts up
to MAX_CONCURRENT_WORKERS_PER_MACHINE, submits each to a
ThreadPoolExecutor. Honors stop_event for graceful shutdown
(drains in-flight before exit).
tools/controller/db/session.py: StaticPool for in-memory SQLite so
the heartbeat thread + runner write + dequeue all see the same DB
(without this, ":memory:" gives each connection an independent DB).
16 new tests in test_worker.py: instance ID format/uniqueness;
heartbeat tick (hold + steal); runner happy path; 5 error paths
(worker error / unexpected exception / WorkerLostLock raised /
stolen lock at write / lost_lock_event set defense-in-depth); 4
loop scenarios (single attempt, role filter skip, empty queue
exit-on-stop, explicit instance_id).
Total: 157 controller tests; full auto_agents suite 2519 pass.
|
||
|
drew
|
36b133ec5e |
feat(controller): Phase 1b — DB schema + dequeue helper + payload guard
Five SQLAlchemy 2.0 declarative models implementing plan v6/v9's
unified workflow schema. Cross-dialect (SQLite for tests + local dev,
Postgres for multi-machine production). Lock columns on
workflow_attempts implement the multi-machine-safe dequeue protocol
(plan v5).
Modules:
- tools/controller/db/models.py:
- Workflow (kind discriminator pr/issue, unique on owner+repo+kind+
entity_number, parent_workflow_id FK for issue→PR linkage)
- WorkflowAttempt (status/locked_by_instance/locked_at/
lock_heartbeat_at/lock_ttl_seconds/pickup_count + CHECK
constraints on status enum and pickup_count≥0; partial indexes
on the pending/in_progress/complete hot paths)
- ControllerEvent (Forgejo-write replay support kept in-schema even
though v9 simplified to Forgejo-first protocol; allows v3-style
upgrade later without migration)
- FlakeHistory (composite PK; supports the v6 flake-learning
heuristic)
- CIObservation (raw CI state history; 90-day retention to be
enforced by a sweep task)
- AutoincrementPk variant (Integer on SQLite where it autoincrements
via rowid; BigInteger on Postgres for BIGSERIAL); JsonColumn
variant (JSON on SQLite, JSONB on Postgres)
- tools/controller/db/session.py: build_engine (per-dialect tuning —
SQLite WAL + foreign_keys + busy_timeout; Postgres pool_pre_ping);
create_all (idempotent); session_scope (transactional context).
- tools/controller/db/dequeue.py: dequeue_one (one row atomically;
Postgres path uses SELECT FOR UPDATE SKIP LOCKED, SQLite path uses
UPDATE-WHERE-id-IN-SELECT-LIMIT-1 with RETURNING). Bumps pickup_count
on dequeue; respects max_pickups guard (default 3 per v6 blocker fix).
Returns DequeueResult dataclass with role/tier/pickup_count and
reason on miss.
- tools/controller/db/payload_guard.py: enforce_input_payload_size
with 4MB cap and 5-step truncation priority (older_summary → oldest
verbatim → comments → full_diff → CI failure excerpts). Raises
PayloadTooLargeError after all steps exhausted; master maps to
workflow STUCK with reason='input-too-large'.
- pyproject.toml: new optional extras `controller-db` pinning
sqlalchemy + psycopg2-binary (latter installed only for prod
multi-machine deploy; tests use stdlib sqlite3 via SQLAlchemy's
SQLite dialect which is already pulled in transitively via alembic).
37 new tests across test_db_schema/dequeue/payload_guard; 141
controller tests total; full auto_agents suite 2503 pass (no
regressions).
|
||
|
drew
|
eebb5718a8 |
feat(controller): Phase 1a — 5 response-builder MCP servers
Per-attempt MCP subprocesses that enforce V1 contract invariants at
construction time. Worker LLM calls builder tools incrementally; the
MCP validates each call against the schema + cross-field invariants;
`{role}_finalize()` emits canonical Pydantic-validated JSON to stdout
for the worker controller to read (Phase 1c). Defense-in-depth: the
controller strict-parses whatever finalize emits.
Builders shipped:
- reviewer_builder: 9 tools. Auto-acks all CISummary gates as passed
at start; reviewer only calls record_gate to discuss specifics.
reviewer_override_gate requires ≥20-char justification. Approve
with any failed gate is refused with an actionable error pointing
at the override path. Request-changes requires ≥1 blocking issue.
Verdict-vs-blocking-issues invariant checked at finalize.
- implementer_builder: 7 tools. Outcome-specific finalize invariants:
resolved → ≥1 commit + ≥1 file; blocked → ≥1 blocker; noop → no
commits/files/blockers.
- estimator_builder: 4 tools. Lightweight; requires
recommended_tier + confidence + reasoning at finalize. Reasoning
capped at 2048 chars.
- conflict_resolver_builder: 8 tools. outcome='resolved' requires
new_head_sha + ≥1 commit + ≥1 file. resolution_strategy enum-checked.
- summarizer_builder: 3 tools. Summary 50-2000 chars (enforced at
MCP layer and Pydantic).
Shared infrastructure:
- _builder_base.py: BuilderState dataclass + invariant guard helpers
(require_started / require_not_finalized) + audit-record-with-summary
+ finalize_and_emit (validates against Pydantic model class,
emits canonical JSON to stdout, marks finalized).
41 builder tests in test_mcp_builders.py (happy paths + every
invariant + outcome-specific paths + audit summarization + JSON
round-trip through Pydantic strict-parse). Plus the existing 62
Phase-0 tests. 103 controller tests total. Full auto_agents suite
(2465 tests) still passes.
|
||
|
drew
|
a10758177c |
feat(controller): Phase 0 foundation — V1 contracts + CI gate enumeration
Lays the foundation for the planned controller + DB-owned PR state machine rewrite (see .drew/controller_state_machine.md). Phase 0a — Pydantic V1 worker I/O contracts (tools/controller/contracts/): - v1.py: ImplementerInputV1/OutputV1, ReviewerInputV1/OutputV1, EstimatorOutputV1, IssueImplementerOutputV1, ConflictResolverInputV1/OutputV1, SummarizerInputV1/OutputV1. Plus CISummary / CIFailure / GateResult / FailedAssertion / FileLocation. All models use extra="forbid" + required output_version Literal discriminator (no defaults). - parse.py: strict_parse() + strict_parse_with_retry() helpers. Corrective prompt template quotes the Pydantic ValidationError + the JSON schema. TTL guard skips retry when remaining time too short. Defense-in-depth around the response-builder MCPs that arrive in Phase 1a. Phase 0b — Static CI parser-coverage discovery: - NOX_SESSION_TO_PARSER map in tools/controller/ci_summary_parsers/. Derived from .forgejo/workflows/*.yml + noxfile.py. Covers ruff, pyright, behave, robot_framework, slipcover, vulture, radon, bandit, semgrep, build. Cron-only sessions (benchmark, benchmark_regression) are in NOX_SESSIONS_SKIP_COVERAGE. - scripts/enumerate-ci-gates.py: reads the workflow YAML, joins against the map, exits non-zero on coverage gaps. Wired into tests so the actual .forgejo/ is regression-guarded. 63 new tests; full auto_agents suite still passes (2424). |