drew
a19b609554
Deployment surface for the controller. Operators copy the unit files
+ env examples to /etc/, customize, and `systemctl enable --now`.
Layout (under tools/controller/deploy/):
- systemd/cleveragents-controller-master.service — master singleton.
One unit per (owner, repo); plan v9 singleton constraint enforced
by deployment.
- systemd/cleveragents-controller-worker@.service — worker template.
`systemctl enable cleveragents-controller-worker@implementer-1`
spins up one instance; scale by adding instances. Per-instance
env override at /etc/cleveragents/worker.<inst>.env (optional)
specialises CLEVERAGENTS_WORKER_ROLES per instance.
- systemd/master.env.example — every env var the master reads,
documented. Copy to /etc/cleveragents/master.env, mode 0640.
- systemd/worker.env.example — every env var the workers read.
- RUNBOOK.md — operator guide:
* Prereqs (Linux + systemd 245+, Python 3.13 + uv, Postgres 14+,
OpenCode server, dedicated cleveragents user).
* First-time setup (7 ordered steps from useradd to first PR).
* Day-to-day ops (where logs live, healthy queries, role pool
sizing, label-based pause, clean restart).
* Incident response (6 named scenarios: STUCK workflows, MERGING
retry exhaustion, no-OpenCode, dequeue starvation, DB loss,
operator-unstick procedure).
* Migration playbook (Phase 2: opt one PR in via label,
monitor controller_events, escalate to full management).
* Tunables cheat sheet covering all 8 env vars + their tradeoffs.
- README.md — index linking the above.
Hardening on both units: NoNewPrivileges, PrivateTmp,
ProtectSystem=strict, ProtectHome, narrow ReadWritePaths, kernel
+ control-group protections. RestartPreventExitStatus=2 prevents
systemd loop-restart on misconfiguration (exit 2 = bad env).
No code changes; verified via `systemd-analyze verify` (unit syntax
parses; only "venv path doesn't exist on dev host" warnings, which
are expected). Test suite: 569/569 pass, 0 regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
47 lines
2.5 KiB
Bash
47 lines
2.5 KiB
Bash
# cleveragents controller — worker EnvironmentFile (shared across
|
|
# all worker instances on this host).
|
|
# Copy to /etc/cleveragents/worker.env. Per-instance overrides go in
|
|
# /etc/cleveragents/worker.<instance>.env (e.g. worker.implementer-1.env).
|
|
|
|
# ─── infrastructure (REQUIRED, must match master.env) ─────────────
|
|
CLEVERAGENTS_DB_URL=postgresql+psycopg2://cleveragents:CHANGEME@db.internal:5432/cleveragents_controller
|
|
CONTROLLER_OPENCODE_URL=http://opencode.internal:4096
|
|
|
|
# Forgejo (same fields as master.env).
|
|
FORGEJO_URL=https://git.cleverthis.com
|
|
FORGEJO_TOKEN=CHANGEME-forgejo-pat
|
|
FORGEJO_USER=cleveragents-bot
|
|
|
|
# ─── role pool ────────────────────────────────────────────────────
|
|
# Comma-separated. Default: all 5 roles. Specialise per instance via
|
|
# /etc/cleveragents/worker.<instance>.env:
|
|
# CLEVERAGENTS_WORKER_ROLES=implementer
|
|
# A typical multi-machine deployment:
|
|
# worker@implementer-1, implementer-2 → CLEVERAGENTS_WORKER_ROLES=implementer
|
|
# worker@reviewer-1 → CLEVERAGENTS_WORKER_ROLES=reviewer
|
|
# worker@estimator-1 → CLEVERAGENTS_WORKER_ROLES=estimator,summarizer,conflict_resolver
|
|
CLEVERAGENTS_WORKER_ROLES=implementer,reviewer,estimator,conflict_resolver,summarizer
|
|
|
|
# ─── concurrency ──────────────────────────────────────────────────
|
|
# Number of concurrent attempts this worker instance dequeues.
|
|
# Default 1. Multi-machine deployment: each worker instance is
|
|
# single-attempt; scale by adding instances rather than bumping
|
|
# this past your OpenCode concurrency ceiling.
|
|
CONTROLLER_MAX_CONCURRENT_WORKERS_PER_MACHINE=1
|
|
|
|
# How often this worker polls the DB for pending attempts.
|
|
CONTROLLER_WORKER_POLL_INTERVAL_S=5
|
|
|
|
# How often the in_progress heartbeat is updated (must be < the
|
|
# lock_ttl_seconds the master writes — currently 600s).
|
|
CONTROLLER_HEARTBEAT_INTERVAL_S=30
|
|
|
|
# ─── per-tier session timeouts ────────────────────────────────────
|
|
CONTROLLER_TIER_0_TIMEOUT_S=600
|
|
CONTROLLER_TIER_1_TIMEOUT_S=1200
|
|
CONTROLLER_TIER_2_TIMEOUT_S=1800
|
|
CONTROLLER_DEFAULT_AGENT_TIMEOUT_S=600
|
|
|
|
# ─── logging ──────────────────────────────────────────────────────
|
|
CONTROLLER_LOG_LEVEL=INFO
|