cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity
2,443 Commits 1,418 Branches 10 Tags
ae940f45644971d5937b755d2459eceffeb6500a
Commit Graph

8 Commits

Author SHA1 Message Date
drew 91073ef104 chore(auto-agents): rebind default + tier-0 to local-claude/claude-haiku-4-5 
Both ``.opencode/models/default.txt`` (the agent default model when
no tier slot is specified) and the tier-0 entry in
``.opencode/models/tiers.yaml`` were pointing at OpenAI's GPT-5
family (gpt-5-mini and gpt-5-nano respectively). Rebinds to
``local-claude/claude-haiku-4-5`` to keep the default workload on
the local proxy at the per-million rate:

  $1 in / $5 out / $0.10 cached  (Haiku 4.5)

versus

  $0.25 in / $2.00 out / $0.025 cached  (gpt-5-mini)
  $0.05 in / $0.40 out / $0.005 cached  (gpt-5-nano)

Haiku-4-5 is more expensive per-token but Anthropic's prompt
caching (90% off cached input) typically wins for workloads with
large repeated system prompts — which the auto-agents pipeline
absolutely has. The actual cost/PR comparison will be visible
once a few cycles have run through the now-functional cost
dashboard (telemetry commit 62d4e8f07).

No code changes — pure configuration. Revert is one-line if the
observed cost profile favours the OpenAI defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 00:18:18 -04:00
drew a6610a022c feat(auto-agents): launcher sidecars for conflict-drive + merge-drive 
Two new long-lived sidecars folded into the dispatcher launcher,
each gated by an env var for safe enable/disable:

**Conflict driver** (``DISPATCHERS_DISABLE_CONFLICT_DRIVE``, default
enabled). Watches Forgejo for ``auto/needs-conflict-resolution``
labels, dispatches ``conflict-resolver-worker`` (the new agent;
bound to ``local-claude/claude-opus-4-6`` for high-quality rebases)
to rebase + resolve conflicts, force-with-leases the result, and
clears the label. Decoupled from approval — keeps PRs mergeable
so the merge driver doesn't have to wait at approval time.

**Merge driver** (``DISPATCHERS_DISABLE_MERGE_DRIVE``, default
DISABLED in this test launcher to avoid accidental merges during
validation runs). Watches Forgejo for APPROVED PRs without blocking
labels, rebases against current master, runs the local CI gate,
merges via squash with force-with-lease semantics. Terminal stage
of the pipeline.

Both sidecars follow the existing warmer pattern: best-effort
respawn on failure, no contribution to the dispatcher crash-loop
budget, clean shutdown on launcher exit. PRs that get
``auto/needs-conflict-resolution`` would sit forever without the
conflict-driver sidecar — the implementer and reviewer don't act
on that label.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 00:05:47 -04:00
drew 80d61de942 feat(auto-agents): R3 wrapper-chain retirement — direct task-implementor variants 
Eliminates the remaining LLM wrapper chain (``tier-dispatcher`` +
``tier-{min,0,1,2}`` selectors) between the Python dispatcher and the
``task-implementor`` worker. Follows the R2 implementation-worker
retirement (6e63073ad, 2026-05-16); both wrappers were pure routing
agents with no per-cycle judgment that could not be moved to Python.

Architecture
------------

Before (R2 baseline):
  dispatch_implementer.py
    → tier-dispatcher (LLM)
        → estimator-implementation (LLM, judgment)
        → tier-N selector (LLM, pure pass-through)
            → task-implementor (LLM, the actual work, via `task` hop)

After (R3):
  dispatch_implementer.py
    → estimator-implementation (LLM, judgment — invoked top-level)
    → task-implementor-tier-N (LLM, the actual work, NO `task` hops)

Two LLM hops eliminated per cycle. The ``task`` tool hop between the
tier-N selector and task-implementor is gone too, so the dispatcher's
prefetched ``## Pre-fetched …`` sections survive intact in the
worker's prompt — closing the structural cause of the ~30-80
per-session ``implementer_pr_context.py read --pr N`` round-trips
the worker burned to recover summarised-away context.

Cost savings (4-day measurement window, $-figures based on
local-claude pricing with caching):

- Eliminating tier-dispatcher sessions (32/day): ~$5-15/day
- Eliminating tier-N selector sessions (15/day): ~$2-5/day
- Eliminating prefetch round-trips (229/4d → expected near 0): ~$20-40/day

Aggregate at current traffic: roughly $30-60/day, $900-1,800/month.

What changed
------------

1. **New ``sync_tier_models.py`` scope** — generates per-tier
   ``task-implementor-{slot}.md`` + matching
   ``.opencode/models/task-implementor-{slot}.txt`` files from
   ``task-implementor.md`` (the byte source). Dropped: the bare
   ``tier-N.txt`` model files (no consumer) and the
   tier-dispatcher.md mapping-table generation (no file).

2. **New ``_call_python_estimator``** in dispatch_implementer.py
   invokes ``estimator-implementation`` as a top-level OpenCode
   session, parses ``{is_confident, recommended_tier}``, returns the
   tier integer or None. Includes a heartbeat-refresh on_poll so a
   30-180 s estimator call cannot trigger the launcher's hung-
   process watchdog. Estimator switched from ``mode: subagent`` to
   ``mode: all`` so the dispatcher can spawn it directly.

3. **New ``_resolve_task_implementor_for_tier(tier)`` helper** maps
   manifest tier integers to the matching ``task-implementor-{slot}``
   variant. Used by both the initial dispatch (in the prompt
   factory) and the in-cycle escalation respawn.

4. **WorkGroup contract extended** with
   ``requires_worker_agent_override: bool`` (default False, opt-in
   per group). The implementer's three WorkGroups set True;
   ``_resolve_effective_worker_agent`` raises a clear RuntimeError
   if the prompt_factory failed to populate the override (a code
   bug that would otherwise silently run every cycle at the static
   fallback tier).

5. **``_implementation_prompt_dispatch`` refactored** to:
   - Resolve the tier in Python (label-driven hint → estimator →
     default 0), honouring both the in-cycle escalation flag and the
     estimator-enabled flag.
   - Stash the resolved ``task-implementor-tier-<slot>`` agent name
     on the item context under
     ``WORKER_AGENT_OVERRIDE_ITEM_KEY`` (single source of truth in
     ``_dispatch_runtime``; imported into the higher layer).
   - Emit the worker body with ``escalation_tier: \`N\``` directly —
     no more ``escalation_tier_hint``, ``task_prompt:`` fence, or
     ``task_agent:``/``estimator_agent:`` outer parameters (all
     consumed by the retired tier-dispatcher).
   - Skip the estimator call on ``--dry-run`` so the operator-
     visible no-I/O contract holds.

6. **Retired agent files DELETED**:
   - ``.opencode/agents/tier-dispatcher.md``
   - ``.opencode/agents/tier-{min,0,1,2}.md``
   - ``.opencode/models/tier-{min,0,1,2}.txt``
   - Matching entries in ``opencode.json``'s agent block.

7. **Prose updates** to ``task-implementor.md`` (the byte-source for
   variants), ``estimator-implementation.md``, and production
   docstrings (``_block_store.py``, ``_pr_context_sentinel.py``,
   ``implementer_workspace.py``, ``_review_post.py``,
   ``_review_finalize.py``) reflecting the post-R3 chain. The
   filesystem handoff scripts (``implementer_pr_context.py``,
   ``implementer_workspace.py``) remain in place as the canonical
   read path — defensive against any future regression that re-
   introduces summarisation.

Tests
-----

2262 auto_agents passing (was 2268 pre-R3; net -6 from
removing tests pinning the retired wrapper-chain contract,
offset by +14 new tests pinning the post-R3 contract):

- ``TestEstimatorEnabledFlag`` rewritten to assert
  ``escalation_tier`` + agent-override semantics.
- New ``TestEstimatorPromptShape`` (5 tests) pins the body shape
  the Python estimator helper passes to the agent and the
  call shape into ``run_session_blocking``.
- New ``TestResolveEffectiveWorkerAgent`` (8 tests) directly
  covers the override priority chain — override present, empty,
  whitespace, non-string, whitespace-stripped, required-but-missing
  (loud fail), required-and-present.
- ``test_dry_run_never_calls_estimator`` pins the dry-run no-I/O
  contract via an exploding-stub guard on the estimator helper.
- ``TestDirectTierDispatch`` replaces the retired
  ``TestTierDispatcherShortCircuit`` suite in
  ``test_worker_permissions.py``.
- ``TestTaskImplementorVariantsAreByteIdentical`` ensures the
  four per-tier variants never hand-diverge from each other.
- ``test_no_legacy_tier_agents_in_opencode_agent_block`` fails
  loudly if any of the retired tier-* entries are re-introduced
  to ``opencode.json``.

Operator notes
--------------

- The C3 footgun (model swaps need OpenCode restart) still applies
  to the generated variants — edit ``tiers.yaml``, re-run
  ``python3 tools/sync_tier_models.py``, then restart OpenCode.
- The estimator now runs as a top-level OpenCode session; an
  operator grepping the session archive will see
  ``[AUTO-IMP-PR-N-estimator] estimator-implementation`` entries
  alongside the worker sessions.
- Roll-back: revert this commit + the R3 prep commit (b8c1e4903).
  Both wrappers + the static-fallback ``worker_agent`` come back;
  no schema migration needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 15:03:30 -04:00
drew b8c1e49032 feat(auto-agents): R3 prep — generate per-tier task-implementor variants 
Prep step for retiring the tier-dispatcher + tier-N wrapper chain
(R3, follow-up to R2's implementation-worker retirement at 6e63073ad).
The cutover commit will swap the dispatcher to invoke these variants
directly; this commit adds the variants without behavior change.

What changed:

1. **sync_tier_models.py extended** to generate
   ``task-implementor-tier-{slot}.md`` + matching
   ``.opencode/models/task-implementor-tier-{slot}.txt`` for every
   tier in ``tiers.yaml``. The ``.md`` body is a byte copy of
   ``task-implementor.md`` (the source of truth) prefixed with an
   HTML-comment generation header. The ``.txt`` carries the slot's
   model, identical to the matching tier-N selector's ``.txt``.

2. **opencode.json agent block** gets four new entries (one per
   variant) wiring each ``task-implementor-tier-{slot}`` to its
   ``.opencode/models/<name>.txt`` via the same ``{file:...}``
   interpolation the existing tier-N selectors use. This is the
   one-time refactor edit the generator's docstring already
   documents — operators add/remove ``opencode.json`` agent block
   entries when tiers are added or removed.

3. **Drift-detection tests** in test_tier_model_registry.py for the
   new variants: existence (md + txt), byte-copy invariant against
   the source, model match against the manifest, and opencode.json
   wiring.

The per-tier variants exist because OpenCode resolves a session's
model from ``agent.<name>.model`` at startup — there is no
per-session model override (documented C3 footgun at 1635229828).
Each tier needs a distinct agent name to hit a distinct model slot;
the variants give the dispatcher that handle without going through a
tier-N pass-through agent.

Tests: 2268 auto_agents passing (was 2268 before; +5 new variant
drift tests offset by no regressions). The variants are not yet
invoked by any code path — that's the cutover commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 13:34:59 -04:00
drew 1635229828 docs(auto-agents): document model-override-needs-restart footgun (C3) 
Replaces the README's optimistic "Changes to a .txt file take effect
on the next dispatched session — no OpenCode restart needed" with
the actual behaviour: BOTH paths require an OpenCode restart for a
.txt edit to change what model OpenCode generates with. The
dispatcher passes the resolved model on every POST /session
(observability / consistency check), but OpenCode itself re-resolves
agent.<name>.model from its startup-cached opencode.json on every
generation. Without a restart, the dispatcher logs say one model
ran and OpenCode actually ran another — a silent regression
invisible from the dispatcher side.

This is the same footgun the long comment in
_opencode_worker.run_session_blocking documents inline; the README
was the missing place where an operator naturally looks before
editing a model file.

Cross-links the consequence for G11 (estimator-driven adaptive
tier selection) and the in-cycle escalation plan — both silently
misbehave if a model swap lands without a restart (a Tier 1
escalation would run on the cached Tier 0 model).

Refs: docs/development/final-working-harvest-plan.md (C3).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 20:20:42 -04:00
drew 1eac4ea233 refactor(auto-agents): manifest-driven tier-model registry, slot-based naming 
Replace the scattered per-agent .txt-file mapping (whose names embedded
model-family identity like tier-qwen-med and tier-kimi and went stale
the moment a model was swapped) with a single source-of-truth manifest
at .opencode/models/tiers.yaml. The four tier slots get model-agnostic
slot-based names (tier-min, tier-0, tier-1, tier-2); the model in each
slot is configured ONLY in the manifest.

Derived artifacts (per-agent .txt files and the mapping table block in
tier-dispatcher.md) are now generated by tools/sync_tier_models.py.
A drift-detection test in tests/auto_agents/test_tier_model_registry.py
fails CI if any derived file diverges from the manifest, if an agent
referenced by the manifest lacks an agent file, if the manifest cites
a provider not declared in opencode.json, or if opencode.json carries
a stale tier-* entry.

To swap a model in a slot: edit tiers.yaml -> run
python3 tools/sync_tier_models.py -> commit. The runtime dispatcher
re-reads the .txt files per cycle (no restart); the static OpenCode
config path needs a server restart.

Tier rename mapping (escalation_tier integers UNCHANGED):
  tier-qwen-small  -> tier-min  (slot -1)
  tier-qwen-med    -> tier-0    (slot  0, default first attempt)
  tier-qwen-large  -> tier-1    (slot  1)
  tier-kimi        -> tier-2    (slot  2)

Vestigial tier-* agents removed (declared but never in the active
mapping): tier-haiku, tier-sonnet, tier-opus, tier-codex,
tier-gpt5-mini, tier-gpt5-nano, tier-o4-mini.

estimator-implementation.md now reasons in capability descriptors
(cheapest / default / advanced / complex) instead of model-family
labels (qwen-small / qwen-med / qwen-large / kimi), so the estimator
stays correct across model swaps. The stale "default tier = gpt-5-mini"
docstring claim (already drifted to claude-haiku-4-5) is removed.

Companion prose updates across every consumer of tier names:
- Agent prompts: tier-dispatcher.md, implementation-worker.md,
  estimator-implementation.md
- Skills: implementer-pr-context, implementer-workspace
- Python: dispatch_implementer.py, _opencode_worker.py,
  _pr_context_sentinel.py, implementer_workspace.py,
  setup_auto_labels.py, _attempt_history.py
- Tests: test_worker_permissions.py (parametrize list + byte-identity
  test now covers 4 slot files instead of 3 family-named files),
  test_opencode_worker_models.py (synthetic-fixture names updated)
- Docs: .opencode/models/README.md, docs/development/models.md,
  docs/development/agent-system-specification.md,
  docs/development/auto-agents-tier-2-3-plan.md,
  docs/development/implementer-in-cycle-escalation-plan.md,
  docs/development/final-working-harvest-plan.md

Validation: 1625 tests pass (+6 net new from the tier-registry test
file), 3 skipped. python3 tools/sync_tier_models.py --check exits 0.
local_ci_gate.sh --gate lint PASS.

Files: 9 added, 22 deleted, 18 modified. The drift-detection test
ran green on every step of the refactor, catching one out-of-sync
.txt file before commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 12:49:31 -04:00
drew 706178d54b fix(auto-agents): tier-selector pass-through lockdown + local-claude provider 
Three coupled fixes from the run-3/4/5 forensic sequence, plus the
local-claude model routing they depend on.

PROVIDER (validated in run-5):
  * opencode.json: local-claude provider uses @ai-sdk/anthropic (was
    @ai-sdk/openai-compatible). The OpenAI-compat adapter dropped
    streaming tool-call args on ~58% of responses — OpenCode saw
    bash({}) schema-errors. The native Anthropic SDK parses the
    proxy's input_json_delta stream correctly. Run-5 confirmed: tool
    args now flow intact (full {command,description} fields observed).
  * apiKey reads {env:LOCAL_ANTHROPIC_API_KEY} (not ANTHROPIC_API_KEY)
    so a shell with a real Anthropic key set cannot leak it to the
    local proxy.
  * .opencode/models/tier-{qwen-med,qwen-large,kimi}.txt point at
    local-claude/claude-{haiku,sonnet,opus}-4-x. These REQUIRE the
    provider block above — committed together to avoid a non-bootable
    intermediate state.

TIER-SELECTOR LOCKDOWN (applied, unvalidated — see below):
  * tier-{qwen-med,qwen-large,kimi}.md: every tool except `task` is
    now denied at the permission-engine level. Run-5 showed an agentic
    model (claude-haiku-4-5) given Read/Grep/Edit/Bash will do the
    implementer work *inside the tier selector*, bypassing
    task-implementor and thrashing against the selector's restrictive
    perms (20+ denials in one Tier 0 cycle; task-implementor never
    spawned). gpt-5-mini honoured the "pure pass-through" prompt;
    Claude-family models do not. The fix makes pass-through
    structural, not prompt-dependent. The 60-line alphabet-kludge
    edit block is also gone.

TASK-IMPLEMENTOR PERMISSION ORDERING (applied, unvalidated):
  * task-implementor.md: edit/write/external_directory rules reordered
    so specific allows precede `*: deny` — the OpenCode permission
    engine evaluates path-perms first-match-wins (confirmed empirically
    from run-4's denial dump; no OpenCode docs exist for this). Path
    globs widened /tmp/* -> /tmp/** since the worktree path is
    multi-segment under /tmp/. This is what was blocking edit/write on
    the dispatcher's pre-cloned worktree.

VALIDATION STATUS: the provider swap is confirmed by run-5 observation.
The two permission changes are UNVALIDATED — run-5 never reached
task-implementor because the tier selector consumed the whole cycle.
The tier-selector lockdown is precisely what unblocks reaching
task-implementor, so the next run validates both at once.

TEST: tests/auto_agents/test_worker_permissions.py gains
TestTierSelectorPassThroughPermissions — 10 tests pinning that all
work-tools are denied, only `task: task-*` is allowed, the bash block
has zero allow rules, and the three selector files stay byte-identical
(they are maintained as a unit; copy-paste drift is the failure mode).
Full suite: 25/25 in test_worker_permissions.py, 1499 passing in
tests/auto_agents/ (3 pre-existing failures in test_pr_comments_cache.py
are a date-rollover time-bomb unrelated to this change).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 00:57:11 -04:00
drew 132a5a2269 feat(auto-agents): centralise model registry under .opencode/models with session-create model stamp 
Every agent's model assignment now lives in a single-line text file at
.opencode/models/<name>.txt; default.txt is the 27-agent catch-all.
opencode.json's agent.<name>.model uses
{file:./.opencode/models/<name>.txt} interpolation, and
tools/_opencode_worker.py reads the same files at session-create to
stamp the resolved model on the session record (observability + drift
sentinel; OpenCode does NOT propagate session-level model to
prompt_async — schema for that is undocumented and deferred to Stage
2). 39 .md frontmatter `model:` lines stripped; the two intentional
inheritors (task-implementor, agent-evolution-pool-supervisor) keep
their model-less frontmatter.

Operator workflow for swapping a model is now: edit
.opencode/models/<role>.txt, restart OpenCode so opencode.json's
{file:...} re-resolves, run the dispatcher. Live-swap without restart
was attempted (override on prompt_async); OpenCode 0.x silently
dropped those requests (200 OK, no assistant message) and the
prompt_async override was reverted. The session-create override
remains for observability + drift detection.

End-to-end validation (2026-05-10): dispatch_review.py on PR #25 with
default.txt=openai/gpt-5-mini produced a clean REQUEST_CHANGES review
in 26 s for ~$0.016; dispatch_implementer.py on PR #30 with
tier-qwen-* files remapped to openai/{gpt-5-nano, gpt-5-mini,
gpt-5-codex} ran the full implementation-worker → tier-dispatcher →
estimator-implementation → tier-qwen-med → task-implementor →
git-isolator-util chain in 16 min with model=gpt-5-mini end-to-end.

Also documents the printenv VAR form as the only allowed env-read in
implementation-worker.md and task-implementor.md (live testing
showed the worker burning 2–4 turns on permission-denied
trial-and-error trying printf and echo variants).

Tests: 21 in tests/auto_agents/test_opencode_worker_models.py
(resolver semantics with caplog assertions on every malformed-input
path; session-create body shape; prompt_async body never carries
model; three repo-level invariants — every {file:...} reference
resolves, every .md is wired or in the inheritor allowlist, no .md
has a model: frontmatter). 1027 auto_agents tests pass / 3 skipped.

Note: .opencode/models/default.txt and the three tier-qwen-*.txt
files are committed with their OpenAI swaps in place (gpt-5-mini,
gpt-5-nano, gpt-5-mini, gpt-5-codex respectively) because the
CleverThis HuggingFace endpoints are paused. Revert with `git diff
HEAD~1 -- .opencode/models/*.txt | git apply -R` if/when they come
back.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-10 17:23:06 -04:00