drew 751dbfed33 fix(agents): batch Q — tier-variant model injection + Option B (drop dead .txt files)
Two related fixes from empirical testing of OpenCode's model
resolution against the controller's tier-escalation ladder.

ROOT-CAUSE FINDING (empirical, 2026-05-18)
==========================================

Spun up a probe agent (.opencode/agents/model-probe.md), asked the
model to self-identify, and tried three model-routing mechanisms:

1. **POST /session ``model`` in body**: OpenCode 0.x silently
   IGNORES this — all probes returned ``openai/gpt-5.3-chat-latest``
   (OpenCode's fallback default), not the requested haiku/sonnet/opus.

2. **.md frontmatter ``model:`` line** (after OpenCode restart):
   HONORED — pinning to claude-haiku-4-5 yielded haiku, pinning to
   sonnet yielded sonnet, etc.

3. **opencode.json ``agent.<name>.model``**: HONORED — same result
   as .md frontmatter.

CONSEQUENCE: pre-batch-Q the tier variants had NO ``model:`` in
their .md frontmatter; the model lived only in
``.opencode/models/task-implementor-tier-<N>.txt`` files that the
dispatcher passed via POST /session body. Since OpenCode ignores
that pass-through, ALL FOUR tier variants ran on the SAME default
model (gpt-5.3 in this configuration) — the entire tier-escalation
ladder was cosmetic for model selection. The trial-2 sessions
logged ``model override -> claude-haiku-4-5`` but the actual
generation was on something else entirely.

WHAT THIS COMMIT DOES
=====================

1. **sync_tier_models.py rewrite** (already in batch P, refined here):
   inject ``model: <providerID/modelID>`` line into each generated
   tier variant's .md frontmatter. This is the mechanism OpenCode
   actually reads at startup. The model values come from tiers.yaml
   (source of truth).

2. **Drop the dead .opencode/models/task-implementor-tier-*.txt
   files** (Option B): the dispatcher pass-through they fed was
   empirically dead — OpenCode doesn't read the model from POST
   /session. ``sync_tier_models.py`` now removes any stale .txt
   files on each run (so a developer can't accidentally re-create
   them).

3. **Test updates**:
   - ``test_no_stale_variant_txt_files_remain``: pins that the .txt
     files stay deleted (was ``test_every_tier_has_a_variant_txt``).
   - ``test_each_variant_md_carries_correct_model_from_manifest``:
     pins that the .md frontmatter model: matches tiers.yaml (was
     ``test_each_variant_txt_matches_manifest_model``).
   - ``test_every_agent_file_reference_in_opencode_json_resolves``:
     relaxed to skip when opencode.json has no agent block (which is
     the Option B steady state). Still pins {file:...} resolution
     for any future use.
   - ``test_each_variant_md_matches_renderer_output_for_its_tier``:
     updated to call the new renderer signature ``render_task_
     implementer_variant(source, model)`` (was the byte-copy
     identity test, retired because variants now differ by the
     injected model: line).
   - ``TestTaskImplementorVariantsAreByteIdentical`` → renamed
     ``test_variants_identical_except_for_model_line``: strips the
     model: line via regex and asserts the rest is byte-identical.

OPERATOR WORKFLOW (unchanged surface)
=====================================

To swap a tier's model:
1. Edit ``.opencode/models/tiers.yaml`` (one line)
2. ``python3 tools/sync_tier_models.py`` (regenerates .md; removes
   any stale .txt)
3. Commit both files
4. Restart OpenCode (it caches .md frontmatter at startup)

What's still ahead (deferred):
- ``_resolve_role_model()`` in tools/_opencode_worker.py is now
  proven dead code (reads .txt files that don't exist; injects
  model into POST /session body that OpenCode ignores). Should be
  deleted in a follow-up — kept now to minimize blast radius.
- The misleading "if generation uses a different model, restart
  OpenCode so opencode.json's {file:...} re-resolves" log line in
  ``_opencode_worker.py:1569`` is wrong post-Option-B; should be
  retired with the dead code above.
- Whether opencode.json's agent block supports a ``permission`` field
  is the gate for an even deeper simplification (Option D). Skipped
  for now per operator direction; the controller path uses .md
  frontmatter for permissions.

3148 tests pass, 4 skipped, 0 failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 20:17:41 -04:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:14:00 -05:00
2024-01-25 23:14:00 -05:00
2026-04-21 19:44:48 +00:00

CleverAgents Core

CleverAgents is a Python-first automation platform. It provides a unified agents CLI, an interactive Textual TUI, embedded runtime, and service orchestration tools while embracing modern Python tooling.

Highlights

  • Unified CLI entry points: cleveragents and agents
  • Interactive TUI (agents tui) — full-screen Textual app with multi-session tabs, persona switching, slash commands, reference picker, and context-sensitive F1 help
  • Persona system — YAML-backed personas bind actors, argument presets, and scope references to named identities; persisted in ~/.config/cleveragents/personas/
  • Session management — create, list, export, and import conversation sessions; full JSON export/import for portability; Markdown transcript export (--format md) for human-readable sharing
  • First-run experienceActorSelectionOverlay guides new users to pick an actor on first TUI launch; creates a "default" persona automatically
  • Server modeagents server connect configures a remote CleverAgents server; Kubernetes Helm chart in k8s/ for production deployment
  • A2A integration — Agent-to-Agent protocol facade wires CLI and TUI to live application services (session, plan, registry, event)
  • Permissions screen — TUI overlay for reviewing tool permission requests with unified, side-by-side, and context diff views; session-scoped allow/reject decisions
  • Actor thought blocks — expandable reasoning trace widgets rendered inline in the conversation stream with muted styling
  • UKO runtime — Universal Knowledge Ontology query interface, inference engine, and graph persistence for ACMS context strategies
  • Database resource handler — full CRUD and checkpoint/rollback support for SQLite, PostgreSQL, MySQL, and DuckDB resources
  • Estimation lifecycleactor.default.estimation config key wires an estimation actor into the Strategize-to-Estimate lifecycle hook
  • Shell danger detection — TUI shell mode (! prefix) classifies commands by danger level (LOW → CRITICAL) and surfaces a warning overlay before executing destructive, privilege-escalating, or exfiltration-risk commands
  • Inline permission questionsPermissionQuestionWidget renders single-file permission requests directly in the conversation stream with single-key shortcuts
  • Invariant reconciliationInvariantReconciliationActor runs automatically at every plan phase transition; failures block the transition and emit INVARIANT_VIOLATED
  • UKO provenance tracking — every typed triple now carries sourceResource, validFrom, and isCurrent metadata; a revision chain enables temporal queries
  • JSON-RPC 2.0 A2A wire formatA2aRequest/A2aResponse fields renamed to standard JSON-RPC 2.0 names (method, id, result, error)
  • Fast Typer/Click-based interface with parity for help/version behavior
  • Behavior-driven coverage via Behave and Robot Framework
  • Nox automation for linting, typing, testing, docs, builds, and benchmarks
  • MkDocs-powered documentation with CleverAgents branding

Quick Start

# clone the CleverAgents core repository
git clone https://git.cleverthis.com/cleveragents/core.git
cd core

# install dependencies
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,tests,docs]"

# set up pre-commit hooks and verify tooling
bash scripts/setup-dev.sh

# verify the CLI
agents --help
agents --version

Launch the TUI

# install TUI extra
pip install -e ".[tui]"

# launch the interactive terminal UI
agents tui

Inside the TUI:

  • Type a message and press Enter to chat with the active actor
  • Press / to open the slash command overlay (67 commands across 14 groups)
  • Press @ to open the reference picker and insert file/resource references
  • Press ! to enter shell mode and run a subprocess command
  • Press F1 to toggle the context-sensitive help panel
  • Press Ctrl+T to cycle through argument presets for the active persona
  • Press Ctrl+Q to quit

Session management

agents session create --actor openai/gpt-4o
agents session list
agents session export --session-id <ID> --output session.json
agents session import --input session.json

Server mode

# connect to a remote CleverAgents server
agents server connect --url https://my-server.example.com --token <TOKEN>

# check connection status
agents server status

Developing

Pre-commit hooks run automatically on every git commit (formatting, linting, type checking, security scanning). To run checks manually:

# core validation
nox -s format            # ruff auto-formatting
nox -s lint              # ruff linting
nox -s typecheck         # pyright type checking
nox -s unit_tests        # behave unit tests
nox -s integration_tests # robot integration tests

# quality & security
nox -s security_scan     # bandit security scanning
nox -s dead_code         # vulture dead code detection
nox -s complexity        # radon complexity analysis
nox -s pre_commit        # run all pre-commit hooks
nox -s adr_compliance    # verify ADR compliance

For the full quality automation guide, see docs/development/quality-automation.md.

Documentation

nox -s docs
nox -s serve_docs

Tests

Behave feature scenarios live under features/ and Robot suites under robot/. Use the Nox sessions above to execute them in parity with the implementation plan.

Observability

LangSmith tracing is optional and off by default. Enable it by exporting CLEVERAGENTS_LANGSMITH_ENABLED=true along with a project name and API key (CLEVERAGENTS_LANGSMITH_PROJECT, CLEVERAGENTS_LANGSMITH_API_KEY). The settings module automatically mirrors these values to LANGCHAIN_TRACING_V2, LANGCHAIN_PROJECT, and LANGCHAIN_API_KEY, so LangChain/LangGraph agents emit traces without extra wiring. Additional knobs such as CLEVERAGENTS_LANGSMITH_ENDPOINT, CLEVERAGENTS_LANGSMITH_USER_ID, and CLEVERAGENTS_LANGSMITH_TAGS are documented in docs/observability.md.

LLM provider configuration

CleverAgents ships with a LangChain/LangGraph powered provider registry that discovers whichever API keys you export and automatically selects the best available provider. The CLI now uses actors: select an actor with --actor <name> (or set a default via agents actor set-default). Actors embed provider/model choices; built-in actors are seeded from CLEVERAGENTS_DEFAULT_PROVIDER / CLEVERAGENTS_DEFAULT_MODEL, then fall back to the built-in order (openai → anthropic → google → azure → openrouter → groq → together → cohere → gemini).

Required environment variables

Provider Primary variables
OpenAI OPENAI_API_KEY
Anthropic ANTHROPIC_API_KEY
Google GOOGLE_API_KEY or GOOGLE_GENAI_API_KEY
Azure OpenAI AZURE_OPENAI_API_KEY plus AZURE_OPENAI_ENDPOINT/AZURE_OPENAI_DEPLOYMENT
OpenRouter OPENROUTER_API_KEY (+ optional CLEVERAGENTS_OPENROUTER_ORGANIZATION for sanitized headers)
Gemini GEMINI_API_KEY or GOOGLE_GEMINI_API_KEY
Cohere COHERE_API_KEY
Groq GROQ_API_KEY
Together TOGETHER_API_KEY

Set CLEVERAGENTS_DEFAULT_PROVIDER to pin the global provider (for example export CLEVERAGENTS_DEFAULT_PROVIDER=openai) and CLEVERAGENTS_DEFAULT_MODEL to lock in a model ID. When unset, the registry picks the first configured provider and uses its published default model such as gpt-4o for OpenAI or claude-sonnet-4-20250514 for Anthropic.

Diagnostics and testing shortcuts

  • agents diagnostics prints whether the registry can see your credentials and which actor/provider is selected.
  • agents tell and agents build require --actor <name> unless a default actor is set; use agents actor set-default <name> to configure one.
  • Built-in actors (<provider>/<model>) are immutable, custom actors must be named local/<id>, and the default actor cannot be removed. Use --unsafe when adding/updating configs marked unsafe; runtime only warns when invoking unsafe actors.
  • CLEVERAGENTS_TESTING_USE_MOCK_AI=true forces the in-repo mock provider so Behave/Robot suites never hit external APIs.
  • The full capability matrix (streaming, tool calls, JSON mode, etc.) is documented in docs/reference/providers.md.
S
Description
test
Readme 84 MiB
Languages
Python 75.8%
Gherkin 18.3%
RobotFramework 4.8%
TypeScript 0.9%
Shell 0.2%