drew b154d48027 feat(auto-agents): deterministic worker-side improvements
Four changes that move judgment off the LLM and into the dispatcher,
motivated by the live PR #30 escalation pilot (2026-05-12): Tier 0
spent 16 min and gave up on flaky-looking unrelated tests; Tier 2
spent 88 min discovering the PR was already correct and only needed
two compliance entries. Together these collapse the typical
"PR is correct, only needs compliance fixups" case from 88 min on
Kimi to ~5 min on gpt-5-mini at Tier 0.

- Outcome-JSON synthesis in dispatch_implementer
  (_synthesize_outcome_if_missing): synthesises a concrete outcome
  from terminal_state when the worker emits no contract JSON,
  routing the escalation predicate to ESCALATE instead of UNKNOWN.

- Diff-aware gate parser (_diff_aware_gate.py): pure-Python
  classifier that splits failing BDD scenarios into related vs.
  unrelated to the PR's changed files via a feature-stem heuristic.

- Flaky-test pre-flight (_implementer_gate_preflight.py): runs
  local_ci_gate.sh --fast twice and surfaces only the persistent
  failures. Off-by-default behind IMPLEMENTER_DISPATCHER_GATE_PREFLIGHT.

- Compliance gap detector (_implementer_compliance.py): deterministic
  check of CHANGELOG / CONTRIBUTORS / commit-footer / worktree-clean
  state. Result is embedded as a "Compliance gap report" stanza so
  the agent fills in known gaps instead of discovering them.

Both prompt stanzas are appended via _append_deterministic_stanzas,
flag-gated, skipped in dry-run, and skipped when no preclone exists.
Flag-off path is byte-equivalent to the pre-feature build.

Suite: 1382 passed, 3 skipped (+59 new tests over 4 modules).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 00:39:39 -04:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:10:04 -05:00
2024-01-25 23:14:00 -05:00
2024-01-25 23:14:00 -05:00
2026-04-21 19:44:48 +00:00

CleverAgents Core

CleverAgents is a Python-first automation platform. It provides a unified agents CLI, an interactive Textual TUI, embedded runtime, and service orchestration tools while embracing modern Python tooling.

Highlights

  • Unified CLI entry points: cleveragents and agents
  • Interactive TUI (agents tui) — full-screen Textual app with multi-session tabs, persona switching, slash commands, reference picker, and context-sensitive F1 help
  • Persona system — YAML-backed personas bind actors, argument presets, and scope references to named identities; persisted in ~/.config/cleveragents/personas/
  • Session management — create, list, export, and import conversation sessions; full JSON export/import for portability; Markdown transcript export (--format md) for human-readable sharing
  • First-run experienceActorSelectionOverlay guides new users to pick an actor on first TUI launch; creates a "default" persona automatically
  • Server modeagents server connect configures a remote CleverAgents server; Kubernetes Helm chart in k8s/ for production deployment
  • A2A integration — Agent-to-Agent protocol facade wires CLI and TUI to live application services (session, plan, registry, event)
  • Permissions screen — TUI overlay for reviewing tool permission requests with unified, side-by-side, and context diff views; session-scoped allow/reject decisions
  • Actor thought blocks — expandable reasoning trace widgets rendered inline in the conversation stream with muted styling
  • UKO runtime — Universal Knowledge Ontology query interface, inference engine, and graph persistence for ACMS context strategies
  • Database resource handler — full CRUD and checkpoint/rollback support for SQLite, PostgreSQL, MySQL, and DuckDB resources
  • Estimation lifecycleactor.default.estimation config key wires an estimation actor into the Strategize-to-Estimate lifecycle hook
  • Shell danger detection — TUI shell mode (! prefix) classifies commands by danger level (LOW → CRITICAL) and surfaces a warning overlay before executing destructive, privilege-escalating, or exfiltration-risk commands
  • Inline permission questionsPermissionQuestionWidget renders single-file permission requests directly in the conversation stream with single-key shortcuts
  • Invariant reconciliationInvariantReconciliationActor runs automatically at every plan phase transition; failures block the transition and emit INVARIANT_VIOLATED
  • UKO provenance tracking — every typed triple now carries sourceResource, validFrom, and isCurrent metadata; a revision chain enables temporal queries
  • JSON-RPC 2.0 A2A wire formatA2aRequest/A2aResponse fields renamed to standard JSON-RPC 2.0 names (method, id, result, error)
  • Fast Typer/Click-based interface with parity for help/version behavior
  • Behavior-driven coverage via Behave and Robot Framework
  • Nox automation for linting, typing, testing, docs, builds, and benchmarks
  • MkDocs-powered documentation with CleverAgents branding

Quick Start

# clone the CleverAgents core repository
git clone https://git.cleverthis.com/cleveragents/core.git
cd core

# install dependencies
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,tests,docs]"

# set up pre-commit hooks and verify tooling
bash scripts/setup-dev.sh

# verify the CLI
agents --help
agents --version

Launch the TUI

# install TUI extra
pip install -e ".[tui]"

# launch the interactive terminal UI
agents tui

Inside the TUI:

  • Type a message and press Enter to chat with the active actor
  • Press / to open the slash command overlay (67 commands across 14 groups)
  • Press @ to open the reference picker and insert file/resource references
  • Press ! to enter shell mode and run a subprocess command
  • Press F1 to toggle the context-sensitive help panel
  • Press Ctrl+T to cycle through argument presets for the active persona
  • Press Ctrl+Q to quit

Session management

agents session create --actor openai/gpt-4o
agents session list
agents session export --session-id <ID> --output session.json
agents session import --input session.json

Server mode

# connect to a remote CleverAgents server
agents server connect --url https://my-server.example.com --token <TOKEN>

# check connection status
agents server status

Developing

Pre-commit hooks run automatically on every git commit (formatting, linting, type checking, security scanning). To run checks manually:

# core validation
nox -s format            # ruff auto-formatting
nox -s lint              # ruff linting
nox -s typecheck         # pyright type checking
nox -s unit_tests        # behave unit tests
nox -s integration_tests # robot integration tests

# quality & security
nox -s security_scan     # bandit security scanning
nox -s dead_code         # vulture dead code detection
nox -s complexity        # radon complexity analysis
nox -s pre_commit        # run all pre-commit hooks
nox -s adr_compliance    # verify ADR compliance

For the full quality automation guide, see docs/development/quality-automation.md.

Documentation

nox -s docs
nox -s serve_docs

Tests

Behave feature scenarios live under features/ and Robot suites under robot/. Use the Nox sessions above to execute them in parity with the implementation plan.

Observability

LangSmith tracing is optional and off by default. Enable it by exporting CLEVERAGENTS_LANGSMITH_ENABLED=true along with a project name and API key (CLEVERAGENTS_LANGSMITH_PROJECT, CLEVERAGENTS_LANGSMITH_API_KEY). The settings module automatically mirrors these values to LANGCHAIN_TRACING_V2, LANGCHAIN_PROJECT, and LANGCHAIN_API_KEY, so LangChain/LangGraph agents emit traces without extra wiring. Additional knobs such as CLEVERAGENTS_LANGSMITH_ENDPOINT, CLEVERAGENTS_LANGSMITH_USER_ID, and CLEVERAGENTS_LANGSMITH_TAGS are documented in docs/observability.md.

LLM provider configuration

CleverAgents ships with a LangChain/LangGraph powered provider registry that discovers whichever API keys you export and automatically selects the best available provider. The CLI now uses actors: select an actor with --actor <name> (or set a default via agents actor set-default). Actors embed provider/model choices; built-in actors are seeded from CLEVERAGENTS_DEFAULT_PROVIDER / CLEVERAGENTS_DEFAULT_MODEL, then fall back to the built-in order (openai → anthropic → google → azure → openrouter → groq → together → cohere → gemini).

Required environment variables

Provider Primary variables
OpenAI OPENAI_API_KEY
Anthropic ANTHROPIC_API_KEY
Google GOOGLE_API_KEY or GOOGLE_GENAI_API_KEY
Azure OpenAI AZURE_OPENAI_API_KEY plus AZURE_OPENAI_ENDPOINT/AZURE_OPENAI_DEPLOYMENT
OpenRouter OPENROUTER_API_KEY (+ optional CLEVERAGENTS_OPENROUTER_ORGANIZATION for sanitized headers)
Gemini GEMINI_API_KEY or GOOGLE_GEMINI_API_KEY
Cohere COHERE_API_KEY
Groq GROQ_API_KEY
Together TOGETHER_API_KEY

Set CLEVERAGENTS_DEFAULT_PROVIDER to pin the global provider (for example export CLEVERAGENTS_DEFAULT_PROVIDER=openai) and CLEVERAGENTS_DEFAULT_MODEL to lock in a model ID. When unset, the registry picks the first configured provider and uses its published default model such as gpt-4o for OpenAI or claude-sonnet-4-20250514 for Anthropic.

Diagnostics and testing shortcuts

  • agents diagnostics prints whether the registry can see your credentials and which actor/provider is selected.
  • agents tell and agents build require --actor <name> unless a default actor is set; use agents actor set-default <name> to configure one.
  • Built-in actors (<provider>/<model>) are immutable, custom actors must be named local/<id>, and the default actor cannot be removed. Use --unsafe when adding/updating configs marked unsafe; runtime only warns when invoking unsafe actors.
  • CLEVERAGENTS_TESTING_USE_MOCK_AI=true forces the in-repo mock provider so Behave/Robot suites never hit external APIs.
  • The full capability matrix (streaming, tool calls, JSON mode, etc.) is documented in docs/reference/providers.md.
S
Description
test
Readme 84 MiB
Languages
Python 75.8%
Gherkin 18.3%
RobotFramework 4.8%
TypeScript 0.9%
Shell 0.2%