cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity
Files
5bdca88eca82dfee757df95bf37ae34bf86df951
cleveragents-core/docs/reference/automation_profiles.md
T
freemo 3bd02a7c6e feat(cli): add automation-profile commands 
Add CLI command group `agents automation-profile` with add, remove,
list, and show subcommands for managing automation profiles that
control plan execution autonomy.

Changes:
- New `automation_profile.py` CLI module with YAML config input,
  schema_version guard, namespaced name validation, --update support,
  and all output formats (json/yaml/plain/table/rich)
- Register automation-profile in main CLI app
- Add deprecation warnings for --automation-level on plan use and
  set-automation-level commands
- Update CLI reference docs with command examples, built-in profiles
  list, and deprecation notes
- 26 Behave scenarios covering all commands and error paths
- 9 Robot Framework integration smoke tests
- ASV benchmarks for CLI parsing performance
- Mark A6.cli items complete in implementation_plan.md
2026-02-20 08:50:06 -05:00

7.4 KiB
Raw Blame History

Automation Profiles

Automation Profiles control how much autonomy the CleverAgents system has at each phase of plan execution. Each profile defines a set of threshold values and safety requirements that determine when the system may proceed automatically versus when it must wait for human approval.

Threshold Semantics

Each threshold field is a float in the range [0.0, 1.0]:

Value Meaning
0.0 Fully automatic — no human gate required
1.0 Always requires human approval
0.0 < v < 1.0 Probabilistic — the system may proceed if its confidence exceeds the threshold

Threshold Fields

Field Category Description
auto_strategize Phase transition Gate before entering the strategy phase
auto_execute Phase transition Gate before entering the execution phase
auto_apply Phase transition Gate before applying changes
auto_decisions_strategize Decision autonomy Gate for decisions during strategy
auto_decisions_execute Decision autonomy Gate for decisions during execution
auto_validation_fix Self-repair Gate for automatic validation fixes
auto_strategy_revision Self-repair Gate for automatic strategy revision
auto_reversion_from_apply Self-repair Gate for reverting from the apply phase
auto_child_plans Child plans Gate for spawning child plans
auto_retry_transient Retry Gate for retrying transient failures
auto_checkpoint_restore Checkpoint Gate for automatic checkpoint restoration

Safety Fields

Field Type Default Description
require_sandbox bool true Execution must happen in a sandbox
require_checkpoints bool true Checkpoints must be created before writes
allow_unsafe_tools bool false Tools flagged as unsafe may be invoked

Built-in Profiles

Eight profiles ship with every CleverAgents installation:

Flag manual review supervised cautious trusted auto ci full-auto
auto_strategize 1.0 0.0 0.0 0.7 0.0 0.0 0.0 0.0
auto_execute 1.0 0.0 1.0 0.7 0.0 0.0 0.0 0.0
auto_apply 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0
auto_decisions_strategize 1.0 1.0 0.0 0.6 0.0 0.0 0.0 0.0
auto_decisions_execute 1.0 1.0 1.0 0.8 0.0 0.0 0.0 0.0
auto_validation_fix 1.0 1.0 1.0 0.7 0.0 0.0 0.0 0.0
auto_strategy_revision 1.0 1.0 1.0 0.8 1.0 0.0 0.0 0.0
auto_reversion_from_apply 1.0 1.0 1.0 0.9 1.0 1.0 0.0 0.0
auto_child_plans 1.0 0.0 1.0 0.7 0.0 0.0 0.0 0.0
auto_retry_transient 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
auto_checkpoint_restore 1.0 1.0 1.0 0.6 1.0 0.0 0.0 0.0
require_sandbox true true true true true true true false
require_checkpoints true true true true true true true false
allow_unsafe_tools false false false false false false false true

Profile Descriptions

  • manual — Human approves every action. Maximum safety, minimum autonomy.
  • review — Strategy and execution proceed automatically; human reviews before apply. Good default for development.
  • supervised — Human reviews strategy and execution decisions. Strategy creation itself is automatic.
  • cautious — Probabilistic gates on most actions. The system proceeds when confident, asks when unsure.
  • trusted — Automatic for most phases, but human approval required for apply and reversion. Suitable for experienced teams.
  • auto — Fully automatic except reversion from apply. Suitable for well-tested pipelines.
  • ci — Designed for CI/CD pipelines. All thresholds at 0.0 but sandbox and checkpoints remain required.
  • full-auto — No gates, no sandbox, no checkpoints, unsafe tools allowed. Use with extreme caution.

Resolution Precedence

When determining which profile applies to a given plan execution, the system resolves profiles in the following order (highest priority first):

  1. Plan-level override — A profile specified directly on the plan.
  2. Project-level setting — The default profile configured for the project.
  3. Organization-level default — The organization's default profile.
  4. System default — Falls back to the review built-in profile.

At each level, the profile may be specified by:

  • A built-in name (e.g. cautious)
  • A namespaced custom profile (e.g. acme/strict)

Custom Profiles

Custom profiles use a namespace/name naming convention:

name: acme/strict
description: Strict profile for production deployments
schema_version: "1.0"

auto_strategize: 0.9
auto_execute: 0.9
auto_apply: 1.0
auto_decisions_strategize: 0.8
auto_decisions_execute: 0.9
auto_validation_fix: 0.5
auto_strategy_revision: 0.9
auto_reversion_from_apply: 1.0
auto_child_plans: 0.8
auto_retry_transient: 0.3
auto_checkpoint_restore: 0.7

require_sandbox: true
require_checkpoints: true
allow_unsafe_tools: false

See docs/schema/automation_profile.schema.yaml for the full YAML schema and examples/profiles/ for example configurations.

CLI Commands

The agents automation-profile command group manages automation profiles from the terminal.

automation-profile add

Register a custom profile from a YAML configuration file:

agents automation-profile add --config ./profiles/strict.yaml
agents automation-profile add --config ./profiles/strict.yaml --update
agents automation-profile add --config ./profiles/strict.yaml --format json

Options:

  • --config / -c — Path to the YAML configuration file (required).
  • --update — Overwrite an existing custom profile instead of raising a conflict error.
  • --format / -f — Output format: json, yaml, plain, table, or rich (default: rich).

automation-profile remove

Remove a custom profile by name (built-in profiles cannot be removed):

agents automation-profile remove acme/strict --yes
agents automation-profile remove acme/strict --yes --format json

Options:

  • --yes / -y — Skip the interactive confirmation prompt.
  • --format / -f — Output format.

automation-profile list

List all profiles with optional filtering:

agents automation-profile list
agents automation-profile list --namespace acme
agents automation-profile list "caut.*"
agents automation-profile list --format json

Options:

  • regex (positional) — Optional regex pattern to filter profile names.
  • --namespace / -n — Filter by the namespace portion of namespaced names.
  • --format / -f — Output format.

automation-profile show

Display full details for a single profile:

agents automation-profile show manual
agents automation-profile show acme/strict --format yaml

Options:

  • --format / -f — Output format.

Deprecation: --automation-level

The --automation-level option on plan use and the plan set-automation-level command are deprecated and will be removed in a future release.

Legacy automation levels are mapped to built-in profiles:

Legacy level Built-in profile
manual manual
supervised supervised
auto auto
full_auto full-auto

Use --automation-profile <name> instead of --automation-level.

Reference in New Issue View Git Blame Copy Permalink