forked from cleveragents/cleveragents-core
freemo
3bd02a7c6e
Add CLI command group `agents automation-profile` with add, remove, list, and show subcommands for managing automation profiles that control plan execution autonomy. Changes: - New `automation_profile.py` CLI module with YAML config input, schema_version guard, namespaced name validation, --update support, and all output formats (json/yaml/plain/table/rich) - Register automation-profile in main CLI app - Add deprecation warnings for --automation-level on plan use and set-automation-level commands - Update CLI reference docs with command examples, built-in profiles list, and deprecation notes - 26 Behave scenarios covering all commands and error paths - 9 Robot Framework integration smoke tests - ASV benchmarks for CLI parsing performance - Mark A6.cli items complete in implementation_plan.md
188 lines
7.4 KiB
Markdown
188 lines
7.4 KiB
Markdown
# Automation Profiles
|
|
|
|
Automation Profiles control how much autonomy the CleverAgents system has at each phase of plan execution. Each profile defines a set of **threshold values** and **safety requirements** that determine when the system may proceed automatically versus when it must wait for human approval.
|
|
|
|
## Threshold Semantics
|
|
|
|
Each threshold field is a float in the range `[0.0, 1.0]`:
|
|
|
|
| Value | Meaning |
|
|
|-------|---------|
|
|
| `0.0` | Fully automatic — no human gate required |
|
|
| `1.0` | Always requires human approval |
|
|
| `0.0 < v < 1.0` | Probabilistic — the system may proceed if its confidence exceeds the threshold |
|
|
|
|
### Threshold Fields
|
|
|
|
| Field | Category | Description |
|
|
|-------|----------|-------------|
|
|
| `auto_strategize` | Phase transition | Gate before entering the strategy phase |
|
|
| `auto_execute` | Phase transition | Gate before entering the execution phase |
|
|
| `auto_apply` | Phase transition | Gate before applying changes |
|
|
| `auto_decisions_strategize` | Decision autonomy | Gate for decisions during strategy |
|
|
| `auto_decisions_execute` | Decision autonomy | Gate for decisions during execution |
|
|
| `auto_validation_fix` | Self-repair | Gate for automatic validation fixes |
|
|
| `auto_strategy_revision` | Self-repair | Gate for automatic strategy revision |
|
|
| `auto_reversion_from_apply` | Self-repair | Gate for reverting from the apply phase |
|
|
| `auto_child_plans` | Child plans | Gate for spawning child plans |
|
|
| `auto_retry_transient` | Retry | Gate for retrying transient failures |
|
|
| `auto_checkpoint_restore` | Checkpoint | Gate for automatic checkpoint restoration |
|
|
|
|
### Safety Fields
|
|
|
|
| Field | Type | Default | Description |
|
|
|-------|------|---------|-------------|
|
|
| `require_sandbox` | bool | `true` | Execution must happen in a sandbox |
|
|
| `require_checkpoints` | bool | `true` | Checkpoints must be created before writes |
|
|
| `allow_unsafe_tools` | bool | `false` | Tools flagged as `unsafe` may be invoked |
|
|
|
|
## Built-in Profiles
|
|
|
|
Eight profiles ship with every CleverAgents installation:
|
|
|
|
| Flag | manual | review | supervised | cautious | trusted | auto | ci | full-auto |
|
|
|------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
|
| auto_strategize | 1.0 | 0.0 | 0.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_execute | 1.0 | 0.0 | 1.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_apply | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 |
|
|
| auto_decisions_strategize | 1.0 | 1.0 | 0.0 | 0.6 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_decisions_execute | 1.0 | 1.0 | 1.0 | 0.8 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_validation_fix | 1.0 | 1.0 | 1.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_strategy_revision | 1.0 | 1.0 | 1.0 | 0.8 | 1.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_reversion_from_apply | 1.0 | 1.0 | 1.0 | 0.9 | 1.0 | 1.0 | 0.0 | 0.0 |
|
|
| auto_child_plans | 1.0 | 0.0 | 1.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_retry_transient | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
|
|
| auto_checkpoint_restore | 1.0 | 1.0 | 1.0 | 0.6 | 1.0 | 0.0 | 0.0 | 0.0 |
|
|
| require_sandbox | true | true | true | true | true | true | true | false |
|
|
| require_checkpoints | true | true | true | true | true | true | true | false |
|
|
| allow_unsafe_tools | false | false | false | false | false | false | false | true |
|
|
|
|
### Profile Descriptions
|
|
|
|
- **manual** — Human approves every action. Maximum safety, minimum autonomy.
|
|
- **review** — Strategy and execution proceed automatically; human reviews before apply. Good default for development.
|
|
- **supervised** — Human reviews strategy and execution decisions. Strategy creation itself is automatic.
|
|
- **cautious** — Probabilistic gates on most actions. The system proceeds when confident, asks when unsure.
|
|
- **trusted** — Automatic for most phases, but human approval required for apply and reversion. Suitable for experienced teams.
|
|
- **auto** — Fully automatic except reversion from apply. Suitable for well-tested pipelines.
|
|
- **ci** — Designed for CI/CD pipelines. All thresholds at 0.0 but sandbox and checkpoints remain required.
|
|
- **full-auto** — No gates, no sandbox, no checkpoints, unsafe tools allowed. Use with extreme caution.
|
|
|
|
## Resolution Precedence
|
|
|
|
When determining which profile applies to a given plan execution, the system resolves profiles in the following order (highest priority first):
|
|
|
|
1. **Plan-level override** — A profile specified directly on the plan.
|
|
2. **Project-level setting** — The default profile configured for the project.
|
|
3. **Organization-level default** — The organization's default profile.
|
|
4. **System default** — Falls back to the `review` built-in profile.
|
|
|
|
At each level, the profile may be specified by:
|
|
- A built-in name (e.g. `cautious`)
|
|
- A namespaced custom profile (e.g. `acme/strict`)
|
|
|
|
## Custom Profiles
|
|
|
|
Custom profiles use a `namespace/name` naming convention:
|
|
|
|
```yaml
|
|
name: acme/strict
|
|
description: Strict profile for production deployments
|
|
schema_version: "1.0"
|
|
|
|
auto_strategize: 0.9
|
|
auto_execute: 0.9
|
|
auto_apply: 1.0
|
|
auto_decisions_strategize: 0.8
|
|
auto_decisions_execute: 0.9
|
|
auto_validation_fix: 0.5
|
|
auto_strategy_revision: 0.9
|
|
auto_reversion_from_apply: 1.0
|
|
auto_child_plans: 0.8
|
|
auto_retry_transient: 0.3
|
|
auto_checkpoint_restore: 0.7
|
|
|
|
require_sandbox: true
|
|
require_checkpoints: true
|
|
allow_unsafe_tools: false
|
|
```
|
|
|
|
See `docs/schema/automation_profile.schema.yaml` for the full YAML schema and `examples/profiles/` for example configurations.
|
|
|
|
## CLI Commands
|
|
|
|
The `agents automation-profile` command group manages automation profiles
|
|
from the terminal.
|
|
|
|
### `automation-profile add`
|
|
|
|
Register a custom profile from a YAML configuration file:
|
|
|
|
```bash
|
|
agents automation-profile add --config ./profiles/strict.yaml
|
|
agents automation-profile add --config ./profiles/strict.yaml --update
|
|
agents automation-profile add --config ./profiles/strict.yaml --format json
|
|
```
|
|
|
|
Options:
|
|
- `--config / -c` — Path to the YAML configuration file (required).
|
|
- `--update` — Overwrite an existing custom profile instead of raising a conflict error.
|
|
- `--format / -f` — Output format: `json`, `yaml`, `plain`, `table`, or `rich` (default: `rich`).
|
|
|
|
### `automation-profile remove`
|
|
|
|
Remove a custom profile by name (built-in profiles cannot be removed):
|
|
|
|
```bash
|
|
agents automation-profile remove acme/strict --yes
|
|
agents automation-profile remove acme/strict --yes --format json
|
|
```
|
|
|
|
Options:
|
|
- `--yes / -y` — Skip the interactive confirmation prompt.
|
|
- `--format / -f` — Output format.
|
|
|
|
### `automation-profile list`
|
|
|
|
List all profiles with optional filtering:
|
|
|
|
```bash
|
|
agents automation-profile list
|
|
agents automation-profile list --namespace acme
|
|
agents automation-profile list "caut.*"
|
|
agents automation-profile list --format json
|
|
```
|
|
|
|
Options:
|
|
- *regex* (positional) — Optional regex pattern to filter profile names.
|
|
- `--namespace / -n` — Filter by the namespace portion of namespaced names.
|
|
- `--format / -f` — Output format.
|
|
|
|
### `automation-profile show`
|
|
|
|
Display full details for a single profile:
|
|
|
|
```bash
|
|
agents automation-profile show manual
|
|
agents automation-profile show acme/strict --format yaml
|
|
```
|
|
|
|
Options:
|
|
- `--format / -f` — Output format.
|
|
|
|
## Deprecation: `--automation-level`
|
|
|
|
The `--automation-level` option on `plan use` and the `plan set-automation-level`
|
|
command are **deprecated** and will be removed in a future release.
|
|
|
|
Legacy automation levels are mapped to built-in profiles:
|
|
|
|
| Legacy level | Built-in profile |
|
|
|-------------|-----------------|
|
|
| `manual` | `manual` |
|
|
| `supervised` | `supervised` |
|
|
| `auto` | `auto` |
|
|
| `full_auto` | `full-auto` |
|
|
|
|
Use `--automation-profile <name>` instead of `--automation-level`.
|