cleveragents-core/docs/adr/ADR-017-automation-profiles.md

adr_number, title, status_history, tier, authors, superseded_by, related_adrs, acceptance

adr_number	title	status_history	tier	authors	superseded_by	related_adrs	acceptance
17	Automation Profiles	2026-02-16 Proposed Jeffrey Phillips Freeman 2026-02-16 Accepted Jeffrey Phillips Freeman	3	Jeffrey Phillips Freeman	null	number title relationship 6 Plan Lifecycle Profiles control which lifecycle phase transitions require human approval number title relationship 7 Decision Tree and Correction Confidence thresholds determine which decisions require human intervention number title relationship 16 Invariant System Invariant violation escalation is governed by the active automation profile number title relationship 41 Safety Profile Extraction Safety constraints are extracted into a composed SafetyProfile sub-model within AutomationProfile	votes_for votes_against abstentions voter comment Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> Named profiles with confidence thresholds provide a clean spectrum from full manual to full autonomous control

Context

Different tasks require different levels of autonomy. A trusted refactoring in a well-tested codebase can run fully automatically, while a production infrastructure change requires human approval at every step. The system needs a configurable mechanism that controls which phase transitions happen automatically, which decisions require human input, and what safety constraints (sandboxing, checkpoints, approval gates) are enforced — without requiring code changes.

Decision Drivers

• Different tasks require fundamentally different autonomy levels (trusted refactoring vs. production infrastructure changes)
• Phase transitions, decision-making, and safety constraints must be configurable without code changes
• The system needs a spectrum from fully manual to fully autonomous, not just a binary auto/manual toggle
• Confidence thresholds should gate transitions so high-confidence operations auto-proceed while uncertain ones prompt the user
• The effective profile must be locked at plan creation to prevent mid-execution behavioral changes
• Common use cases (CI pipelines, supervised mode, manual review) should have sensible built-in presets

Decision

CleverAgents uses automation profiles — named configuration bundles that control autonomy thresholds across the plan lifecycle. Eight built-in profiles span the spectrum from fully manual to fully autonomous. Custom profiles can be created. The effective profile for a plan is resolved via a precedence chain (plan > action > project > global) and is locked to the plan at plan use time.

Design

Profile Fields

Each automation profile is a set of confidence thresholds plus a composed safety profile sub-model (see ADR-041):

Field	Type	Description (spec § Automatable Tasks)
decompose_task	float (0.0-1.0)	Automatically enter Strategize after plan use. 0.0 = always auto, 1.0 = always manual.
create_tool	float (0.0-1.0)	Automatically proceed from Strategize to Execute.
select_tool	float (0.0-1.0)	Automatically proceed from Execute to Apply.
edit_code	float (0.0-1.0)	Automatically make decisions during Strategize.
execute_command	float (0.0-1.0)	Automatically make decisions during Execute.
create_file	float (0.0-1.0)	Automatically attempt to fix validation failures.
delete_content	float (0.0-1.0)	Automatically revise strategy when Execute hits constraints.
access_network	float (0.0-1.0)	Automatically revert from Apply (constrained) to Strategize.
install_dependency	float (0.0-1.0)	Automatically spawn and execute child plans.
modify_config	float (0.0-1.0)	Automatically retry on transient failures.
approve_plan	float (0.0-1.0)	Automatically restore from checkpoint on failure.
safety	SafetyProfile	Composed sub-model with hard safety constraints (see below).

The safety field is a SafetyProfile sub-model containing:

Field	Type	Default	Description
require_sandbox	boolean	true	Whether sandbox isolation is required.
require_checkpoints	boolean	true	Whether checkpointing is required.
allow_unsafe_tools	boolean	false	Whether tools marked as unsafe may be invoked.
require_human_approval	boolean	false	Whether human approval is required before each action step.
allowed_skill_categories	list[string]	[]	Skill categories permitted for execution (empty = all).
max_cost_per_plan	float | null	null	Maximum cost in USD per plan execution.
max_total_cost	float | null	null	Maximum total cost in USD across all plans.
max_retries_per_step	integer	3	Maximum retry attempts per action step.

Threshold Semantics

A threshold of 0.0 means "always proceed automatically." A threshold of 1.0 means "always require human approval." Intermediate values (e.g., 0.7) mean "auto-proceed only when the system's confidence score meets or exceeds the threshold; otherwise prompt the user."

The Semantic Escalation system produces confidence scores for each transition. When the score is below the profile's threshold, the system pauses and prompts the user.

Eight Built-In Profiles

Profile	Intent	Key Settings
manual	Human controls everything	All thresholds 1.0, safety.require_sandbox: true, safety.require_checkpoints: true
review	Auto-strategize, human reviews before execute and apply	decompose_task: 0.0, execute/apply thresholds high
supervised	Auto through strategize/execute, human approves apply	select_tool: 1.0, others low
cautious	Mostly auto with conservative thresholds	Moderate thresholds (~0.7), safety.require_sandbox: true
trusted	Fully auto for well-tested domains	Low thresholds (~0.3), safety.require_sandbox: true
auto	Fully automatic with safety nets	All thresholds 0.0, safety.require_sandbox: true, safety.require_checkpoints: true
ci	CI/CD pipeline automation	All thresholds 0.0, safety.require_sandbox: true, safety.require_checkpoints: true
full-auto	Maximum autonomy, minimal guardrails	All thresholds 0.0, safety.require_sandbox: false, safety.allow_unsafe_tools: true

Profile Resolution

The effective profile for a plan is resolved at plan use time using precedence:

plan (--automation-profile flag) > action (automation_profile field) > project (core.automation-profile) > global (core.automation-profile)

Once resolved, the profile is locked to the plan and does not change if the upstream configuration changes.

Custom Profiles

Custom automation profiles are registered via YAML configuration (agents automation-profile add --config <FILE>) with the namespace system. Custom profiles specify all the same fields as built-in profiles and can use any threshold values.

Constraints

• The effective automation profile is resolved once at plan use time and locked to the plan. Changing the profile after plan creation requires creating a new plan.
• Threshold values must be in the range [0.0, 1.0].
• safety.require_sandbox: false is only valid if the sandbox strategy is none. Plans with safety.require_sandbox: true must have a non-none sandbox strategy.
• safety.require_checkpoints: false is only valid if sandbox.checkpoint.enabled is false. Plans with safety.require_checkpoints: true reject tools with checkpointable: false.
• Safety constraints are composed within the profile via a SafetyProfile sub-model (see ADR-041). The safety booleans do not appear as top-level fields on AutomationProfile.
• The default global profile is supervised (configurable via core.automation-profile).
• Built-in profile names are reserved and cannot be overridden by custom profiles.

Consequences

Positive

• A single mechanism controls all autonomy aspects of the plan lifecycle, avoiding ad-hoc per-feature autonomy flags.
• Eight built-in profiles cover the common use cases without requiring custom configuration.
• Profile locking at plan creation prevents mid-execution changes that could invalidate assumptions.
• The precedence chain enables sensible defaults (e.g., project-level profile) with per-plan overrides when needed.

Negative

• The number of configurable fields (10+) makes profiles complex to understand and tune.
• Profile locking means that if a profile is too restrictive mid-execution, the plan must be recreated with a different profile.
• Confidence score calibration affects the behavior of intermediate thresholds — poorly calibrated scores make thresholds unreliable.

Risks

• The full-auto profile with safety.require_sandbox: false removes key safety nets. Misuse could cause unintended resource modifications.
• Custom profiles with overly permissive settings could bypass important safety constraints.
• Users may not understand the interaction between threshold values and confidence scores, leading to unexpected automation behavior.

Alternatives Considered

Per-transition boolean flags (auto/manual) — Simpler but lacks nuance. The confidence threshold approach enables the system to auto-proceed for high-confidence operations while prompting for uncertain ones.

No built-in profiles (always custom) — Increases barrier to entry. Built-in profiles provide sensible defaults that cover common use cases.

Compliance

• Profile resolution tests: Tests verify the full precedence chain (plan > action > project > global) and profile locking behavior.
• Threshold enforcement tests: Tests verify that each threshold correctly gates or permits its corresponding transition for various confidence scores.
• Safety constraint tests: Tests verify that require_sandbox and require_checkpoints correctly reject non-compliant configurations and tools.
• Built-in profile tests: Tests verify that all eight built-in profiles produce the expected behavior for standard plan lifecycles.
• BDD scenarios: Behave features exercise plans under different profiles including transitions between manual approval, auto-proceed, and confidence-gated decisions.