From c8259353091f9d8638276d3689df3bd8ae720f5d Mon Sep 17 00:00:00 2001 From: Jeffrey Phillips Freeman Date: Fri, 27 Feb 2026 22:48:53 +0000 Subject: [PATCH] docs(spec): recognize SafetyProfile as composed sub-model of AutomationProfile Add SafetyProfile as a first-class concept in the specification, composed within AutomationProfile via a 'safety' field. This eliminates the dual-authority problem where both AutomationProfile and a separate SafetyProfile defined the same three safety booleans (require_sandbox, require_checkpoints, allow_unsafe_tools) with no spec-defined resolution. Changes: - specification.md: Add Safety Profile glossary entry, split Automatable Tasks into thresholds + Safety Profile sub-section, update built-in profile matrix with safety.* prefix, update YAML examples - ADR-041 (new): Document composition decision, field schema, relationship to Guards, constraints, consequences, rejected alternatives (inheritance, mixin, flat) - ADR-017: Update profile fields table, built-in profiles, constraints, risks, and cross-reference to ADR-041 - reference/automation_profiles.md: Rename Safety Fields to Safety Profile sub-section, expand built-in matrix, update YAML examples - schema/automation_profile.schema.yaml: Nest safety fields under safety object with all SafetyProfile fields - adr/index.md: Add ADR-041 to Tier 3 inventory Resolves spec gap identified in issue #332. --- docs/adr/ADR-017-automation-profiles.md | 39 +++-- docs/adr/ADR-041-safety-profile-extraction.md | 143 ++++++++++++++++++ docs/adr/index.md | 3 +- docs/reference/automation_profiles.md | 50 ++++-- docs/schema/automation_profile.schema.yaml | 56 +++++-- docs/specification.md | 61 ++++++-- 6 files changed, 299 insertions(+), 53 deletions(-) create mode 100644 docs/adr/ADR-041-safety-profile-extraction.md diff --git a/docs/adr/ADR-017-automation-profiles.md b/docs/adr/ADR-017-automation-profiles.md index 96a198b6b..acbe62105 100644 --- a/docs/adr/ADR-017-automation-profiles.md +++ b/docs/adr/ADR-017-automation-profiles.md @@ -18,7 +18,7 @@ CleverAgents uses **automation profiles** — named configuration bundles that c ### Profile Fields -Each automation profile is a set of confidence thresholds and boolean flags: +Each automation profile is a set of confidence thresholds plus a composed **safety profile** sub-model (see [ADR-041](ADR-041-safety-profile-extraction.md)): | Field | Type | Description | |-------|------|-------------| @@ -29,9 +29,20 @@ Each automation profile is a set of confidence thresholds and boolean flags: | `auto_decisions_execute` | float (0.0-1.0) | Threshold for automatic decision-making during Execute. | | `auto_strategy_revision` | float (0.0-1.0) | Threshold for automatic Execute → Strategize reversion. | | `auto_reversion_from_apply` | float (0.0-1.0) | Threshold for automatic Apply → Strategize reversion. | -| `require_sandbox` | boolean | Whether sandbox isolation is required. | -| `require_checkpoints` | boolean | Whether checkpointing is required. | -| `max_correction_attempts` | integer | Maximum auto-correction attempts before escalating to user. | +| `safety` | SafetyProfile | Composed sub-model with hard safety constraints (see below). | + +The `safety` field is a `SafetyProfile` sub-model containing: + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `require_sandbox` | boolean | `true` | Whether sandbox isolation is required. | +| `require_checkpoints` | boolean | `true` | Whether checkpointing is required. | +| `allow_unsafe_tools` | boolean | `false` | Whether tools marked as unsafe may be invoked. | +| `require_human_approval` | boolean | `false` | Whether human approval is required before each action step. | +| `allowed_skill_categories` | list[string] | `[]` | Skill categories permitted for execution (empty = all). | +| `max_cost_per_plan` | float \| null | `null` | Maximum cost in USD per plan execution. | +| `max_total_cost` | float \| null | `null` | Maximum total cost in USD across all plans. | +| `max_retries_per_step` | integer | `3` | Maximum retry attempts per action step. | ### Threshold Semantics @@ -43,14 +54,14 @@ The Semantic Escalation system produces confidence scores for each transition. W | Profile | Intent | Key Settings | |---------|--------|-------------| -| `manual` | Human controls everything | All thresholds 1.0, sandbox required, checkpoints required | +| `manual` | Human controls everything | All thresholds 1.0, `safety.require_sandbox: true`, `safety.require_checkpoints: true` | | `review` | Auto-strategize, human reviews before execute and apply | `auto_strategize: 0.0`, execute/apply thresholds high | | `supervised` | Auto through strategize/execute, human approves apply | `auto_apply: 1.0`, others low | -| `cautious` | Mostly auto with conservative thresholds | Moderate thresholds (~0.7), sandbox required | -| `trusted` | Fully auto for well-tested domains | Low thresholds (~0.3), sandbox required | -| `auto` | Fully automatic with safety nets | All thresholds 0.0, sandbox required, checkpoints required | -| `ci` | CI/CD pipeline automation | All thresholds 0.0, no sandbox required, no checkpoint requirement | -| `full-auto` | Maximum autonomy, minimal guardrails | All thresholds 0.0, no sandbox required, no checkpoints required | +| `cautious` | Mostly auto with conservative thresholds | Moderate thresholds (~0.7), `safety.require_sandbox: true` | +| `trusted` | Fully auto for well-tested domains | Low thresholds (~0.3), `safety.require_sandbox: true` | +| `auto` | Fully automatic with safety nets | All thresholds 0.0, `safety.require_sandbox: true`, `safety.require_checkpoints: true` | +| `ci` | CI/CD pipeline automation | All thresholds 0.0, `safety.require_sandbox: true`, `safety.require_checkpoints: true` | +| `full-auto` | Maximum autonomy, minimal guardrails | All thresholds 0.0, `safety.require_sandbox: false`, `safety.allow_unsafe_tools: true` | ### Profile Resolution @@ -70,8 +81,9 @@ Custom automation profiles are registered via YAML configuration (`agents automa - The effective automation profile is resolved once at `plan use` time and locked to the plan. Changing the profile after plan creation requires creating a new plan. - Threshold values must be in the range [0.0, 1.0]. -- `require_sandbox: false` is only valid if the sandbox strategy is `none`. Plans with `require_sandbox: true` must have a non-none sandbox strategy. -- `require_checkpoints: false` is only valid if `sandbox.checkpoint.enabled` is `false`. Plans with `require_checkpoints: true` reject tools with `checkpointable: false`. +- `safety.require_sandbox: false` is only valid if the sandbox strategy is `none`. Plans with `safety.require_sandbox: true` must have a non-none sandbox strategy. +- `safety.require_checkpoints: false` is only valid if `sandbox.checkpoint.enabled` is `false`. Plans with `safety.require_checkpoints: true` reject tools with `checkpointable: false`. +- Safety constraints are composed within the profile via a `SafetyProfile` sub-model (see [ADR-041](ADR-041-safety-profile-extraction.md)). The safety booleans do not appear as top-level fields on `AutomationProfile`. - The default global profile is `supervised` (configurable via `core.automation-profile`). - Built-in profile names are reserved and cannot be overridden by custom profiles. @@ -89,7 +101,7 @@ Custom automation profiles are registered via YAML configuration (`agents automa - Confidence score calibration affects the behavior of intermediate thresholds — poorly calibrated scores make thresholds unreliable. ### Risks -- The `full-auto` and `ci` profiles with `require_sandbox: false` remove key safety nets. Misuse could cause unintended resource modifications. +- The `full-auto` profile with `safety.require_sandbox: false` removes key safety nets. Misuse could cause unintended resource modifications. - Custom profiles with overly permissive settings could bypass important safety constraints. - Users may not understand the interaction between threshold values and confidence scores, leading to unexpected automation behavior. @@ -114,6 +126,7 @@ Custom automation profiles are registered via YAML configuration (`agents automa | [ADR-006](ADR-006-plan-lifecycle.md) | Plan Lifecycle | Profiles control which lifecycle phase transitions require human approval | | [ADR-007](ADR-007-decision-tree-and-correction.md) | Decision Tree and Correction | Confidence thresholds determine which decisions require human intervention | | [ADR-016](ADR-016-invariant-system.md) | Invariant System | Invariant violation escalation is governed by the active automation profile | +| [ADR-041](ADR-041-safety-profile-extraction.md) | Safety Profile Extraction | Safety constraints are extracted into a composed SafetyProfile sub-model within AutomationProfile | ## Acceptance diff --git a/docs/adr/ADR-041-safety-profile-extraction.md b/docs/adr/ADR-041-safety-profile-extraction.md new file mode 100644 index 000000000..16dbbc970 --- /dev/null +++ b/docs/adr/ADR-041-safety-profile-extraction.md @@ -0,0 +1,143 @@ +# ADR-041: Safety Profile Extraction + +**Status:** Accepted +**Date:** 2026-02-27 +**Supersedes:** None +**Author(s):** Jeffrey Phillips Freeman +**Approver(s):** Jeffrey Phillips Freeman + +## Context + +Issue [#332](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/332) introduced a standalone `SafetyProfile` domain model to capture hard safety constraints (sandbox requirements, checkpoint enforcement, unsafe-tool gating, skill category restrictions, cost/retry limits, and human-approval flags). However, the original specification placed three of these fields (`require_sandbox`, `require_checkpoints`, `allow_unsafe_tools`) directly on the `AutomationProfile` model alongside its 11 confidence thresholds. + +Implementing `SafetyProfile` as a separate, parallel model created **overlapping authority**: both `AutomationProfile` and `SafetyProfile` defined the same three boolean safety fields, with no spec-defined resolution when their values conflicted. The documentation attempted to resolve this by stating "SafetyProfile takes precedence for overlapping safety flags," but this precedence rule existed only in the new docs, not in the specification. + +This dual-model overlap was identified during implementation review (see [#332 comment](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/332#issuecomment-43934)). + +## Decision + +CleverAgents extracts all safety constraints into a **`SafetyProfile` sub-model** that is **composed** within `AutomationProfile` via a `safety` field. The three previously top-level safety booleans (`require_sandbox`, `require_checkpoints`, `allow_unsafe_tools`) move from `AutomationProfile`'s top level into the composed `SafetyProfile`, which also adds new safety-specific fields (`require_human_approval`, `allowed_skill_categories`, `max_cost_per_plan`, `max_retries_per_step`, `max_total_cost`). + +`SafetyProfile` may also be attached directly to an `Action` (via `safety_profile`) when only safety constraints — without full autonomy thresholds — are needed. + +## Design + +### SafetyProfile Model + +`SafetyProfile` is a frozen Pydantic `BaseModel` with the following fields: + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `require_sandbox` | `bool` | `True` | Whether sandbox isolation is required for execution | +| `require_checkpoints` | `bool` | `True` | Whether checkpoints are required before writes | +| `allow_unsafe_tools` | `bool` | `False` | Whether tools marked as unsafe may be invoked | +| `require_human_approval` | `bool` | `False` | Whether human approval is required before each action step | +| `allowed_skill_categories` | `list[str]` | `[]` | Skill categories permitted for execution (empty = all allowed) | +| `max_cost_per_plan` | `float \| None` | `None` | Maximum cost in USD per plan execution | +| `max_total_cost` | `float \| None` | `None` | Maximum total cost in USD across all plans | +| `max_retries_per_step` | `int` | `3` | Maximum retry attempts per action step (range: 0–100) | + +Cross-field validation enforces `max_cost_per_plan <= max_total_cost` when both are set. + +### Composition in AutomationProfile + +`AutomationProfile` gains a `safety: SafetyProfile` field. The three previously top-level booleans (`require_sandbox`, `require_checkpoints`, `allow_unsafe_tools`) are removed from `AutomationProfile`'s top level and accessed exclusively via the `safety` sub-model: + +```python +# Before (flat, overlapping): +profile.require_sandbox # on AutomationProfile +safety.require_sandbox # on separate SafetyProfile — conflict + +# After (composed, single source of truth): +profile.safety.require_sandbox # SafetyProfile is the sole authority +``` + +All eight built-in profiles compose their safety constraints via this nested structure. + +### Standalone Use on Action + +The `Action` model may carry a `safety_profile: SafetyProfile | None` field for cases where safety constraints are needed without full autonomy thresholds. When a plan is created from such an action, the safety profile is available for enforcement independently of the automation profile. + +### Resolution Precedence + +Safety profile resolution follows the same precedence as automation profiles: `plan > action > project > global`. The `SafetyProfileRef` captures provenance (which level the profile was resolved from) via the `SafetyProfileProvenance` enum (`PLAN`, `ACTION`, `PROJECT`, `GLOBAL`). + +### Relationship to Automation Guards + +`SafetyProfile.max_total_cost` sets a **plan-level** budget cap (broad scope), while `AutomationGuard.max_total_cost` sets a **per-invocation** budget cap (narrow scope). These operate at different granularities and both may be active simultaneously — the tighter constraint takes precedence at any given point. + +## Constraints + +- `SafetyProfile` is frozen (immutable) once constructed. +- `max_cost_per_plan` must not exceed `max_total_cost` when both are set. +- `max_retries_per_step` must be in the range [0, 100]. +- `allowed_skill_categories` entries are trimmed, deduplicated, and must be non-empty strings. +- All eight built-in automation profiles must compose a `SafetyProfile` via the `safety` field. +- The three safety booleans must not appear as top-level fields on `AutomationProfile` — they exist only on the composed `SafetyProfile`. + +## Consequences + +### Positive +- **Single source of truth**: Safety constraints live in one place (`SafetyProfile`), eliminating the dual-authority problem between `AutomationProfile` and a separate `SafetyProfile` model. +- **Standalone usability**: `SafetyProfile` can be used on its own (e.g., on `Action.safety_profile`) without requiring a full automation profile, enabling lightweight safety-only configurations. +- **Clean separation of concerns**: Autonomy thresholds (confidence-gated behavior) and safety constraints (binary invariants) are clearly separated within the profile structure. +- **Extensibility**: New safety constraints can be added to `SafetyProfile` without modifying `AutomationProfile`'s threshold fields. + +### Negative +- **Breaking change**: Code that previously accessed `profile.require_sandbox` must now use `profile.safety.require_sandbox`. All built-in profiles, tests, and serialization must be updated. +- **YAML schema change**: Custom profile YAML files must nest safety fields under a `safety:` key. Existing flat-format profiles require migration. +- **Slightly deeper nesting**: Accessing safety fields requires one additional level of attribute access. + +### Risks +- **Migration burden**: Existing custom profiles in user configurations will break until updated to the nested format. A migration guide or backward-compatibility shim may be needed. +- **Serialization impact**: Persistence mappings (e.g., `safety_profile_json` column on `LifecycleActionModel`) must serialize the composed structure correctly. + +## Alternatives Considered + +**Inheritance (AutomationProfile extends SafetyProfile)** — This would give `AutomationProfile` all `SafetyProfile` fields via inheritance. Rejected because it conflates the identity of the two models (an `AutomationProfile` IS-A `SafetyProfile`) and would add the new cost/retry/category fields to `AutomationProfile`'s top level, bloating it further. Composition ("has-a") is cleaner than inheritance ("is-a") here. + +**Shared mixin base class** — Extract the 3 overlapping booleans into a `SafetyConstraintsMixin` that both models inherit from. Rejected because it still results in two separate models with the same fields, just sourced from a common parent — the dual-authority problem remains. + +**Keep flat (no extraction)** — Leave the safety booleans on `AutomationProfile` and add the new fields there too. Rejected because it makes `AutomationProfile` a grab-bag of unrelated concerns and prevents standalone safety-only use on Actions. + +## Compliance + +- **Composition tests**: Tests verify that `AutomationProfile.safety` is a `SafetyProfile` instance with correct defaults. +- **Built-in profile tests**: Tests verify all eight built-in profiles correctly compose their safety constraints via the `safety` field. +- **Standalone tests**: Tests verify that `SafetyProfile` can be created, validated, and serialized independently of `AutomationProfile`. +- **Cross-field validation tests**: Tests verify that `max_cost_per_plan > max_total_cost` raises `ValueError`. +- **BDD scenarios**: Behave features exercise safety profile parsing, constraint validation, YAML loading, and plan-level resolution. +- **Persistence tests**: Tests verify correct JSON serialization/deserialization of the composed `safety` field in the database. + +## Related ADRs + +| ADR | Title | Relationship | +|-----|-------|-------------| +| [ADR-017](ADR-017-automation-profiles.md) | Automation Profiles | SafetyProfile is composed within AutomationProfile; this ADR refines ADR-017's profile structure | +| [ADR-006](ADR-006-plan-lifecycle.md) | Plan Lifecycle | Safety constraints gate lifecycle phase execution (sandbox, checkpoints) | +| [ADR-015](ADR-015-sandbox-and-checkpoint.md) | Sandbox and Checkpoint | `require_sandbox` and `require_checkpoints` flags control sandbox/checkpoint enforcement | +| [ADR-018](ADR-018-semantic-error-prevention.md) | Semantic Error Prevention | Safety profile constraints are a form of pre-flight guardrail | + +## Acceptance + +### Votes For + +| Voter | Comment | +|-------|---------| +| Jeffrey Phillips Freeman | Composition eliminates dual authority while preserving standalone safety profile use on Actions | + +**Total: 1** + +### Votes Against + +| Voter | Comment | +|-------|---------| + +**Total: 0** + +### Abstentions + +| Voter | Comment | +|-------|---------| + +**Total: 0** diff --git a/docs/adr/index.md b/docs/adr/index.md index bf45ec104..3ed2fdb57 100644 --- a/docs/adr/index.md +++ b/docs/adr/index.md @@ -99,8 +99,9 @@ These ADRs govern cross-cutting behavioral systems and infrastructure concerns. | [ADR-014](ADR-014-context-management-acms.md) | Context Management (ACMS) | Unified Knowledge Ontology, Context Retrieval Pipeline, 10-component assembly pipeline. | | [ADR-015](ADR-015-sandbox-and-checkpoint.md) | Sandbox and Checkpoint | Five isolation strategies, lazy sandboxing, checkpoint/restore per tool. | | [ADR-016](ADR-016-invariant-system.md) | Invariant System | Four scopes with precedence, reconciliation actor, and child plan inheritance. | -| [ADR-017](ADR-017-automation-profiles.md) | Automation Profiles | Confidence thresholds controlling autonomy across eight built-in profiles. | +| [ADR-017](ADR-017-automation-profiles.md) | Automation Profiles | Confidence thresholds controlling autonomy across eight built-in profiles. Composes a SafetyProfile sub-model (ADR-041). | | [ADR-018](ADR-018-semantic-error-prevention.md) | Semantic Error Prevention | Four prevention layers from decision-time validation to predictive guards. | +| [ADR-041](ADR-041-safety-profile-extraction.md) | Safety Profile Extraction | Safety constraints extracted into a composed SafetyProfile sub-model within AutomationProfile. | | [ADR-019](ADR-019-storage-and-persistence.md) | Storage and Persistence | SQLite/SQLAlchemy with repository pattern, Unit of Work, and Alembic migrations. | | [ADR-020](ADR-020-session-model.md) | Session Model | Conversation threads with orchestrator binding and session-scoped context. | diff --git a/docs/reference/automation_profiles.md b/docs/reference/automation_profiles.md index 1bf05ae8e..336b14c15 100644 --- a/docs/reference/automation_profiles.md +++ b/docs/reference/automation_profiles.md @@ -28,13 +28,24 @@ Each threshold field is a float in the range `[0.0, 1.0]`: | `auto_retry_transient` | Retry | Gate for retrying transient failures | | `auto_checkpoint_restore` | Checkpoint | Gate for automatic checkpoint restoration | -### Safety Fields +### Safety Profile (Composed Sub-Model) + +Each automation profile composes a `SafetyProfile` via its `safety` field. The safety profile is the single source of truth for all hard safety constraints. See [ADR-041](../adr/ADR-041-safety-profile-extraction.md) for the architectural rationale. + +A `SafetyProfile` may also be attached directly to an `Action` (via `safety_profile`) when only safety constraints are needed without full autonomy thresholds. | Field | Type | Default | Description | |-------|------|---------|-------------| -| `require_sandbox` | bool | `true` | Execution must happen in a sandbox | -| `require_checkpoints` | bool | `true` | Checkpoints must be created before writes | -| `allow_unsafe_tools` | bool | `false` | Tools flagged as `unsafe` may be invoked | +| `safety.require_sandbox` | bool | `true` | Execution must happen in a sandbox | +| `safety.require_checkpoints` | bool | `true` | Checkpoints must be created before writes | +| `safety.allow_unsafe_tools` | bool | `false` | Tools flagged as `unsafe` may be invoked | +| `safety.require_human_approval` | bool | `false` | Human approval required before each action step | +| `safety.allowed_skill_categories` | list[str] | `[]` | Skill categories permitted (empty = all allowed) | +| `safety.max_cost_per_plan` | float \| null | `null` | Maximum cost in USD per plan (null = no limit) | +| `safety.max_total_cost` | float \| null | `null` | Maximum total cost in USD across all plans (null = no limit) | +| `safety.max_retries_per_step` | int | `3` | Maximum retry attempts per action step (0–100) | + +**Relationship to Automation Guards**: `safety.max_total_cost` is a plan-level budget cap, while `guards.max_total_cost` is a per-invocation budget cap. Both may be active; the tighter constraint wins. ## Built-in Profiles @@ -53,9 +64,15 @@ Eight profiles ship with every CleverAgents installation: | auto_child_plans | 1.0 | 0.0 | 1.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 | | auto_retry_transient | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | auto_checkpoint_restore | 1.0 | 1.0 | 1.0 | 0.6 | 1.0 | 0.0 | 0.0 | 0.0 | -| require_sandbox | true | true | true | true | true | true | true | false | -| require_checkpoints | true | true | true | true | true | true | true | false | -| allow_unsafe_tools | false | false | false | false | false | false | false | true | +| **Safety Profile** | | | | | | | | | +| safety.require_sandbox | true | true | true | true | true | true | true | false | +| safety.require_checkpoints | true | true | true | true | true | true | true | false | +| safety.allow_unsafe_tools | false | false | false | false | false | false | false | true | +| safety.require_human_approval | false | false | false | false | false | false | false | false | +| safety.allowed_skill_categories | [] | [] | [] | [] | [] | [] | [] | [] | +| safety.max_cost_per_plan | null | null | null | null | null | null | null | null | +| safety.max_total_cost | null | null | null | null | null | null | null | null | +| safety.max_retries_per_step | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | ### Profile Descriptions @@ -102,9 +119,15 @@ auto_child_plans: 0.8 auto_retry_transient: 0.3 auto_checkpoint_restore: 0.7 -require_sandbox: true -require_checkpoints: true -allow_unsafe_tools: false +# Safety profile (composed sub-model) +safety: + require_sandbox: true + require_checkpoints: true + allow_unsafe_tools: false + allowed_skill_categories: [] + max_cost_per_plan: 50.0 + max_total_cost: 200.0 + max_retries_per_step: 3 ``` See `docs/schema/automation_profile.schema.yaml` for the full YAML schema and `examples/profiles/` for example configurations. @@ -155,6 +178,13 @@ auto_strategize: 0.7 auto_execute: 0.7 auto_apply: 1.0 +# Safety profile (plan-level constraints) +safety: + require_sandbox: true + require_checkpoints: true + allow_unsafe_tools: false + +# Guards (per-invocation constraints) guards: max_tool_calls_per_step: 10 max_total_cost: 5.0 diff --git a/docs/schema/automation_profile.schema.yaml b/docs/schema/automation_profile.schema.yaml index ca0e3d4ef..47213282f 100644 --- a/docs/schema/automation_profile.schema.yaml +++ b/docs/schema/automation_profile.schema.yaml @@ -95,16 +95,46 @@ properties: default: 0.0 description: "Threshold for automatic checkpoint restore" - # Safety requirements - require_sandbox: - type: boolean - default: true - description: "Whether a sandbox is required for execution" - require_checkpoints: - type: boolean - default: true - description: "Whether checkpoints are required" - allow_unsafe_tools: - type: boolean - default: false - description: "Whether unsafe tools may be used" + # Safety profile (composed sub-model) + safety: + type: object + description: "Composed SafetyProfile sub-model with hard safety constraints" + properties: + require_sandbox: + type: boolean + default: true + description: "Whether a sandbox is required for execution" + require_checkpoints: + type: boolean + default: true + description: "Whether checkpoints are required" + allow_unsafe_tools: + type: boolean + default: false + description: "Whether unsafe tools may be used" + require_human_approval: + type: boolean + default: false + description: "Whether human approval is required before each action step" + allowed_skill_categories: + type: array + items: + type: string + default: [] + description: "Skill categories permitted for execution (empty = all allowed)" + max_cost_per_plan: + type: ["number", "null"] + minimum: 0.0 + default: null + description: "Maximum cost in USD per plan execution (null = no limit)" + max_total_cost: + type: ["number", "null"] + minimum: 0.0 + default: null + description: "Maximum total cost in USD across all plans (null = no limit)" + max_retries_per_step: + type: integer + minimum: 0 + maximum: 100 + default: 3 + description: "Maximum retry attempts per action step" diff --git a/docs/specification.md b/docs/specification.md index 6d04779e9..a23ebdc32 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -92,7 +92,10 @@ The following standards are integrated into the architecture: : A natural-language constraint on plan execution scoped to global, project, action, or plan level. Action invariants are promoted to plan-level when the action is used, so the runtime precedence chain is three-tier: ==plan > project > global==. Reconciled by the Invariant Reconciliation Actor at the start of Strategize; recorded as `invariant_enforced` decisions that propagate to child plans. Automation Profile - : A named set of confidence thresholds (each `0.0`–`1.0`) gating which plan operations proceed automatically versus requiring human approval. `0.0` = always automatic; `1.0` = always manual. Eight built-in profiles (`manual` through `full-auto`). Custom profiles namespaced as `[[server:]namespace/]name`. + : A named set of confidence thresholds (each `0.0`–`1.0`) gating which plan operations proceed automatically versus requiring human approval. `0.0` = always automatic; `1.0` = always manual. Eight built-in profiles (`manual` through `full-auto`). Custom profiles namespaced as `[[server:]namespace/]name`. Each profile composes a **Safety Profile** that controls hard safety constraints (sandbox, checkpoint, unsafe-tool gating, skill restrictions, cost/retry limits). + + Safety Profile + : A composed sub-model of an Automation Profile that groups all hard safety constraints: `require_sandbox`, `require_checkpoints`, `allow_unsafe_tools`, `require_human_approval`, `allowed_skill_categories`, `max_cost_per_plan`, `max_retries_per_step`, and `max_total_cost`. Safety profiles can also be referenced standalone on Actions when only safety constraints (without full autonomy thresholds) are needed. See [ADR-041](adr/ADR-041-safety-profile-extraction.md). ???+ abstract "Projects & Resources" @@ -27686,8 +27689,6 @@ An **automation profile** is a named collection of **confidence thresholds** — Each automation profile specifies a **confidence threshold** (a floating-point value from 0.0 to 1.0 inclusive) for each of the following automatable tasks. The threshold determines the minimum confidence level at which the system proceeds automatically. When the computed confidence (from Semantic Escalation) falls below the threshold, the system drops to manual mode for that task. Setting a threshold to **0.0** makes the task always automatic; setting it to **1.0** makes it always manual. -The three safety flags (`require_sandbox`, `require_checkpoints`, `allow_unsafe_tools`) remain boolean since they are binary constraints rather than confidence-dependent behaviors. - | Flag | Type | Description | Behavior | |------|------|-------------|----------| | `auto_strategize` | float (0.0–1.0) | Automatically enter Strategize after `plan use` | When confidence >= threshold, Strategize begins immediately. Below threshold, system pauses after plan creation for user confirmation before entering Strategize. | @@ -27701,9 +27702,28 @@ The three safety flags (`require_sandbox`, `require_checkpoints`, `allow_unsafe_ | `auto_child_plans` | float (0.0–1.0) | Automatically spawn and execute child plans | When confidence >= threshold, child plans are created and executed without pausing. Below threshold, each spawn requires approval. | | `auto_retry_transient` | float (0.0–1.0) | Automatically retry on transient failures (network, timeout, rate-limit) | When confidence >= threshold, system retries with backoff. Below threshold, system pauses and asks user. | | `auto_checkpoint_restore` | float (0.0–1.0) | Automatically restore from checkpoint on failure | When confidence >= threshold, system rolls back and retries. Below threshold, user decides whether to restore. | -| `require_sandbox` | boolean | Require sandbox isolation for Execute phase | When `true`, Execute must run in a sandbox. When `false`, sandbox is optional. | -| `require_checkpoints` | boolean | Require checkpointing during Execute | When `true`, tools must create checkpoints before writes. When `false`, checkpointing is optional. | -| `allow_unsafe_tools` | boolean | Allow execution of tools marked as unsafe | When `true`, unsafe tools can be invoked. When `false`, unsafe tools are blocked. | + +#### Safety Profile (Composed Sub-Model) + +!!! adr "Architecture Decision" + The extraction of safety constraints into a composed SafetyProfile sub-model is defined in [ADR-041: Safety Profile Extraction](adr/ADR-041-safety-profile-extraction.md). + +Each automation profile composes a **`safety`** sub-model (`SafetyProfile`) that groups all hard safety constraints. These are binary constraints rather than confidence-dependent behaviors — they enforce invariant safety guarantees regardless of the system's confidence level. The `SafetyProfile` is the **single source of truth** for safety constraints; the `AutomationProfile` accesses them via its `safety` field. + +A `SafetyProfile` may also be attached directly to an `Action` (via the `safety_profile` field) when only safety constraints are needed without full autonomy thresholds. + +| Flag | Type | Default | Description | Behavior | +|------|------|---------|-------------|----------| +| `require_sandbox` | boolean | `true` | Require sandbox isolation for Execute phase | When `true`, Execute must run in a sandbox. When `false`, sandbox is optional. | +| `require_checkpoints` | boolean | `true` | Require checkpointing during Execute | When `true`, tools must create checkpoints before writes. When `false`, checkpointing is optional. | +| `allow_unsafe_tools` | boolean | `false` | Allow execution of tools marked as unsafe | When `true`, unsafe tools can be invoked. When `false`, unsafe tools are blocked. | +| `require_human_approval` | boolean | `false` | Require human approval before each action step | When `true`, every action step pauses for explicit human approval before execution. | +| `allowed_skill_categories` | list[string] | `[]` (all) | Skill categories permitted for execution | When non-empty, only skills in the listed categories may be used. Empty list means all categories are allowed. | +| `max_cost_per_plan` | float \| null | `null` | Maximum cost in USD per plan execution | When set, the plan is paused or terminated if the cost limit is reached. `null` means no limit. Must be <= `max_total_cost` when both are set. | +| `max_total_cost` | float \| null | `null` | Maximum total cost in USD across all plans | When set, execution is paused or terminated if the aggregate cost limit is reached. `null` means no limit. | +| `max_retries_per_step` | integer | `3` | Maximum retry attempts per action step | Limits the number of retries for a single step before escalating to the user. Range: 0–100. | + +**Relationship to Automation Guards**: The `SafetyProfile.max_total_cost` field sets a **plan-level** budget cap (broad scope), while `AutomationGuard.max_total_cost` sets a **per-invocation** budget cap (narrow scope). These operate at different granularities and both may be active simultaneously — the tighter constraint takes precedence at any given point. #### Built-in Automation Profiles @@ -27724,9 +27744,15 @@ Threshold values are shown for each flag. A value of **0.0** means always automa | `auto_child_plans` | 1.0 | 0.0 | 1.0 | 0.7 | 0.0 | 0.0 | 0.0 | 0.0 | | `auto_retry_transient` | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | `auto_checkpoint_restore` | 1.0 | 1.0 | 1.0 | 0.6 | 1.0 | 0.0 | 0.0 | 0.0 | -| `require_sandbox` | true | true | true | true | true | true | true | false | -| `require_checkpoints` | true | true | true | true | true | true | true | false | -| `allow_unsafe_tools` | false | false | false | false | false | false | false | true | +| **Safety Profile** | | | | | | | | | +| `safety.require_sandbox` | true | true | true | true | true | true | true | false | +| `safety.require_checkpoints` | true | true | true | true | true | true | true | false | +| `safety.allow_unsafe_tools` | false | false | false | false | false | false | false | true | +| `safety.require_human_approval` | false | false | false | false | false | false | false | false | +| `safety.allowed_skill_categories` | [] | [] | [] | [] | [] | [] | [] | [] | +| `safety.max_cost_per_plan` | null | null | null | null | null | null | null | null | +| `safety.max_total_cost` | null | null | null | null | null | null | null | null | +| `safety.max_retries_per_step` | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | **`manual`**: Maximum human control. All thresholds set to 1.0 — every phase transition, every decision, every child plan requires explicit human approval regardless of confidence. Sandbox and checkpoints are mandatory. Unsafe tools are blocked. This is the default starting point. Use for: new users learning the system, critical production systems, first-time exploration of an unfamiliar codebase, sensitive projects, regulatory environments. @@ -27781,10 +27807,11 @@ Custom profiles are created via YAML configuration files and registered with `ag auto_retry_transient: 0.0 auto_checkpoint_restore: 0.0 -# Safety flags (boolean) -require_sandbox: true -require_checkpoints: true -allow_unsafe_tools: false +# Safety profile (composed sub-model) +safety: + require_sandbox: true + require_checkpoints: true + allow_unsafe_tools: false A profile with intermediate thresholds provides nuanced control. For example, `auto_decisions_execute: 0.7` means decisions during Execute proceed automatically when confidence >= 0.7, but drop to manual review when confidence is below 0.7: @@ -27806,9 +27833,11 @@ A profile with intermediate thresholds provides nuanced control. For example, `a auto_retry_transient: 0.0 # Always auto-retry transient errors auto_checkpoint_restore: 0.6 # Auto-restore when fairly confident -require_sandbox: true -require_checkpoints: true -allow_unsafe_tools: false +# Safety profile (composed sub-model) +safety: + require_sandbox: true + require_checkpoints: true + allow_unsafe_tools: false