cleveragents-core/docs/reference/safety_profile.md

Safety Profile

Overview

A Safety Profile defines constraints on plan execution to enforce security and cost boundaries. It controls which skill categories are permitted, whether sandboxes and checkpoints are required, whether unsafe tools may be invoked, whether human approval is needed, and sets cost and retry limits.

When an Action carries both a safety_profile and an automation_profile, the SafetyProfile takes precedence for overlapping safety flags (require_sandbox, require_checkpoints, allow_unsafe_tools). The AutomationProfile controls autonomy thresholds (confidence-based gating) while the SafetyProfile enforces hard safety constraints.

Schema

Field	Type	Default	Description
allowed_skill_categories	list[str]	[]	Permitted skill categories (empty = all)
require_sandbox	bool	True	Require sandbox for execution
require_checkpoints	bool	True	Require checkpoints during execution
require_human_approval	bool	False	Require human approval per action step
allow_unsafe_tools	bool	False	Allow tools marked as unsafe to be invoked
max_cost_per_plan	float | None	None	Max cost in USD per plan (None = unlimited)
max_retries_per_step	int	3	Max retry attempts per step (0–100)
max_total_cost	float | None	None	Max total cost in USD (None = unlimited)

Default Profile

The DEFAULT_SAFETY_PROFILE constant provides sensible defaults:

DEFAULT_SAFETY_PROFILE = SafetyProfile(
    allowed_skill_categories=[],    # all categories allowed
    require_sandbox=True,
    require_checkpoints=True,
    require_human_approval=False,
    allow_unsafe_tools=False,
    max_cost_per_plan=None,         # no limit
    max_retries_per_step=3,
    max_total_cost=None,            # no limit
)

The default profile is immutable (frozen=True). Attempting to modify fields after construction raises a ValidationError.

Cross-Field Validation

The following cross-field constraints are enforced:

• max_cost_per_plan <= max_total_cost: When both cost limits are set, the per-plan cost must not exceed the total cost. Violating this raises a ValidationError.

Input Validation

• allowed_skill_categories: duplicates removed, blanks stripped. Non-list input raises ValueError. Non-string items in the list raise ValueError.
• max_cost_per_plan: must be >= 0.0 if set
• max_total_cost: must be >= 0.0 if set
• max_retries_per_step: must be 0–100

Precedence over AutomationProfile

When an Action carries both a SafetyProfile and an AutomationProfile, the overlapping safety flags are resolved as follows:

Flag	Wins	Rationale
require_sandbox	SafetyProfile	Hard safety constraint
require_checkpoints	SafetyProfile	Hard safety constraint
allow_unsafe_tools	SafetyProfile	Hard safety constraint

Non-overlapping fields from each profile apply independently. The AutomationProfile governs autonomy thresholds (e.g., confidence-based gating for auto-execution), while the SafetyProfile enforces hard safety boundaries.

YAML Configuration

Safety profiles can be loaded from YAML:

require_sandbox: false
require_checkpoints: true
require_human_approval: true
allow_unsafe_tools: false
max_cost_per_plan: 50.0
max_retries_per_step: 5
max_total_cost: 200.0
allowed_skill_categories:
  - code
  - test
  - deploy

Load with:

from cleveragents.domain.models.core.safety_profile import SafetyProfile

profile = SafetyProfile.from_yaml("path/to/safety.yaml")
# or from a dict:
profile = SafetyProfile.from_config(config_dict)

YAML Error Handling

• File not found: from_yaml() raises FileNotFoundError
• Non-dict content: from_yaml() raises ValueError if the YAML root is not a mapping

Action Integration

Actions can include an optional safety_profile field:

name: local/secure-deploy
description: Deploy with safety constraints
definition_of_done: Deployment verified
strategy_actor: local/default
execution_actor: local/default
safety_profile:
  require_sandbox: true
  require_checkpoints: true
  require_human_approval: true
  allow_unsafe_tools: false
  max_cost_per_plan: 100.0
  allowed_skill_categories:
    - deploy
    - test

Resolution Precedence

Safety profiles are resolved at plan-use time following:

plan > action > project > global

The resolved profile is captured in a SafetyProfileRef with provenance:

Provenance	Description
PLAN	Profile set directly on the plan
ACTION	Inherited from the action template
PROJECT	Inherited from the project
GLOBAL	Global default profile

Stub Enforcement (Local Mode)

In the current local-mode implementation, resolve_safety_profile() raises NotImplementedError. This signals that real enforcement is deferred to a future server-mode release. The resolution logic is:

from cleveragents.domain.models.core.safety_profile import resolve_safety_profile

try:
    profile, provenance = resolve_safety_profile(
        plan_profile=plan_sp,
        action_profile=action_sp,
        project_profile=project_sp,
        global_profile=global_sp,
    )
except NotImplementedError:
    # Expected in local mode — enforcement not yet available
    pass

Persistence

Safety profiles are stored as JSON in the safety_profile_json column on the actions table. Serialization uses model_dump_json() and deserialization uses SafetyProfile.model_validate().

The column is added by migration c4_001_safety_profile_column.

Specification Reference

Based on docs/specification.md sections "Automation Profiles" and "Guardrails".