Luis Mendes
b4b96d213c
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 29s
CI / security (pull_request) Successful in 34s
CI / typecheck (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 2m19s
CI / docker (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 3m1s
CI / coverage (pull_request) Successful in 7m20s
CI / lint (push) Successful in 13s
CI / quality (push) Successful in 16s
CI / build (push) Successful in 18s
CI / security (push) Successful in 31s
CI / typecheck (push) Successful in 35s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m10s
CI / docker (push) Successful in 39s
CI / integration_tests (push) Successful in 3m4s
CI / coverage (push) Successful in 4m45s
CI / benchmark-publish (push) Successful in 14m51s
CI / benchmark-regression (pull_request) Successful in 26m49s
Implement safety profile resolution and enforcement in the tool
execution pipeline, replacing the NotImplementedError stub with
working precedence logic and runtime safety checks.
Core changes:
- resolve_safety_profile() now resolves plan > action > project >
global precedence, returning the highest-priority non-None profile
(or DEFAULT_SAFETY_PROFILE with GLOBAL provenance when all None)
- ToolExecutionContext gains an optional safety_profile field
- ToolRuntime._enforce_capabilities() extended with three new checks:
* Unsafe tool gating: blocks tools with unsafe=True when profile
has allow_unsafe_tools=False (ToolSafetyViolationError)
* Skill category allow-list: blocks tools whose skill category
is not in allowed_skill_categories (ToolSafetyViolationError)
* Checkpoint requirement: OR-combines ctx.require_checkpoints
with safety_profile.require_checkpoints
- New ToolSafetyViolationError in tool error hierarchy
Test coverage:
- 30 updated BDD scenarios in safety_profile.feature (resolve
precedence replaces NotImplementedError stub test)
- 24 new BDD scenarios in safety_profile_enforcement.feature
- 9 Robot Framework integration smoke tests
- 4 ASV benchmark suites (construction, serialization, resolution,
provenance enum)
All nox sessions pass (typecheck 0 errors, unit_tests 7735 scenarios
0 failures, coverage 97%, integration_tests 9/9 passed, benchmarks
complete).
ISSUES CLOSED: #345
186 lines
8.7 KiB
Gherkin
186 lines
8.7 KiB
Gherkin
Feature: Safety Profile Enforcement
|
|
As a developer
|
|
I want the tool runtime to enforce safety profile constraints
|
|
So that unsafe tools and disallowed skill categories are blocked
|
|
|
|
Background:
|
|
Given a registered tool "test/writer" that writes and is safe
|
|
And a registered tool "test/unsafe-tool" that is unsafe
|
|
And a registered tool "test/reader" that is read-only and safe
|
|
|
|
# ---- Unsafe tool gating ----
|
|
|
|
Scenario: Unsafe tool is blocked when safety profile forbids unsafe tools
|
|
Given a safety profile with allow_unsafe_tools false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/unsafe-tool"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "unsafe"
|
|
|
|
Scenario: Unsafe tool is allowed when safety profile permits unsafe tools
|
|
Given a safety profile with allow_unsafe_tools true
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/unsafe-tool"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Safe tool is allowed regardless of allow_unsafe_tools setting
|
|
Given a safety profile with allow_unsafe_tools false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Skill category enforcement ----
|
|
|
|
Scenario: Tool blocked when skill category not in allow-list
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "deploy"
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "deploy"
|
|
And the safety violation error should mention "not in the allowed"
|
|
|
|
Scenario: Tool allowed when skill category is in allow-list
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "code"
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: All categories allowed when allow-list is empty
|
|
Given a safety profile with empty allowed_skill_categories
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "anything"
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Checkpoint requirement from safety profile ----
|
|
|
|
Scenario: Non-checkpointable tool blocked when safety profile requires checkpoints
|
|
Given a safety profile with require_checkpoints true
|
|
And a tool execution context with the safety profile and require_checkpoints false
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolCheckpointRequiredError should be raised
|
|
|
|
Scenario: Non-checkpointable tool allowed when safety profile does not require checkpoints
|
|
Given a safety profile with require_checkpoints false
|
|
And a tool execution context with the safety profile and require_checkpoints false
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Context without safety profile (backward compatibility) ----
|
|
|
|
Scenario: Unsafe tool allowed when no safety profile on context
|
|
Given a tool execution context without a safety profile
|
|
When I enforce safety and run tool "test/unsafe-tool"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Enforcement still applies read-only checks without safety profile
|
|
Given a tool execution context that is read-only without a safety profile
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolAccessDeniedError should be raised
|
|
|
|
# ---- Sandbox requirement ----
|
|
|
|
Scenario: Writing tool blocked when sandbox required but no sandbox_id
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a ToolSandboxRequiredError should be raised
|
|
And the sandbox error should mention "require_sandbox"
|
|
|
|
Scenario: Writing tool allowed when sandbox required and sandbox_id set
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and sandbox_id "sandbox-001"
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Writing tool allowed when sandbox not required and no sandbox_id
|
|
Given a safety profile with require_sandbox false
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Read-only tool allowed when sandbox required but no sandbox_id
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Human approval requirement ----
|
|
|
|
Scenario: Tool blocked when human approval required but not granted
|
|
Given a safety profile with require_human_approval true
|
|
And a tool execution context with the safety profile and no approval
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolHumanApprovalRequiredError should be raised
|
|
And the approval error should mention "human approval"
|
|
|
|
Scenario: Tool allowed when human approval required and granted
|
|
Given a safety profile with require_human_approval true
|
|
And a tool execution context with the safety profile and human approval granted
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Tool allowed when human approval not required
|
|
Given a safety profile with require_human_approval false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Cost limit enforcement ----
|
|
|
|
Scenario: Tool blocked when accumulated cost exceeds max_cost_per_plan
|
|
Given a safety profile with max_cost_per_plan 10.0
|
|
And a tool execution context with the safety profile and accumulated_cost 10.0
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolCostLimitExceededError should be raised
|
|
And the cost error should mention "max_cost_per_plan"
|
|
|
|
Scenario: Tool allowed when accumulated cost is below max_cost_per_plan
|
|
Given a safety profile with max_cost_per_plan 10.0
|
|
And a tool execution context with the safety profile and accumulated_cost 5.0
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Tool blocked when total cost exceeds max_total_cost
|
|
Given a safety profile with max_total_cost 100.0
|
|
And a tool execution context with the safety profile and total_accumulated_cost 100.0
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolCostLimitExceededError should be raised
|
|
And the cost error should mention "max_total_cost"
|
|
|
|
# ---- Retry limit enforcement ----
|
|
|
|
Scenario: Tool blocked when retry count exceeds max_retries_per_step
|
|
Given a safety profile with max_retries_per_step 3
|
|
And a tool execution context with the safety profile and step_retry_count 4
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolRetryLimitExceededError should be raised
|
|
And the retry error should mention "max_retries_per_step"
|
|
|
|
Scenario: Tool allowed when retry count is within max_retries_per_step
|
|
Given a safety profile with max_retries_per_step 3
|
|
And a tool execution context with the safety profile and step_retry_count 2
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Missing skill category metadata ----
|
|
|
|
Scenario: Tool blocked when skill category metadata missing and allow-list set
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile and no skill category metadata
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "no skill category"
|
|
|
|
# ---- Combined constraints ----
|
|
|
|
Scenario: Multiple safety violations reported for first failing check
|
|
Given a combined safety profile blocking unsafe tools with categories "code"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "deploy"
|
|
When I enforce safety and try to run tool "test/unsafe-tool"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "unsafe"
|