249 lines
12 KiB
Gherkin
249 lines
12 KiB
Gherkin
Feature: Safety Profile Enforcement
|
|
As a developer
|
|
I want the tool runtime to enforce safety profile constraints
|
|
So that unsafe tools and disallowed skill categories are blocked
|
|
|
|
Background:
|
|
Given a registered tool "test/writer" that writes and is safe
|
|
And a registered tool "test/unsafe-tool" that is unsafe
|
|
And a registered tool "test/reader" that is read-only and safe
|
|
|
|
# ---- Unsafe tool gating ----
|
|
|
|
Scenario: Unsafe tool is blocked when safety profile forbids unsafe tools
|
|
Given a safety profile with allow_unsafe_tools false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/unsafe-tool"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "unsafe"
|
|
|
|
Scenario: Unsafe tool is allowed when safety profile permits unsafe tools
|
|
Given a safety profile with allow_unsafe_tools true
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/unsafe-tool"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Safe tool is allowed regardless of allow_unsafe_tools setting
|
|
Given a safety profile with allow_unsafe_tools false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Skill category enforcement ----
|
|
|
|
Scenario: Tool blocked when skill category not in allow-list
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "deploy"
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "deploy"
|
|
And the safety violation error should mention "not in the allowed"
|
|
|
|
Scenario: Tool allowed when skill category is in allow-list
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "code"
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: All categories allowed when allow-list is empty
|
|
Given a safety profile with empty allowed_skill_categories
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "anything"
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Checkpoint requirement from safety profile ----
|
|
|
|
Scenario: Non-checkpointable tool blocked when safety profile requires checkpoints
|
|
Given a safety profile with require_checkpoints true
|
|
And a tool execution context with the safety profile and require_checkpoints false
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolCheckpointRequiredError should be raised
|
|
|
|
Scenario: Non-checkpointable tool allowed when safety profile does not require checkpoints
|
|
Given a safety profile with require_checkpoints false
|
|
And a tool execution context with the safety profile and require_checkpoints false
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Context without safety profile (backward compatibility) ----
|
|
|
|
Scenario: Unsafe tool allowed when no safety profile on context
|
|
Given a tool execution context without a safety profile
|
|
When I enforce safety and run tool "test/unsafe-tool"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Enforcement still applies read-only checks without safety profile
|
|
Given a tool execution context that is read-only without a safety profile
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolAccessDeniedError should be raised
|
|
|
|
# ---- Sandbox requirement ----
|
|
|
|
Scenario: Writing tool blocked when sandbox required but no sandbox_id
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a ToolSandboxRequiredError should be raised
|
|
And the sandbox error should mention "require_sandbox"
|
|
|
|
Scenario: Writing tool allowed when sandbox required and sandbox_id set
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and sandbox_id "sandbox-001"
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Writing tool allowed when sandbox not required and no sandbox_id
|
|
Given a safety profile with require_sandbox false
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Read-only tool allowed when sandbox required but no sandbox_id
|
|
Given a safety profile with require_sandbox true
|
|
And a tool execution context with the safety profile and no sandbox_id
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Human approval requirement ----
|
|
|
|
Scenario: Tool blocked when human approval required but not granted
|
|
Given a safety profile with require_human_approval true
|
|
And a tool execution context with the safety profile and no approval
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolHumanApprovalRequiredError should be raised
|
|
And the approval error should mention "human approval"
|
|
|
|
Scenario: Tool allowed when human approval required and granted
|
|
Given a safety profile with require_human_approval true
|
|
And a tool execution context with the safety profile and human approval granted
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Tool allowed when human approval not required
|
|
Given a safety profile with require_human_approval false
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Cost limit enforcement ----
|
|
|
|
Scenario: Tool blocked when accumulated cost exceeds max_cost_per_plan
|
|
Given a safety profile with max_cost_per_plan 10.0
|
|
And a tool execution context with the safety profile and accumulated_cost 10.0
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolCostLimitExceededError should be raised
|
|
And the cost error should mention "max_cost_per_plan"
|
|
|
|
Scenario: Tool allowed when accumulated cost is below max_cost_per_plan
|
|
Given a safety profile with max_cost_per_plan 10.0
|
|
And a tool execution context with the safety profile and accumulated_cost 5.0
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Tool blocked when total cost exceeds max_total_cost
|
|
Given a safety profile with max_total_cost 100.0
|
|
And a tool execution context with the safety profile and total_accumulated_cost 100.0
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolCostLimitExceededError should be raised
|
|
And the cost error should mention "max_total_cost"
|
|
|
|
# ---- Retry limit enforcement ----
|
|
|
|
Scenario: Tool blocked when retry count exceeds max_retries_per_step
|
|
Given a safety profile with max_retries_per_step 3
|
|
And a tool execution context with the safety profile and step_retry_count 4
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolRetryLimitExceededError should be raised
|
|
And the retry error should mention "max_retries_per_step"
|
|
|
|
Scenario: Tool allowed when retry count is within max_retries_per_step
|
|
Given a safety profile with max_retries_per_step 3
|
|
And a tool execution context with the safety profile and step_retry_count 2
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Missing skill category metadata ----
|
|
|
|
Scenario: Tool blocked when skill category metadata missing and allow-list set
|
|
Given a safety profile with allowed_skill_categories "code,test"
|
|
And a tool execution context with the safety profile and no skill category metadata
|
|
When I enforce safety and try to run tool "test/reader"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "no skill category"
|
|
|
|
# ---- Combined constraints ----
|
|
|
|
Scenario: Multiple safety violations reported for first failing check
|
|
Given a combined safety profile blocking unsafe tools with categories "code"
|
|
And a tool execution context with the safety profile
|
|
And the tool skill category is "deploy"
|
|
When I enforce safety and try to run tool "test/unsafe-tool"
|
|
Then a ToolSafetyViolationError should be raised
|
|
And the safety violation error should mention "unsafe"
|
|
|
|
# ---- Tool allow-list enforcement ----
|
|
|
|
Scenario: Tool blocked when not on allow-list
|
|
Given a safety profile with allowed_tools "test/reader"
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolAccessDeniedError should be raised
|
|
And the access denied error should mention "allow-list"
|
|
|
|
Scenario: Tool allowed when on allow-list
|
|
Given a safety profile with allowed_tools "test/reader,test/writer"
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: All tools allowed when allow-list is empty
|
|
Given a safety profile with empty allowed_tools
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/writer"
|
|
Then the tool execution should succeed
|
|
|
|
# ---- Tool deny-list enforcement ----
|
|
|
|
Scenario: Tool blocked when on deny-list
|
|
Given a safety profile with denied_tools "test/writer"
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolAccessDeniedError should be raised
|
|
And the access denied error should mention "deny-list"
|
|
|
|
Scenario: Tool allowed when not on deny-list
|
|
Given a safety profile with denied_tools "test/unsafe-tool"
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/reader"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Deny-list takes precedence over allow-list
|
|
Given a safety profile with both allowed_tools "test/writer" and denied_tools "test/writer"
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolAccessDeniedError should be raised
|
|
And the access denied error should mention "deny-list"
|
|
|
|
# ---- Built-in safety profiles ----
|
|
|
|
Scenario: Unrestricted built-in profile allows unsafe tools
|
|
Given the unrestricted built-in safety profile
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and run tool "test/unsafe-tool"
|
|
Then the tool execution should succeed
|
|
|
|
Scenario: Read-only built-in profile blocks unsafe tools
|
|
Given the read-only built-in safety profile
|
|
And a tool execution context with the safety profile
|
|
When I enforce safety and try to run tool "test/unsafe-tool"
|
|
Then a ToolSafetyViolationError should be raised
|
|
|
|
Scenario: Sandboxed built-in profile requires checkpoints
|
|
Given the sandboxed built-in safety profile
|
|
And a tool execution context with the safety profile and require_checkpoints false
|
|
When I enforce safety and try to run tool "test/writer"
|
|
Then a safety ToolCheckpointRequiredError should be raised
|