forked from cleveragents/cleveragents-core
khyari hamza
2b09568cfa
Tighten ToolRuntime._enforce_capabilities() to block ANY tool with writes=True when plan_read_only is set, removing the not-cap.read_only loophole that allowed certain write tools through. Tool name is now always included in the ToolAccessDeniedError message. Add read_only flag to ChangeSetCapture with ReadOnlyViolationError raised when write-capable tools are wrapped on a read-only plan. Add CLI fail-fast guards on plan execute and plan apply commands that abort before calling the service layer if plan.read_only is True. SkillContext.enforce_write_guard() already included tool name correctly and required no changes. Includes 18 Behave scenarios (90 steps), Robot integration tests, ASV benchmarks, and docs/reference/read_only_actions.md. ISSUES CLOSED: #322
133 lines
6.8 KiB
Gherkin
133 lines
6.8 KiB
Gherkin
@unit
|
|
Feature: Read-only action enforcement
|
|
As a plan operator
|
|
I need the system to enforce read-only constraints at every layer
|
|
So that read-only plans never produce side-effects
|
|
|
|
Background:
|
|
Given the read-only enforcement modules are available
|
|
|
|
# ── ToolRuntime._enforce_capabilities ────────────────────────────────
|
|
|
|
Scenario: Read-only plan blocks a write-capable tool with tool name in error
|
|
Given a tool "builtin/file-write" with writes=True and read_only=False
|
|
And a tool execution context with plan_id "plan-ro-1" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then a ToolAccessDeniedError should be raised for the tool
|
|
And the access-denied message should contain "builtin/file-write"
|
|
And the access-denied message should contain "read-only"
|
|
|
|
Scenario: Read-only plan allows a read-only tool
|
|
Given a tool "builtin/search" with writes=False and read_only=True
|
|
And a tool execution context with plan_id "plan-ro-2" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then no enforcement error should be raised
|
|
|
|
Scenario: Non-read-only plan allows a write-capable tool
|
|
Given a tool "builtin/file-write" with writes=True and read_only=False
|
|
And a tool execution context with plan_id "plan-rw-1" and read_only False
|
|
When I enforce capabilities on the tool
|
|
Then no enforcement error should be raised
|
|
|
|
Scenario: Tool with writes=True is blocked even when cap.read_only is unset
|
|
Given a tool "builtin/hybrid-tool" with writes=True and read_only=False
|
|
And a tool execution context with plan_id "plan-ro-3" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then a ToolAccessDeniedError should be raised for the tool
|
|
And the access-denied message should contain "builtin/hybrid-tool"
|
|
|
|
Scenario: Tool with neither read_only nor writes passes on read-only plan
|
|
Given a tool "builtin/noop" with writes=False and read_only=False
|
|
And a tool execution context with plan_id "plan-ro-4" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then no enforcement error should be raised
|
|
|
|
# ── SkillContext.enforce_write_guard ──────────────────────────────────
|
|
|
|
Scenario: SkillContext enforce_write_guard blocks write tool with tool name
|
|
Given a read-only SkillContext for plan "plan-sk-1"
|
|
When I call enforce_write_guard with tool_name "skill/writer"
|
|
Then a SkillExecutionError should be raised for the guard
|
|
And the write-guard error message should contain "skill/writer"
|
|
And the write-guard error type should be PERMISSION_DENIED
|
|
|
|
Scenario: SkillContext enforce_write_guard allows on writable context
|
|
Given a writable SkillContext for plan "plan-sk-2"
|
|
When I call enforce_write_guard with tool_name "skill/writer"
|
|
Then no write-guard error should be raised
|
|
|
|
# ── ChangeSetCapture read-only enforcement ───────────────────────────
|
|
|
|
Scenario: ChangeSet builder rejects write entry when plan is read-only
|
|
Given a ChangeSetCapture with plan_id "plan-cs-1" and read_only True
|
|
When I try to wrap a write-capable tool spec
|
|
Then a ReadOnlyViolationError should be raised for the capture
|
|
And the capture error message should contain "read-only"
|
|
|
|
Scenario: ChangeSet builder allows wrapping when not read-only
|
|
Given a ChangeSetCapture with plan_id "plan-cs-2" and read_only False
|
|
When I try to wrap a write-capable tool spec
|
|
Then no capture error should be raised
|
|
|
|
Scenario: ChangeSet builder passes read-only tool through unchanged
|
|
Given a ChangeSetCapture with plan_id "plan-cs-3" and read_only True
|
|
When I wrap a read-only tool spec
|
|
Then the wrapped tool spec should be returned unchanged
|
|
|
|
# ── CLI plan apply fail-fast ─────────────────────────────────────────
|
|
|
|
Scenario: CLI lifecycle-apply blocks read-only plan
|
|
Given a Plan domain model with read_only=True and plan_id "plan-cli-1"
|
|
When I check if the plan can be applied
|
|
Then a read-only plan error should be raised
|
|
And the plan apply error message should contain "read-only"
|
|
|
|
Scenario: CLI lifecycle-apply allows writable plan
|
|
Given a Plan domain model with read_only=False and plan_id "plan-cli-2"
|
|
When I check if the plan can be applied
|
|
Then no plan apply error should be raised
|
|
|
|
# ── File write tool blocked ──────────────────────────────────────────
|
|
|
|
Scenario: File write tool blocked under read-only plan via ToolRuntime
|
|
Given a tool "builtin/file-write" with writes=True and read_only=False
|
|
And a tool execution context with plan_id "plan-fw-1" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then a ToolAccessDeniedError should be raised for the tool
|
|
And the access-denied message should contain "builtin/file-write"
|
|
|
|
# ── Git push tool blocked ────────────────────────────────────────────
|
|
|
|
Scenario: Git push tool blocked under read-only plan via ToolRuntime
|
|
Given a tool "git/push" with writes=True and read_only=False
|
|
And a tool execution context with plan_id "plan-gp-1" and read_only True
|
|
When I enforce capabilities on the tool
|
|
Then a ToolAccessDeniedError should be raised for the tool
|
|
And the access-denied message should contain "git/push"
|
|
|
|
# ── Read-only action validates skills ────────────────────────────────
|
|
|
|
Scenario: Read-only action rejects non-read-only skill metadata
|
|
Given a SkillMetadata with read_only=False
|
|
And an Action with read_only=True
|
|
When I validate the action's skill compatibility
|
|
Then a skill compatibility error should be raised
|
|
|
|
Scenario: Read-only action accepts read-only skill metadata
|
|
Given a SkillMetadata with read_only=True
|
|
And an Action with read_only=True
|
|
When I validate the action's skill compatibility
|
|
Then no skill compatibility error should be raised
|
|
|
|
# ── CLI plan execute propagates read_only ────────────────────────────
|
|
|
|
Scenario: Plan execute creates context with read_only from plan
|
|
Given a Plan domain model with read_only=True and plan_id "plan-ex-1"
|
|
When I create a ToolExecutionContext from the plan
|
|
Then the propagated context plan_read_only should be True
|
|
|
|
Scenario: Plan execute creates writable context for writable plan
|
|
Given a Plan domain model with read_only=False and plan_id "plan-ex-2"
|
|
When I create a ToolExecutionContext from the plan
|
|
Then the propagated context plan_read_only should be False
|