Files
cleveragents-core/features/tool_builtins.feature
T
HAL9000 e18ac5f23c
CI / lint (pull_request) Successful in 20s
CI / quality (pull_request) Successful in 21s
CI / push-validation (pull_request) Successful in 20s
CI / build (pull_request) Successful in 24s
CI / typecheck (pull_request) Successful in 50s
CI / security (pull_request) Successful in 59s
CI / helm (pull_request) Successful in 44s
CI / integration_tests (pull_request) Successful in 4m41s
CI / unit_tests (pull_request) Successful in 5m24s
CI / docker (pull_request) Successful in 52s
CI / coverage (pull_request) Successful in 7m38s
CI / e2e_tests (pull_request) Successful in 2m14s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-regression (push) Failing after 0s
CI / benchmark-publish (push) Failing after 0s
CI / lint (push) Successful in 18s
CI / quality (push) Successful in 40s
CI / typecheck (push) Successful in 41s
CI / security (push) Successful in 42s
CI / build (push) Successful in 25s
CI / push-validation (push) Successful in 30s
CI / helm (push) Successful in 56s
CI / e2e_tests (push) Successful in 3m3s
CI / unit_tests (push) Successful in 4m3s
CI / integration_tests (push) Successful in 4m8s
CI / docker (push) Successful in 52s
CI / coverage (push) Successful in 7m22s
CI / status-check (push) Successful in 1s
fix(security): replace startswith sandbox check with Path.relative_to() in validate_path #7558
The validate_path() function in file_tools.py used str.startswith() to
verify that a resolved path stays within the sandbox root. This allowed
sibling directories whose names share a string prefix with the sandbox
(e.g. /tmp/sandbox-escape/ bypassing /tmp/sandbox/) to escape the
containment check.

Replace the string prefix check with Path.relative_to(root), which
performs a proper OS-level path prefix comparison using path separators.
Add a Behave regression scenario tagged @tdd_issue @tdd_issue_7558 that
exercises the prefix-collision bypass to prevent regressions.

ISSUES CLOSED: #7558
2026-04-17 04:50:55 +00:00

264 lines
11 KiB
Gherkin

Feature: Built-in File Tools
As a developer
I want built-in file operation tools
So that agents can read, write, edit, delete, list, and search files safely
# ---- File Read ----
Scenario: Read a file with default encoding
Given a temporary sandbox directory
And a file "hello.txt" with content "Hello, World!"
When I execute the "builtin/file-read" tool with path "hello.txt"
Then the tool result should be successful
And the output "content" should be "Hello, World!"
And the output "encoding" should be "utf-8"
And the output "size" should be greater than 0
Scenario: Read a file with offset and limit
Given a temporary sandbox directory
And a file "lines.txt" with lines "line1,line2,line3,line4,line5"
When I execute the "builtin/file-read" tool with path "lines.txt" offset 1 limit 2
Then the tool result should be successful
And the output "content" should contain "line2"
And the output "content" should contain "line3"
And the output "content" should not contain "line1"
# ---- File Write ----
Scenario: Write a new file
Given a temporary sandbox directory
When I execute the "builtin/file-write" tool with path "new.txt" and content "created"
Then the tool result should be successful
And the output "created" should be boolean true
And the output "size" should be greater than 0
And the file "new.txt" should exist in the sandbox
Scenario: Overwrite an existing file
Given a temporary sandbox directory
And a file "existing.txt" with content "old content"
When I execute the "builtin/file-write" tool with path "existing.txt" and content "new content"
Then the tool result should be successful
And the output "created" should be boolean false
Scenario: Write file creates parent directories
Given a temporary sandbox directory
When I execute the "builtin/file-write" tool with path "sub/dir/file.txt" and content "nested"
Then the tool result should be successful
And the file "sub/dir/file.txt" should exist in the sandbox
# ---- File Edit ----
Scenario: Edit file with single replacement
Given a temporary sandbox directory
And a file "edit.txt" with content "foo bar foo"
When I execute the "builtin/file-edit" tool replacing "foo" with "baz"
Then the tool result should be successful
And the output "replacements" should equal 1
Scenario: Edit file with replace_all
Given a temporary sandbox directory
And a file "edit.txt" with content "foo bar foo"
When I execute the "builtin/file-edit" tool replacing all "foo" with "baz"
Then the tool result should be successful
And the output "replacements" should equal 2
Scenario: Edit file with old_text not found raises error
Given a temporary sandbox directory
And a file "edit.txt" with content "foo bar"
When I execute the "builtin/file-edit" tool replacing "notfound" with "baz"
Then the tool result should not be successful
And the tool result error should mention "not found"
# ---- File Delete ----
Scenario: Delete an existing file
Given a temporary sandbox directory
And a file "delete-me.txt" with content "doomed"
When I execute the "builtin/file-delete" tool with path "delete-me.txt"
Then the tool result should be successful
And the output "existed" should be boolean true
And the file "delete-me.txt" should not exist in the sandbox
Scenario: Delete a nonexistent file
Given a temporary sandbox directory
When I execute the "builtin/file-delete" tool with path "ghost.txt"
Then the tool result should be successful
And the output "existed" should be boolean false
# ---- File List ----
Scenario: List directory contents
Given a temporary sandbox directory
And a file "a.txt" with content "a"
And a file "b.py" with content "b"
When I list directory "." with the file-list tool
Then the tool result should be successful
And the output files list should contain at least 2 entries
Scenario: List directory with pattern filter
Given a temporary sandbox directory
And a file "a.txt" with content "a"
And a file "b.py" with content "b"
When I list directory "." filtering by pattern "*.txt"
Then the tool result should be successful
And all listed files should match pattern "*.txt"
Scenario: List directory recursively
Given a temporary sandbox directory
And a file "sub/deep.txt" with content "deep"
When I list directory "." recursively with the file-list tool
Then the tool result should be successful
And the output files list should contain at least 1 entries
# ---- File Search ----
Scenario: Search files for pattern
Given a temporary sandbox directory
And a file "haystack.txt" with content "needle in a haystack"
When I search directory "." for pattern "needle"
Then the tool result should be successful
And the search matches should contain at least 1 result
Scenario: Search with max_results limit
Given a temporary sandbox directory
And a file "many.txt" with lines "match,match,match,match,match"
When I search directory "." for pattern "match" with max_results 2
Then the tool result should be successful
And the search matches should contain exactly 2 results
# ---- Path Traversal Prevention ----
Scenario: Path traversal with dotdot is rejected
Given a temporary sandbox directory
When I execute the "builtin/file-read" tool with path "../../etc/passwd"
Then the tool result should not be successful
And the tool result error should mention "traversal"
Scenario: Path traversal in write is rejected
Given a temporary sandbox directory
When I execute the "builtin/file-write" tool with path "../escape.txt" and content "evil"
Then the tool result should not be successful
And the tool result error should mention "traversal"
Scenario: Path traversal in delete is rejected
Given a temporary sandbox directory
When I execute the "builtin/file-delete" tool with path "../../etc/passwd"
Then the tool result should not be successful
And the tool result error should mention "traversal"
@tdd_issue @tdd_issue_7558
Scenario: Path traversal with sandbox name prefix collision is rejected
Given a temporary sandbox directory
And a sibling directory with a name that is a prefix of the sandbox name
When I attempt to read a file in the sibling escape directory
Then the tool result should not be successful
And the tool result error should mention "traversal"
# ---- ChangeSet Capture ----
Scenario: ChangeSet captures write operations
Given a temporary sandbox directory
And a changeset capture with plan_id "plan-001"
When I write a file through changeset-wrapped tools
Then the changeset should have 1 entry
And the changeset entry operation should be "create"
Scenario: ChangeSet captures edit operations
Given a temporary sandbox directory
And a file "cs-edit.txt" with content "hello world"
And a changeset capture with plan_id "plan-002"
When I edit a file through changeset-wrapped tools
Then the changeset should have 1 entry
And the changeset entry operation should be "modify"
Scenario: ChangeSet captures delete operations
Given a temporary sandbox directory
And a file "cs-delete.txt" with content "bye"
And a changeset capture with plan_id "plan-003"
When I delete a file through changeset-wrapped tools
Then the changeset should have 1 entry
And the changeset entry operation should be "delete"
Scenario: ChangeSet summary generation
Given a temporary sandbox directory
And a changeset capture with plan_id "plan-004"
When I perform multiple changeset-tracked operations
Then the changeset summary should mention the total count
And the changeset summary should mention operation types
Scenario: ChangeSet clear resets tracking
Given a temporary sandbox directory
And a changeset capture with plan_id "plan-005"
When I write a file through changeset-wrapped tools
And I clear the changeset
Then the changeset should have 0 entries
Scenario: Empty changeset summary
Given a changeset capture with plan_id "plan-empty"
Then the changeset summary should be "No changes recorded"
# ---- Tool Registration Helper ----
Scenario: Register all file tools
Given a tool registry
When I register all file tools
Then the registry should contain 6 tools
And the registry should contain tool "builtin/file-read"
And the registry should contain tool "builtin/file-write"
And the registry should contain tool "builtin/file-edit"
And the registry should contain tool "builtin/file-delete"
And the registry should contain tool "builtin/file-list"
And the registry should contain tool "builtin/file-search"
Scenario: Register file tools with changeset
Given a tool registry
And a changeset capture with plan_id "plan-reg"
When I register file tools with changeset
Then the registry should contain 6 tools
# ---- ChangeSet model fields ----
Scenario: ChangeSetEntry stores before and after hashes
Given a temporary sandbox directory
And a file "hash-test.txt" with content "original"
And a changeset capture with plan_id "plan-hash"
When I edit "hash-test.txt" through changeset-wrapped tools
Then the changeset entry should have a before_hash
And the changeset entry should have an after_hash
And the before_hash and after_hash should differ
Scenario: File list on non-directory raises error
Given a temporary sandbox directory
And a file "notadir.txt" with content "x"
When I list directory "notadir.txt" with the file-list tool
Then the tool result should not be successful
And the tool result error should mention "not a directory"
Scenario: File search on non-directory raises error
Given a temporary sandbox directory
And a file "notadir.txt" with content "x"
When I search directory "notadir.txt" for pattern "x"
Then the tool result should not be successful
And the tool result error should mention "not a directory"
Scenario: File search skips directories in results
Given a temporary sandbox directory
And a subdirectory "subdir" in the sandbox
And a file "subdir/content.txt" with content "findme"
When I search directory "." for pattern "findme"
Then the tool result should be successful
And the search matches should contain at least 1 result
Scenario: ChangeSet detect_operation fallback create
Given a temporary sandbox directory
And a changeset capture with plan_id "plan-fallback"
When I write a new file through raw changeset capture
Then the changeset entry operation should be "create"
Scenario: ChangeSet detect_operation fallback modify
Given a temporary sandbox directory
And a file "existing-cs.txt" with content "some content"
And a changeset capture with plan_id "plan-modify-fallback"
When I overwrite a file through raw changeset capture
Then the changeset entry operation should be "modify"