CoreRasurae
4fe14c5b77
CI / lint (pull_request) Successful in 1m40s
CI / quality (pull_request) Successful in 2m7s
CI / typecheck (pull_request) Successful in 3m23s
CI / build (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 4m51s
CI / integration_tests (pull_request) Successful in 3m14s
CI / unit_tests (pull_request) Successful in 5m24s
CI / coverage (pull_request) Successful in 6m51s
CI / status-check (pull_request) Successful in 7s
CI / benchmark (pull_request) Failing after 17m20s
CI / lint (push) Successful in 59s
CI / typecheck (push) Successful in 2m21s
CI / security (push) Successful in 1m16s
CI / quality (push) Successful in 51s
CI / build (push) Successful in 1m29s
CI / integration_tests (push) Successful in 3m28s
CI / unit_tests (push) Successful in 4m37s
CI / coverage (push) Successful in 4m58s
CI / status-check (push) Successful in 7s
CI / benchmark (push) Failing after 18m9s
`file_read` always read from position 0; `max_chars` (ADR-2030 D-3) only bounded where the returned content stopped, so once a large file was truncated there was no way to retrieve the next chunk short of falling back to the `shell` tool (`tail -c +N`, `sed -n`, `dd skip=...`) — the same class of awkward, error-prone detour ADR-2030 D-4 already eliminated for file paths mistaken for shell commands. Adds an optional `offset` character-position argument (default 0, byte-for-byte identical output when omitted) to both the tool schema (`llm_tools._BUILTIN_TOOL_SCHEMAS["file_read"]`) and the implementation (`ToolAgent._file_read_tool`). Combined with `max_chars` it returns the window `[offset, offset + max_chars)`, and the `[FILE_READ_SUCCESS]` header now signals continuation: `MORE_CONTENT_AT_OFFSET: N` when more content remains (N is the exact next offset, so callers never compute `offset + max_chars` themselves), or `NO_MORE_CONTENT_AT_OFFSET: N` when the requested offset is at or beyond end-of-file — a success response with empty content, not an error, since that is the expected terminal state of a correctly-paginating caller. Negative or non-integer offsets raise `ExecutionError`, mirroring the existing `max_chars` validation pattern. See docs/adr/ADR-2033-file-read-offset-pagination.md for the full set of design decisions, including why directory-listing pagination and byte-level seeking are deferred to a future issue. ISSUES CLOSED: #83
429 lines
18 KiB
Gherkin
429 lines
18 KiB
Gherkin
Feature: Tool Agent Coverage Gaps
|
|
As a developer
|
|
I want to cover remaining uncovered lines in tool.py
|
|
So that coverage reaches the target
|
|
|
|
Scenario: Tool validation rejects unknown string tool
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with unknown string tool
|
|
Then an AgentCreationError mentioning Unknown tool is raised
|
|
|
|
Scenario: Tool validation rejects dict config without name
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with dict config missing name
|
|
Then an AgentCreationError mentioning must include name is raised
|
|
|
|
Scenario: Tool validation rejects code that is not a string
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with non-string code
|
|
Then an AgentCreationError mentioning must be a string is raised
|
|
|
|
Scenario: Tool validation rejects non-str non-dict tool config
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with invalid tool config type
|
|
Then an AgentCreationError mentioning Invalid tool configuration is raised
|
|
|
|
Scenario: JSON extraction handles invalid JSON in code block
|
|
Given a tool agent coverage test environment
|
|
When I extract JSON from markdown code block with invalid content
|
|
Then the extracted result should be None
|
|
|
|
Scenario: JSON extraction handles valid JSON in code block
|
|
Given a tool agent coverage test environment
|
|
When I extract JSON from markdown code block with valid content
|
|
Then the extracted result should be the expected dict
|
|
|
|
Scenario: Process message handles empty tool request
|
|
Given a tool agent coverage test environment
|
|
When I process an empty message with ToolAgent
|
|
Then an ExecutionError mentioning Empty tool request is raised
|
|
|
|
Scenario: Process message handles invalid JSON lookalike
|
|
Given a tool agent coverage test environment
|
|
When I process a message that looks like invalid JSON
|
|
Then an ExecutionError mentioning Invalid JSON is raised
|
|
|
|
Scenario: Process message handles simple echo format
|
|
Given a tool agent coverage test environment
|
|
When I process an echo message with ToolAgent
|
|
Then the result should contain the echo message
|
|
|
|
Scenario: Execute tool rejects tool not in allowed list
|
|
Given a tool agent coverage test environment
|
|
When I call execute tool with a non-existent tool
|
|
Then an ExecutionError mentioning not in allowed tools list is raised
|
|
|
|
Scenario: Shell command executes successfully via allow_shell
|
|
Given a tool agent coverage test environment
|
|
When I execute a shell command with allow_shell enabled
|
|
Then the shell output should be returned
|
|
|
|
Scenario: Math tool evaluates expression from string
|
|
Given a tool agent coverage test environment
|
|
When I call math tool with expression 2+3
|
|
Then the math result should be 5
|
|
|
|
Scenario: Math tool rejects empty expressions
|
|
Given a tool agent coverage test environment
|
|
When I call math tool with empty args
|
|
Then a math ExecutionError mentioning requires an expression is raised
|
|
|
|
Scenario: Math tool handles evaluation failures
|
|
Given a tool agent coverage test environment
|
|
When I call math tool with expression 1/0
|
|
Then a math ExecutionError mentioning Math evaluation failed is raised
|
|
|
|
Scenario: JSON parse tool parses from args list
|
|
Given a tool agent coverage test environment
|
|
When I call json parse tool with args list
|
|
Then the json parse result contains expected key
|
|
|
|
Scenario: JSON parse tool parses from json key
|
|
Given a tool agent coverage test environment
|
|
When I call json parse tool with json key
|
|
Then the json parse result contains expected key
|
|
|
|
Scenario: JSON parse tool handles invalid JSON
|
|
Given a tool agent coverage test environment
|
|
When I call json parse tool with invalid json
|
|
Then the json parse raises ExecutionError
|
|
|
|
Scenario: HTTP request tool performs GET request
|
|
Given a tool agent coverage test environment
|
|
When I call http request tool with a GET request
|
|
Then the http result contains Status 200
|
|
|
|
Scenario: HTTP request tool requires URL
|
|
Given a tool agent coverage test environment
|
|
When I call http request tool without URL
|
|
Then the http request raises ExecutionError
|
|
|
|
Scenario: HTTP request tool handles connection failures
|
|
Given a tool agent coverage test environment
|
|
When I call http request tool with a failing connection
|
|
Then the http request raises ExecutionError
|
|
|
|
Scenario: File read tool reads file content
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a valid file path
|
|
Then the file read result contains the file content
|
|
|
|
Scenario: File read tool reads file from args list
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with args list
|
|
Then the file read result contains the file content
|
|
|
|
Scenario: File read tool blocks directory traversal
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a traversal path
|
|
Then the file read raises Unsafe file path blocked
|
|
|
|
Scenario: File read tool blocks absolute paths
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with an absolute path
|
|
Then the file read raises Unsafe file path blocked
|
|
|
|
Scenario: File read tool raises File read failed for missing file
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a missing file path
|
|
Then the file read raises File not found
|
|
|
|
Scenario: File read tool requires a file path
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with empty args
|
|
Then the file read raises requires a file path
|
|
|
|
Scenario: File write tool writes content successfully
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with valid content in unsafe mode
|
|
Then the file write result contains Successfully wrote
|
|
|
|
Scenario: File write tool appends content successfully
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with append mode in unsafe mode
|
|
Then the file write result contains Successfully appended
|
|
|
|
Scenario: File write tool inserts content successfully
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with insert mode in unsafe mode
|
|
Then the file write result contains Successfully inserted
|
|
|
|
Scenario: File write tool requires unsafe mode
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool without unsafe mode
|
|
Then the file write raises requires unsafe mode
|
|
|
|
Scenario: File write tool rejects invalid mode
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with invalid mode
|
|
Then the file write raises Invalid mode
|
|
|
|
Scenario: Shell command with echo returns output
|
|
Given a tool agent coverage test environment
|
|
When I execute shell command echo with args
|
|
Then the shell output should be hello
|
|
|
|
Scenario: Shell command with non-zero exit raises error
|
|
Given a tool agent coverage test environment
|
|
When I execute shell command false
|
|
Then a shell ExecutionError mentioning Command failed is raised
|
|
|
|
Scenario: Shell command timeout raises error
|
|
Given a tool agent coverage test environment
|
|
When a shell command times out via asyncio TimeoutError
|
|
Then a shell ExecutionError mentioning timed out is raised
|
|
|
|
Scenario: Shell dangerous command is blocked
|
|
Given a tool agent coverage test environment
|
|
When I execute a dangerous shell command rm
|
|
Then a shell ExecutionError mentioning Dangerous command is raised
|
|
|
|
Scenario: File path safety blocks traversal in safe mode
|
|
Given a tool agent coverage test environment
|
|
When I validate file path with traversal
|
|
Then a path safety ExecutionError is raised
|
|
|
|
Scenario: File path safety blocks absolute paths in safe mode
|
|
Given a tool agent coverage test environment
|
|
When I validate file path with absolute path
|
|
Then a path safety ExecutionError is raised
|
|
|
|
Scenario: File path safety blocks home directory paths
|
|
Given a tool agent coverage test environment
|
|
When I validate file path with home directory
|
|
Then a path safety ExecutionError is raised
|
|
|
|
Scenario: Prepare append content adds newline before section
|
|
Given a tool agent coverage test environment
|
|
When I prepare append content for a section
|
|
Then the content should start with a newline
|
|
|
|
Scenario: Handle insert position appends at end
|
|
Given a tool agent coverage test environment
|
|
When I insert content at the end of a file
|
|
Then the content should be at the end
|
|
|
|
Scenario: Handle insert position inserts at start
|
|
Given a tool agent coverage test environment
|
|
When I insert content at the start of a file
|
|
Then the content should be at the start
|
|
|
|
Scenario: Handle insert position inserts at a specific line
|
|
Given a tool agent coverage test environment
|
|
When I insert content at line 2 of a file
|
|
Then the content should be at line 2
|
|
|
|
Scenario: Handle insert position rejects invalid position
|
|
Given a tool agent coverage test environment
|
|
When I insert content with an invalid position
|
|
Then an ExecutionError mentioning Invalid position is raised in insert
|
|
|
|
Scenario: Handle insert position creates file if nonexistent
|
|
Given a tool agent coverage test environment
|
|
When I insert content into a nonexistent file
|
|
Then a single-line file should be created
|
|
|
|
Scenario: Capabilities include file operations with dict tools
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with dict file tools
|
|
Then capabilities should include file-operations
|
|
|
|
Scenario: Capabilities include http requests with dict tools
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with dict http tools
|
|
Then capabilities should include http-requests
|
|
|
|
Scenario: Capabilities include math evaluation with dict tools
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with dict math tools
|
|
Then capabilities should include math-evaluation
|
|
|
|
Scenario: Context merge preserves agent context
|
|
Given a tool agent coverage test environment
|
|
When I process a message with external context and agent context
|
|
Then the agent should process messages without error
|
|
|
|
Scenario: Inline code tool extracts result from output key
|
|
Given a tool agent coverage test environment
|
|
When I process a message with inline code returning output key
|
|
Then the result should be extracted from output key
|
|
|
|
Scenario: File read handles empty args list
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with empty args list
|
|
Then the file read raises requires a file path
|
|
|
|
Scenario: File write validates empty filepath
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with empty filepath
|
|
Then the file write raises ExecutionError
|
|
|
|
Scenario: File write validates empty content
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with empty content
|
|
Then the file write raises ExecutionError
|
|
|
|
Scenario: File path safety handles ValueError from normpath
|
|
Given a tool agent coverage test environment
|
|
When os.path.normpath raises ValueError
|
|
Then the path safety validates and raises ExecutionError
|
|
|
|
Scenario: File path safety handles OSError from abspath
|
|
Given a tool agent coverage test environment
|
|
When os.path.abspath raises OSError
|
|
Then the path safety validates and raises ExecutionError
|
|
|
|
Scenario: Progress bar tool renders successfully
|
|
Given a tool agent coverage test environment
|
|
When I call progress_bar tool with stage and message
|
|
Then the result should contain Progress updated
|
|
|
|
Scenario: Progress bar tool uses phase and label fallbacks
|
|
Given a tool agent coverage test environment
|
|
When I call progress_bar tool with phase and label args
|
|
Then the result should contain Progress updated
|
|
|
|
Scenario: _to_int returns None for non-convertible values
|
|
Given a tool agent coverage test environment
|
|
When I call progress_bar tool with non-numeric remaining
|
|
Then remaining should be treated as None
|
|
|
|
Scenario: File write dict-only capabilities covers line 802
|
|
Given a tool agent coverage test environment
|
|
When I create ToolAgent with file_write dict only
|
|
Then capabilities should include file-operations
|
|
|
|
Scenario: _prepare_append_content appends with double newline for no trailing newline
|
|
Given a tool agent coverage test environment
|
|
When I prepare append content for file without trailing newline
|
|
Then the prefix should have double newline
|
|
|
|
Scenario: _prepare_append_content prefix empty for double newline ending
|
|
Given a tool agent coverage test environment
|
|
When I prepare append content for file ending with double newline
|
|
Then the prefix should be empty
|
|
|
|
Scenario: _prepare_append_content FileNotFoundError
|
|
Given a tool agent coverage test environment
|
|
When I prepare append content for missing file
|
|
Then the prefix should be empty
|
|
|
|
Scenario: _execute_tool shell disabled cannot execute
|
|
Given a tool agent coverage test environment
|
|
When I execute a non-builtin dict tool without allow_shell
|
|
Then an ExecutionError for shell execution disabled is raised
|
|
|
|
Scenario: _extract_json_from_message regex dict extraction
|
|
Given a tool agent coverage test environment
|
|
When I extract json from prefix text with embedded object
|
|
Then it should return the parsed dict
|
|
|
|
Scenario: _execute_python_code writing_stage else None
|
|
Given a tool agent coverage test environment
|
|
When I execute python code with non-dict context
|
|
Then the code should execute successfully
|
|
|
|
Scenario: python_exec tool requires exec_python enabled
|
|
Given a tool agent coverage test environment
|
|
When I call python_exec tool without exec_python enabled
|
|
Then a shell ExecutionError mentioning Python execution is disabled is raised
|
|
|
|
Scenario: python_exec tool requires code argument
|
|
Given a tool agent coverage test environment
|
|
When I call python_exec tool without code argument
|
|
Then a shell ExecutionError mentioning requires a 'code' argument is raised
|
|
|
|
Scenario: shell tool requires allow_shell enabled
|
|
Given a tool agent coverage test environment
|
|
When I call shell tool without allow_shell
|
|
Then a shell ExecutionError mentioning disabled is raised
|
|
|
|
Scenario: shell tool requires command argument
|
|
Given a tool agent coverage test environment
|
|
When I call shell tool without command argument
|
|
Then a shell ExecutionError mentioning requires a 'command' argument is raised
|
|
|
|
Scenario: _execute_python_code captures stdout fallback
|
|
Given a tool agent coverage test environment
|
|
When I execute python code that prints to stdout
|
|
Then the result should contain printed output
|
|
|
|
Scenario: _execute_python_code NameError in sandbox
|
|
Given a tool agent coverage test environment
|
|
When I execute python code with undefined variable
|
|
Then a shell ExecutionError mentioning not available in the Python sandbox is raised
|
|
|
|
Scenario: File read tool detects shell command in file path
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a shell command path
|
|
Then the file read raises looks like a shell command
|
|
|
|
Scenario: File read tool validates max_chars parameter
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with invalid max_chars
|
|
Then the file read raises must be an integer
|
|
|
|
Scenario: File read tool lists directory contents
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool on an existing directory
|
|
Then the result should contain directory listing header
|
|
|
|
Scenario: File read tool handles max_chars truncation
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with small max_chars limit
|
|
Then the result should be truncated
|
|
|
|
Scenario: File read tool offset zero reproduces default behavior
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with and without offset zero
|
|
Then the two results should be identical
|
|
|
|
Scenario: File read tool offset returns content starting at that position
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with an offset mid-file
|
|
Then the result should contain the windowed content
|
|
|
|
Scenario: File read tool pages through a file across two calls with no gap or overlap
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool twice paginating with offset and max_chars
|
|
Then the combined pages should equal the full file content
|
|
|
|
Scenario: File read tool validates negative offset
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a negative offset
|
|
Then the file read raises offset must be non-negative
|
|
|
|
Scenario: File read tool validates non-integer offset
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with a non-integer offset
|
|
Then the file read raises offset must be an integer
|
|
|
|
Scenario: File read tool offset at end of file signals no more content
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with offset at end of file
|
|
Then the result should signal no more content
|
|
|
|
Scenario: File read tool offset beyond end of file signals no more content
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool with offset beyond end of file
|
|
Then the result should signal no more content
|
|
|
|
Scenario: File read tool raises File read failed for read error
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool causing a read error
|
|
Then the file read raises File read failed
|
|
|
|
Scenario: File write tool blocks absolute path in safe mode
|
|
Given a tool agent coverage test environment
|
|
When I call file write tool with absolute path in safe mode
|
|
Then the file write raises Unsafe file path blocked
|
|
|
|
Scenario: File read tool directory listing handles OSError for entry size
|
|
Given a tool agent coverage test environment
|
|
When I call file read tool on a directory with unreadable entry
|
|
Then the result should contain directory listing header
|
|
|
|
Scenario: Process message handles JSONDecodeError
|
|
Given a tool agent coverage test environment
|
|
When I process a message with malformed JSON
|
|
Then an ExecutionError mentioning Invalid JSON is raised |