CoreRasurae
134 lines
6.3 KiB
Gherkin
134 lines
6.3 KiB
Gherkin
Feature: SEC1 - eval/exec removal security
|
|
As a security-conscious developer
|
|
I want to verify that eval() and exec() are fully removed from config parsing
|
|
So that code injection attacks are impossible
|
|
|
|
Scenario: Config with code injection attempt is rejected
|
|
Given a SimpleToolAgent configured with a code injection payload
|
|
When I attempt to process content through the injected agent
|
|
Then the agent should reject the code block with a routing error
|
|
And the error message should mention code blocks are not supported
|
|
|
|
Scenario: Malicious transform expression does not execute
|
|
Given the CleverAgents reactive system is available
|
|
When I configure a transform with a malicious eval expression
|
|
Then the transform should be rejected with a routing error
|
|
And the error message should mention unknown transform
|
|
|
|
Scenario: Valid config still works after eval removal
|
|
Given a SimpleToolAgent configured with a named operation "uppercase"
|
|
When I process "hello" through the named operation agent
|
|
Then the named operation result should be "HELLO"
|
|
|
|
Scenario: Named transform from registry works after eval removal
|
|
Given the CleverAgents reactive system is available
|
|
When I configure a transform with a registered named function "uppercase"
|
|
Then the named transform should produce the uppercased result
|
|
|
|
Scenario: Code block with import attempt is rejected
|
|
Given a SimpleToolAgent configured with an import injection payload
|
|
When I attempt to process content through the injected agent
|
|
Then the agent should reject the code block with a routing error
|
|
|
|
Scenario: Eval expression mimicking registry name is still rejected
|
|
Given the CleverAgents reactive system is available
|
|
When I configure a transform with expression "lambda x: __import__('os').system('rm -rf /')"
|
|
Then the transform should be rejected with a routing error
|
|
|
|
Scenario: Custom registered operation works
|
|
Given a SimpleToolAgent with a custom registered operation "reverse"
|
|
When I process "abcde" through the custom operation agent
|
|
Then the custom operation result should be "edcba"
|
|
|
|
Scenario: Custom registered transform works
|
|
Given the CleverAgents reactive system is available
|
|
And a custom transform "double" is registered
|
|
When I configure a transform with a registered named function "double"
|
|
Then the custom transform should produce the doubled result
|
|
|
|
Scenario: Config scanner detects eval in content
|
|
Given config content containing "value: eval('hack')"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report at least 1 violation
|
|
And the scanner should report a violation with token "eval("
|
|
And the violation severity should be "CRITICAL"
|
|
|
|
Scenario: Config scanner detects exec in content
|
|
Given config content containing "action: exec('import os')"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report at least 1 violation
|
|
And the scanner should report a violation with token "exec("
|
|
|
|
Scenario: Config scanner passes clean content
|
|
Given config content containing "name: safe-project"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report 0 violations
|
|
|
|
Scenario: Config scanner reports correct line numbers
|
|
Given multiline config content with a violation on line 3
|
|
When I scan the config content for security violations
|
|
Then the scanner should report a violation on line 3
|
|
|
|
Scenario: Config safety validation raises on dangerous content
|
|
Given config content containing "run: exec('bad')"
|
|
When I validate the config content for safety
|
|
Then a ConfigurationError should be raised
|
|
|
|
Scenario: Config safety validation passes clean content
|
|
Given config content containing "name: safe-project"
|
|
When I validate the config content for safety
|
|
Then no configuration error should be raised
|
|
|
|
Scenario: Config scanner skips comment lines
|
|
Given config content containing "# eval('this is just a comment')"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report 0 violations
|
|
|
|
Scenario: Config scanner skips INI-style semicolon comment lines
|
|
Given config content containing "; eval('this is an INI comment')"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report 0 violations
|
|
|
|
Scenario: Config scanner skips inline comments after hash
|
|
Given config content containing "name: safe-project # eval('just a note')"
|
|
When I scan the config content for security violations
|
|
Then the scanner should report 0 violations
|
|
|
|
Scenario: Config scanner preserves hash inside quoted strings
|
|
Given config content containing "value: \"eval('inside quotes') # not a comment\""
|
|
When I scan the config content for security violations
|
|
Then the scanner should report at least 1 violation
|
|
And the scanner should report a violation with token "eval("
|
|
|
|
Scenario: Config scanner detects multiple violation types
|
|
Given config content with eval and subprocess patterns
|
|
When I scan the config content for security violations
|
|
Then the scanner should report at least 2 violations
|
|
|
|
Scenario: Config scanner scans multiple files
|
|
Given two temporary config files with violations
|
|
When I scan the config files for security violations
|
|
Then the scanner should report at least 2 violations
|
|
|
|
Scenario: Config scanner CLI shows usage when no args given
|
|
When I run the config scanner CLI with no arguments
|
|
Then the scanner CLI should return exit code 2
|
|
|
|
Scenario: Config scanner CLI reports violations found
|
|
When I run the config scanner CLI with a dirty temp file
|
|
Then the scanner CLI should return exit code 1
|
|
And the scanner CLI output should contain "violation"
|
|
|
|
Scenario: Config scanner CLI reports clean file
|
|
When I run the config scanner CLI with a clean temp file
|
|
Then the scanner CLI should return exit code 0
|
|
|
|
Scenario: Config scanner scan_file raises for missing file
|
|
When I scan a non-existent file path
|
|
Then a scanner FileNotFoundError should be raised
|
|
|
|
Scenario: Config scanner scan_file handles non-UTF-8 files gracefully
|
|
Given a temporary config file with non-UTF-8 encoding
|
|
When I scan the non-UTF-8 file for violations
|
|
Then the scan should complete without crashing
|