020874536c
Add 12 new Behave BDD scenarios to features/security_scan_hooks.feature to automatically validate the Semgrep broad exception suppression rules and their escape hatch behavior, as suggested by reviewer HAL9001. New scenarios verify: - python-no-suppressed-exception rule exists and targets src/ - python-no-suppress-exception rule exists and targets src/ - Both rules document the nosemgrep escape hatch in their messages - Both rules document the error-propagation: allow annotation - python-no-suppressed-exception has pattern-not for bare re-raise - python-no-suppressed-exception has pattern-not for exception chaining - Nox lint session integrates Semgrep and references .semgrep.yml All 25 scenarios pass (13 original + 12 new). ISSUES CLOSED: #9103
130 lines
5.6 KiB
Gherkin
130 lines
5.6 KiB
Gherkin
Feature: Security scan hooks configuration
|
|
As a QA developer
|
|
I want to verify that Bandit and Semgrep hooks are properly declared
|
|
So that security scanning runs automatically on every commit
|
|
|
|
Scenario: Bandit hook is declared in pre-commit config
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then a hook with id "bandit" should be declared
|
|
|
|
Scenario: Bandit hook targets src directory only
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then the "bandit" hook should have a files pattern matching src
|
|
|
|
Scenario: Bandit hook uses pyproject.toml config
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then the "bandit" hook args should reference pyproject.toml
|
|
|
|
Scenario: Semgrep hook is declared in pre-commit config
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then a hook with id "semgrep-eval-exec" should be declared
|
|
|
|
Scenario: Semgrep hook references semgrep config file
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then the "semgrep-eval-exec" hook entry should reference .semgrep.yml
|
|
|
|
Scenario: Semgrep hook targets src directory
|
|
Given the pre-commit config file exists
|
|
When I parse the pre-commit configuration
|
|
Then the "semgrep-eval-exec" hook should have a files pattern matching src
|
|
|
|
Scenario: Semgrep is listed as a project dev dependency
|
|
Given the pyproject.toml exists
|
|
When I parse the dev dependencies from pyproject.toml
|
|
Then "semgrep" should be in the dev dependencies
|
|
|
|
Scenario: Bandit is listed as a project dev dependency
|
|
Given the pyproject.toml exists
|
|
When I parse the dev dependencies from pyproject.toml
|
|
Then "bandit" should be in the dev dependencies
|
|
|
|
Scenario: Nox security_scan session exists in noxfile
|
|
Given the noxfile.py exists
|
|
When I parse the nox session names from noxfile.py
|
|
Then "security_scan" should be a registered nox session
|
|
|
|
Scenario: Nox security_scan session runs bandit
|
|
Given the noxfile.py exists
|
|
When I read the security_scan session source from noxfile.py
|
|
Then the session source should contain a bandit invocation
|
|
|
|
Scenario: Nox security_scan session runs semgrep
|
|
Given the noxfile.py exists
|
|
When I read the security_scan session source from noxfile.py
|
|
Then the session source should contain a semgrep invocation
|
|
|
|
Scenario: Nox security_scan session runs vulture
|
|
Given the noxfile.py exists
|
|
When I read the security_scan session source from noxfile.py
|
|
Then the session source should contain a vulture invocation
|
|
|
|
Scenario: Semgrep config file exists with custom rules
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the semgrep config should contain at least 3 rules
|
|
|
|
Scenario: Semgrep config contains broad exception suppression rule
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the semgrep config should contain a rule with id "python-no-suppressed-exception"
|
|
|
|
Scenario: Semgrep config contains contextlib suppress rule
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the semgrep config should contain a rule with id "python-no-suppress-exception"
|
|
|
|
Scenario: Broad exception suppression rule targets src directory
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppressed-exception" should include path "src/"
|
|
|
|
Scenario: Contextlib suppress rule targets src directory
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppress-exception" should include path "src/"
|
|
|
|
Scenario: Broad exception suppression rule documents nosemgrep escape hatch
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppressed-exception" message should mention "nosemgrep"
|
|
|
|
Scenario: Contextlib suppress rule documents nosemgrep escape hatch
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppress-exception" message should mention "nosemgrep"
|
|
|
|
Scenario: Broad exception suppression rule documents error-propagation annotation
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppressed-exception" message should mention "error-propagation: allow"
|
|
|
|
Scenario: Contextlib suppress rule documents error-propagation annotation
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppress-exception" message should mention "error-propagation: allow"
|
|
|
|
Scenario: Broad exception suppression rule allows bare re-raise
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppressed-exception" should have a pattern-not for bare re-raise
|
|
|
|
Scenario: Broad exception suppression rule allows exception chaining
|
|
Given the semgrep config file exists
|
|
When I parse the semgrep configuration
|
|
Then the rule "python-no-suppressed-exception" should have a pattern-not for exception chaining
|
|
|
|
Scenario: Nox lint session integrates Semgrep
|
|
Given the noxfile.py exists
|
|
When I read the lint session source from noxfile.py
|
|
Then the lint session source should contain a semgrep invocation
|
|
|
|
Scenario: Nox lint session uses semgrep config file
|
|
Given the noxfile.py exists
|
|
When I read the lint session source from noxfile.py
|
|
Then the lint session source should reference ".semgrep.yml"
|