Files
cleveragents-core/features/security_scan_hooks.feature
HAL9000 020874536c test(test-infra): add BDD scenarios for Semgrep exception suppression rules
Add 12 new Behave BDD scenarios to features/security_scan_hooks.feature
to automatically validate the Semgrep broad exception suppression rules
and their escape hatch behavior, as suggested by reviewer HAL9001.

New scenarios verify:
- python-no-suppressed-exception rule exists and targets src/
- python-no-suppress-exception rule exists and targets src/
- Both rules document the nosemgrep escape hatch in their messages
- Both rules document the error-propagation: allow annotation
- python-no-suppressed-exception has pattern-not for bare re-raise
- python-no-suppressed-exception has pattern-not for exception chaining
- Nox lint session integrates Semgrep and references .semgrep.yml

All 25 scenarios pass (13 original + 12 new).

ISSUES CLOSED: #9103
2026-06-14 19:05:21 -04:00

130 lines
5.6 KiB
Gherkin

Feature: Security scan hooks configuration
As a QA developer
I want to verify that Bandit and Semgrep hooks are properly declared
So that security scanning runs automatically on every commit
Scenario: Bandit hook is declared in pre-commit config
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then a hook with id "bandit" should be declared
Scenario: Bandit hook targets src directory only
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then the "bandit" hook should have a files pattern matching src
Scenario: Bandit hook uses pyproject.toml config
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then the "bandit" hook args should reference pyproject.toml
Scenario: Semgrep hook is declared in pre-commit config
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then a hook with id "semgrep-eval-exec" should be declared
Scenario: Semgrep hook references semgrep config file
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then the "semgrep-eval-exec" hook entry should reference .semgrep.yml
Scenario: Semgrep hook targets src directory
Given the pre-commit config file exists
When I parse the pre-commit configuration
Then the "semgrep-eval-exec" hook should have a files pattern matching src
Scenario: Semgrep is listed as a project dev dependency
Given the pyproject.toml exists
When I parse the dev dependencies from pyproject.toml
Then "semgrep" should be in the dev dependencies
Scenario: Bandit is listed as a project dev dependency
Given the pyproject.toml exists
When I parse the dev dependencies from pyproject.toml
Then "bandit" should be in the dev dependencies
Scenario: Nox security_scan session exists in noxfile
Given the noxfile.py exists
When I parse the nox session names from noxfile.py
Then "security_scan" should be a registered nox session
Scenario: Nox security_scan session runs bandit
Given the noxfile.py exists
When I read the security_scan session source from noxfile.py
Then the session source should contain a bandit invocation
Scenario: Nox security_scan session runs semgrep
Given the noxfile.py exists
When I read the security_scan session source from noxfile.py
Then the session source should contain a semgrep invocation
Scenario: Nox security_scan session runs vulture
Given the noxfile.py exists
When I read the security_scan session source from noxfile.py
Then the session source should contain a vulture invocation
Scenario: Semgrep config file exists with custom rules
Given the semgrep config file exists
When I parse the semgrep configuration
Then the semgrep config should contain at least 3 rules
Scenario: Semgrep config contains broad exception suppression rule
Given the semgrep config file exists
When I parse the semgrep configuration
Then the semgrep config should contain a rule with id "python-no-suppressed-exception"
Scenario: Semgrep config contains contextlib suppress rule
Given the semgrep config file exists
When I parse the semgrep configuration
Then the semgrep config should contain a rule with id "python-no-suppress-exception"
Scenario: Broad exception suppression rule targets src directory
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppressed-exception" should include path "src/"
Scenario: Contextlib suppress rule targets src directory
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppress-exception" should include path "src/"
Scenario: Broad exception suppression rule documents nosemgrep escape hatch
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppressed-exception" message should mention "nosemgrep"
Scenario: Contextlib suppress rule documents nosemgrep escape hatch
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppress-exception" message should mention "nosemgrep"
Scenario: Broad exception suppression rule documents error-propagation annotation
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppressed-exception" message should mention "error-propagation: allow"
Scenario: Contextlib suppress rule documents error-propagation annotation
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppress-exception" message should mention "error-propagation: allow"
Scenario: Broad exception suppression rule allows bare re-raise
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppressed-exception" should have a pattern-not for bare re-raise
Scenario: Broad exception suppression rule allows exception chaining
Given the semgrep config file exists
When I parse the semgrep configuration
Then the rule "python-no-suppressed-exception" should have a pattern-not for exception chaining
Scenario: Nox lint session integrates Semgrep
Given the noxfile.py exists
When I read the lint session source from noxfile.py
Then the lint session source should contain a semgrep invocation
Scenario: Nox lint session uses semgrep config file
Given the noxfile.py exists
When I read the lint session source from noxfile.py
Then the lint session source should reference ".semgrep.yml"