freemo
30f8e08f7e
CI / lint (push) Failing after 16s
CI / typecheck (push) Successful in 24s
CI / coverage (push) Has been skipped
CI / security (push) Successful in 16s
CI / quality (push) Successful in 14s
CI / build (push) Successful in 13s
CI / behave (3.13) (push) Failing after 4m24s
CI / docker (push) Has been skipped
132 lines
6.8 KiB
Gherkin
132 lines
6.8 KiB
Gherkin
Feature: Quality gates checker script coverage
|
|
Full line and branch coverage for scripts/check-quality-gates.py
|
|
|
|
# --- check_coverage ---
|
|
|
|
Scenario: Coverage gate passes when coverage exceeds minimum threshold
|
|
Given the coverage tool reports a total of 92.5 percent
|
|
When the quality gates check_coverage function is called with minimum 85
|
|
Then the quality gates coverage result should be passed
|
|
And the quality gates coverage message should contain "92.5%"
|
|
|
|
Scenario: Coverage gate fails when coverage is below minimum threshold
|
|
Given the coverage tool reports a total of 70.0 percent
|
|
When the quality gates check_coverage function is called with minimum 85
|
|
Then the quality gates coverage result should be failed
|
|
And the quality gates coverage message should contain "70.0%"
|
|
|
|
Scenario: Coverage gate fails when coverage command returns non-zero exit code
|
|
Given the coverage tool returns a non-zero exit code with stderr "No data to report"
|
|
When the quality gates check_coverage function is called with minimum 85
|
|
Then the quality gates coverage result should be failed
|
|
And the quality gates coverage message should contain "Coverage command failed"
|
|
|
|
Scenario: Coverage gate fails when coverage output cannot be parsed as a number
|
|
Given the coverage tool returns unparseable output "not-a-number"
|
|
When the quality gates check_coverage function is called with minimum 85
|
|
Then the quality gates coverage result should be failed
|
|
And the quality gates coverage message should contain "Could not parse coverage output"
|
|
|
|
# --- check_typecheck ---
|
|
|
|
Scenario: Typecheck gate passes when pyright reports zero errors
|
|
Given pyright reports JSON output with zero errors
|
|
When the quality gates check_typecheck function is called
|
|
Then the quality gates typecheck result should be passed
|
|
And the quality gates typecheck message should contain "0 errors"
|
|
|
|
Scenario: Typecheck gate fails when pyright reports errors
|
|
Given pyright reports JSON output with 5 errors
|
|
When the quality gates check_typecheck function is called
|
|
Then the quality gates typecheck result should be failed
|
|
And the quality gates typecheck message should contain "5 errors"
|
|
|
|
Scenario: Typecheck gate falls back to exit code when JSON parsing fails and passes
|
|
Given pyright returns invalid JSON output with exit code 0
|
|
When the quality gates check_typecheck function is called
|
|
Then the quality gates typecheck result should be passed
|
|
And the quality gates typecheck message should contain "passed"
|
|
|
|
Scenario: Typecheck gate falls back to exit code when JSON parsing fails and fails
|
|
Given pyright returns invalid JSON output with exit code 1
|
|
When the quality gates check_typecheck function is called
|
|
Then the quality gates typecheck result should be failed
|
|
And the quality gates typecheck message should contain "failed (exit code 1)"
|
|
|
|
# --- check_security ---
|
|
|
|
Scenario: Security gate passes when bandit finds no high-severity issues
|
|
Given bandit reports JSON output with zero results
|
|
When the quality gates check_security function is called
|
|
Then the quality gates security result should be passed
|
|
And the quality gates security message should contain "0 high-severity issues"
|
|
|
|
Scenario: Security gate fails when bandit finds high-severity issues
|
|
Given bandit reports JSON output with 3 high-severity results
|
|
When the quality gates check_security function is called
|
|
Then the quality gates security result should be failed
|
|
And the quality gates security message should contain "3 high-severity issues"
|
|
|
|
Scenario: Security gate falls back to exit code on JSON parse error and passes
|
|
Given bandit returns invalid JSON output with exit code 0
|
|
When the quality gates check_security function is called
|
|
Then the quality gates security result should be passed
|
|
And the quality gates security message should contain "passed"
|
|
|
|
Scenario: Security gate falls back to exit code on JSON parse error and fails
|
|
Given bandit returns invalid JSON output with exit code 2
|
|
When the quality gates check_security function is called
|
|
Then the quality gates security result should be failed
|
|
And the quality gates security message should contain "failed (exit code 2)"
|
|
|
|
# --- check_dead_code ---
|
|
|
|
Scenario: Dead code gate passes when vulture finds no dead code
|
|
Given vulture returns exit code 0 with no output
|
|
When the quality gates check_dead_code function is called
|
|
Then the quality gates dead code result should be passed
|
|
And the quality gates dead code message should contain "none detected"
|
|
|
|
Scenario: Dead code gate fails when vulture detects dead code issues
|
|
Given vulture returns exit code 1 with 3 lines of dead code output
|
|
When the quality gates check_dead_code function is called
|
|
Then the quality gates dead code result should be failed
|
|
And the quality gates dead code message should contain "3 issue(s) found"
|
|
|
|
# --- check_complexity ---
|
|
|
|
Scenario: Complexity gate passes when no blocks exceed maximum grade
|
|
Given radon reports JSON output with zero complex blocks
|
|
When the quality gates check_complexity function is called with max grade "F"
|
|
Then the quality gates complexity result should be passed
|
|
And the quality gates complexity message should contain "no blocks with grade >= F"
|
|
|
|
Scenario: Complexity gate fails when blocks exceed maximum grade
|
|
Given radon reports JSON output with 2 complex blocks
|
|
When the quality gates check_complexity function is called with max grade "F"
|
|
Then the quality gates complexity result should be failed
|
|
And the quality gates complexity message should contain "2 block(s) with grade >= F"
|
|
|
|
Scenario: Complexity gate passes on JSON parse error as fallback
|
|
Given radon returns invalid JSON output
|
|
When the quality gates check_complexity function is called with max grade "F"
|
|
Then the quality gates complexity result should be passed
|
|
And the quality gates complexity message should contain "passed (no extreme complexity)"
|
|
|
|
# --- main function ---
|
|
|
|
Scenario: Quality gates main reports all gates passed when everything succeeds
|
|
Given all quality gate functions are mocked to return success
|
|
When the quality gates main function is invoked with default arguments
|
|
Then the quality gates main should return exit code 0
|
|
|
|
Scenario: Quality gates main reports failure when any gate fails
|
|
Given the quality gate coverage function is mocked to return failure
|
|
When the quality gates main function is invoked with default arguments
|
|
Then the quality gates main should return exit code 1
|
|
|
|
Scenario: Quality gates main creates build directory if it does not exist
|
|
Given a temporary working directory with no build subdirectory
|
|
When the quality gates main function is invoked from that temporary directory
|
|
Then a build directory should exist in the temporary working directory
|