259 lines
11 KiB
Gherkin
259 lines
11 KiB
Gherkin
Feature: CI workflow validation
|
|
As a developer
|
|
I want to ensure the CI workflow exists and uses nox sessions
|
|
So that all CI checks run through the same tooling as local development
|
|
|
|
Scenario: CI workflow file exists
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
Then the CI workflow file should exist
|
|
|
|
Scenario: CI workflow references nox for lint
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "lint"
|
|
And the job "lint" should run "nox -s lint"
|
|
|
|
Scenario: CI workflow references nox for typecheck
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "typecheck"
|
|
And the job "typecheck" should run "nox -s typecheck"
|
|
|
|
Scenario: CI workflow references nox for unit tests
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "unit_tests"
|
|
And the job "unit_tests" should run "nox -s unit_tests"
|
|
|
|
Scenario: CI workflow references nox for integration tests
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "integration_tests"
|
|
And the job "integration_tests" should run "nox -s integration_tests"
|
|
|
|
Scenario: CI workflow references nox for coverage
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "coverage"
|
|
And the job "coverage" should run "nox -s coverage_report"
|
|
|
|
Scenario: CI workflow references nox for security
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "security"
|
|
And the job "security" should run "nox -s security_scan"
|
|
|
|
Scenario: CI workflow references nox for build
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "build"
|
|
And the job "build" should run "nox -s build"
|
|
|
|
Scenario: CI workflow uses Python 3.13
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow env should set "PYTHON_VERSION" to "3.13"
|
|
|
|
Scenario: CI workflow coverage job depends on lint and typecheck
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "coverage" should depend on "lint"
|
|
And the job "coverage" should depend on "typecheck"
|
|
|
|
Scenario: All required nox sessions are referenced
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should reference these nox sessions:
|
|
| session |
|
|
| lint |
|
|
| typecheck |
|
|
| unit_tests |
|
|
| integration_tests |
|
|
| coverage_report |
|
|
| security_scan |
|
|
| dead_code |
|
|
| complexity |
|
|
| build |
|
|
|
|
# --- Release pipeline scenarios ---
|
|
|
|
Scenario: Release workflow file exists
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
Then the CI workflow file should exist
|
|
|
|
Scenario: Release workflow YAML is valid
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow YAML should be valid
|
|
|
|
Scenario: Release workflow triggers on version tags
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should trigger on push tags matching "v*"
|
|
|
|
Scenario: Release workflow has build-wheel job
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "build-wheel"
|
|
And the job "build-wheel" should run "nox -s build"
|
|
|
|
Scenario: Release workflow has build-docker job
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "build-docker"
|
|
|
|
Scenario: Release workflow has create-release job
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "create-release"
|
|
|
|
Scenario: Release workflow create-release depends on build jobs
|
|
Given the CI workflow file at ".forgejo/workflows/release.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "create-release" should depend on "build-wheel"
|
|
And the job "create-release" should depend on "build-docker"
|
|
|
|
# --- Status-check consolidation job ---
|
|
|
|
Scenario: CI workflow has status-check consolidation job
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "status-check"
|
|
|
|
Scenario: Status-check job depends on all required jobs
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "status-check" should depend on "lint"
|
|
And the job "status-check" should depend on "typecheck"
|
|
And the job "status-check" should depend on "security"
|
|
And the job "status-check" should depend on "unit_tests"
|
|
And the job "status-check" should depend on "coverage"
|
|
|
|
# --- Coverage threshold ---
|
|
# Note: #4227 is closed, but this specific scenario still fails because
|
|
# the CI YAML does not yet encode a 97% threshold as a machine-readable
|
|
# value that the step definition can assert. The other three #4227
|
|
# scenarios (noxfile-level checks) were promoted to regression guards.
|
|
|
|
@tdd_issue @tdd_issue_4227 @tdd_expected_fail
|
|
Scenario: CI coverage threshold is 97%
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the coverage job should enforce a 97% threshold
|
|
|
|
# --- Branch triggers ---
|
|
|
|
Scenario: CI workflow triggers on push to master
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should trigger on push to "master"
|
|
|
|
Scenario: CI workflow triggers on pull requests to master
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should trigger on pull_request to "master"
|
|
|
|
# --- Dependency caching ---
|
|
|
|
Scenario: CI workflow uses dependency caching
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then at least one job should use actions/cache
|
|
|
|
# --- Helm CI job ---
|
|
|
|
Scenario: CI workflow has helm validation job
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "helm"
|
|
|
|
Scenario: Helm job runs helm lint
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "helm" should run "helm lint"
|
|
|
|
Scenario: Helm job runs helm template
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "helm" should run "helm template"
|
|
|
|
Scenario: Helm job validates rendered manifests with kubeconform
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "helm" should run "kubeconform"
|
|
|
|
Scenario: Status-check job depends on helm job
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "status-check" should depend on "helm"
|
|
|
|
# --- Tool version centralization ---
|
|
|
|
Scenario: Tool versions file exists
|
|
Given the tool versions file at ".tool-versions"
|
|
Then the tool versions file should exist
|
|
|
|
Scenario: Tool versions file contains UV_VERSION
|
|
Given the tool versions file at ".tool-versions"
|
|
When I parse the tool versions file
|
|
Then the tool versions should contain "UV_VERSION"
|
|
|
|
Scenario: Tool versions file contains PYTHON_VERSION
|
|
Given the tool versions file at ".tool-versions"
|
|
When I parse the tool versions file
|
|
Then the tool versions should contain "PYTHON_VERSION"
|
|
|
|
Scenario: Tool versions file contains HELM_VERSION
|
|
Given the tool versions file at ".tool-versions"
|
|
When I parse the tool versions file
|
|
Then the tool versions should contain "HELM_VERSION"
|
|
|
|
Scenario: Tool versions file contains KUBECONFORM_VERSION
|
|
Given the tool versions file at ".tool-versions"
|
|
When I parse the tool versions file
|
|
Then the tool versions should contain "KUBECONFORM_VERSION"
|
|
|
|
Scenario: CI workflow reads UV_VERSION from tool versions file
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the workflow should have a job named "load-versions"
|
|
And the job "load-versions" should run "load tool versions from .tool-versions"
|
|
|
|
Scenario: CI workflow load-versions job outputs uv-version
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "load-versions" should have output "uv-version"
|
|
|
|
Scenario: CI workflow load-versions job outputs python-version
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "load-versions" should have output "python-version"
|
|
|
|
Scenario: CI workflow load-versions job outputs helm-version
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "load-versions" should have output "helm-version"
|
|
|
|
Scenario: CI workflow load-versions job outputs kubeconform-version
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "load-versions" should have output "kubeconform-version"
|
|
|
|
Scenario: CI workflow jobs depend on load-versions
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "lint" should depend on "load-versions"
|
|
And the job "typecheck" should depend on "load-versions"
|
|
And the job "unit_tests" should depend on "load-versions"
|
|
And the job "integration_tests" should depend on "load-versions"
|
|
And the job "e2e_tests" should depend on "load-versions"
|
|
And the job "helm" should depend on "load-versions"
|
|
And the job "build" should depend on "load-versions"
|
|
|
|
Scenario: CI workflow uses load-versions outputs for uv installation
|
|
Given the CI workflow file at ".forgejo/workflows/ci.yml"
|
|
When I parse the CI workflow YAML
|
|
Then the job "lint" should reference "needs.load-versions.outputs.uv-version"
|
|
And the job "typecheck" should reference "needs.load-versions.outputs.uv-version"
|
|
And the job "unit_tests" should reference "needs.load-versions.outputs.uv-version"
|