fix(ci): address reviewer feedback on Dockerfile.server security scan
CI / lint (pull_request) Successful in 57s
CI / typecheck (pull_request) Successful in 1m3s
CI / security (pull_request) Successful in 1m2s
CI / quality (pull_request) Successful in 1m11s
CI / helm (pull_request) Successful in 33s
CI / build (pull_request) Successful in 37s
CI / push-validation (pull_request) Successful in 30s
CI / e2e_tests (pull_request) Successful in 3m15s
CI / integration_tests (pull_request) Failing after 6m23s
CI / unit_tests (pull_request) Failing after 8m29s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Failing after 3s
CI / benchmark-publish (pull_request) Has been cancelled
CI / benchmark-regression (pull_request) Has been cancelled

- Pin Trivy installation to v0.57.1 with checksum verification instead
  of the insecure curl-pipe-sh install pattern
- Fix BDD step context initialization: load workflow_content in the
  Background step so scenarios 17/24/31 no longer error with AttributeError
- Fix ruff format violations in step definitions
- Add Robot Framework integration test verifying CI scan configuration
- Add CHANGELOG entry for issue #1927

ISSUES CLOSED: #1927
This commit is contained in:
2026-06-10 02:56:05 -04:00
parent ab974c49a9
commit b09d25421e
4 changed files with 111 additions and 31 deletions
+13 -6
View File
@@ -470,16 +470,23 @@ jobs:
- name: Security scan Dockerfile.server image with Trivy
run: |
# Install Trivy
apk add --no-cache curl
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
# Install Trivy at a pinned version with checksum verification
apk add --no-cache curl tar
TRIVY_VERSION="0.57.1"
ARCH="amd64"
TRIVY_TARBALL="trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz"
curl -fsSL "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/${TRIVY_TARBALL}" -o "/tmp/${TRIVY_TARBALL}"
curl -fsSL "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_checksums.txt" -o /tmp/trivy_checksums.txt
cd /tmp && grep "${TRIVY_TARBALL}" trivy_checksums.txt | sha256sum -c
tar -xzf "/tmp/${TRIVY_TARBALL}" -C /usr/local/bin trivy
trivy --version
# Scan the Dockerfile.server image for vulnerabilities
# Exit with non-zero status if HIGH or CRITICAL vulnerabilities are found
trivy image --severity HIGH,CRITICAL --exit-code 1 cleveragents-server:test
# Also generate a detailed report for visibility
echo "=== Detailed Trivy Scan Report ==="
echo "=== Detailed Trivy Scan Report ==="
trivy image --format table cleveragents-server:test || true
helm:
+9
View File
@@ -5,6 +5,15 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Added
- **CI Dockerfile.server security scan with Trivy** (#1927): Added Trivy vulnerability
scan step to the CI docker job. The scan runs against the built `Dockerfile.server`
image and fails the build on HIGH or CRITICAL severity findings. Trivy is installed
at a pinned version with checksum verification to prevent supply chain attacks. Scan
results are surfaced in CI output. Robot Framework integration tests verify the
workflow configuration.
### Changed
- Restored `benchmark-regression` CI job to `master.yml` with `pull_request` trigger guard
@@ -16,15 +16,23 @@ def step_dockerfile_server_exists(context):
@given("Trivy is available in the CI environment")
def step_trivy_available(context):
"""Verify that Trivy is available (or will be in CI)."""
# In CI, Trivy will be installed. For local testing, we just note this requirement.
context.trivy_required = True
workflow_path = Path(".forgejo/workflows/ci.yml")
if workflow_path.exists():
with open(workflow_path) as f:
context.workflow_content = f.read()
else:
context.workflow_content = ""
context.workflow_path = workflow_path
@given("the CI workflow file exists at .forgejo/workflows/ci.yml")
def step_ci_workflow_exists(context):
"""Verify that the CI workflow file exists."""
workflow_path = Path(".forgejo/workflows/ci.yml")
assert workflow_path.exists(), "CI workflow file not found at .forgejo/workflows/ci.yml"
assert workflow_path.exists(), (
"CI workflow file not found at .forgejo/workflows/ci.yml"
)
context.workflow_path = workflow_path
@@ -40,31 +48,38 @@ def step_examine_docker_job(context):
def step_docker_job_includes_scan_step(context):
"""Verify that the docker job includes a security scan step."""
# Look for a step that scans the Dockerfile.server image
assert "Dockerfile.server" in context.workflow_content, \
assert "Dockerfile.server" in context.workflow_content, (
"Dockerfile.server not referenced in docker job"
)
# Look for Trivy or security scanning references
assert "trivy" in context.workflow_content.lower() or "scan" in context.workflow_content.lower(), \
"No security scanning step found in docker job"
assert (
"trivy" in context.workflow_content.lower()
or "scan" in context.workflow_content.lower()
), "No security scanning step found in docker job"
@then("the scan step should use Trivy or equivalent tool")
def step_scan_uses_trivy(context):
"""Verify that the scan step uses Trivy."""
assert "trivy" in context.workflow_content.lower(), \
"Trivy not found in CI workflow"
assert "trivy" in context.workflow_content.lower(), "Trivy not found in CI workflow"
@then("the scan step should be configured to fail on HIGH or CRITICAL severity findings")
@then(
"the scan step should be configured to fail on HIGH or CRITICAL severity findings"
)
def step_scan_fails_on_high_severity(context):
"""Verify that the scan is configured to fail on HIGH or CRITICAL findings."""
# Look for severity configuration
assert "HIGH" in context.workflow_content or "CRITICAL" in context.workflow_content, \
"Severity configuration not found in scan step"
assert (
"HIGH" in context.workflow_content or "CRITICAL" in context.workflow_content
), "Severity configuration not found in scan step"
# Look for exit code handling
assert "exit" in context.workflow_content.lower() or "fail" in context.workflow_content.lower(), \
"Exit code handling not configured for scan failures"
assert (
"exit" in context.workflow_content.lower()
or "fail" in context.workflow_content.lower()
), "Exit code handling not configured for scan failures"
@given("a Dockerfile.server image has been built")
@@ -84,23 +99,26 @@ def step_execute_security_scan(context):
def step_scan_results_displayed(context):
"""Verify that scan results are displayed in output."""
# In CI, this is handled by the workflow output
assert "scan" in context.workflow_content.lower(), \
assert "scan" in context.workflow_content.lower(), (
"Scan output not configured in workflow"
)
@then("the output should include a summary of vulnerabilities found")
def step_output_includes_summary(context):
"""Verify that output includes vulnerability summary."""
# Trivy provides summary by default
assert "trivy" in context.workflow_content.lower(), \
assert "trivy" in context.workflow_content.lower(), (
"Trivy not configured to provide output"
)
@then("the output should indicate the severity levels of findings")
def step_output_includes_severity(context):
"""Verify that output includes severity levels."""
assert "HIGH" in context.workflow_content or "CRITICAL" in context.workflow_content, \
"Severity levels not indicated in output configuration"
assert (
"HIGH" in context.workflow_content or "CRITICAL" in context.workflow_content
), "Severity levels not indicated in output configuration"
@given("a Dockerfile.server image with known HIGH severity vulnerabilities")
@@ -113,24 +131,25 @@ def step_image_with_vulnerabilities(context):
def step_scan_exits_nonzero(context):
"""Verify that scan exits with non-zero status on vulnerabilities."""
# This is verified by the workflow configuration
assert "trivy" in context.workflow_content.lower(), \
"Trivy not configured"
assert "trivy" in context.workflow_content.lower(), "Trivy not configured"
@then("the CI job should fail")
def step_ci_job_fails(context):
"""Verify that the CI job fails on vulnerabilities."""
# The workflow should be configured to fail on scan failure
assert "docker" in context.workflow_content.lower(), \
assert "docker" in context.workflow_content.lower(), (
"Docker job not found in workflow"
)
@then("the failure should block merge to master")
def step_failure_blocks_merge(context):
"""Verify that job failure blocks merge."""
# This is enforced by branch protection rules
assert "docker" in context.workflow_content.lower(), \
assert "docker" in context.workflow_content.lower(), (
"Docker job not configured as required check"
)
@given("a Dockerfile.server image with no HIGH or CRITICAL severity vulnerabilities")
@@ -143,21 +162,20 @@ def step_image_without_vulnerabilities(context):
def step_scan_exits_zero(context):
"""Verify that scan exits with zero status on clean image."""
# Trivy exits 0 when no HIGH/CRITICAL vulnerabilities found
assert "trivy" in context.workflow_content.lower(), \
"Trivy not configured"
assert "trivy" in context.workflow_content.lower(), "Trivy not configured"
@then("the CI job should pass")
def step_ci_job_passes(context):
"""Verify that the CI job passes on clean image."""
# The workflow should allow the job to pass
assert "docker" in context.workflow_content.lower(), \
assert "docker" in context.workflow_content.lower(), (
"Docker job not found in workflow"
)
@then("the build can proceed to the next stage")
def step_build_proceeds(context):
"""Verify that the build can proceed after passing scan."""
# The workflow should have subsequent jobs that depend on docker job
assert "needs:" in context.workflow_content, \
"Job dependencies not configured"
assert "needs:" in context.workflow_content, "Job dependencies not configured"
@@ -0,0 +1,46 @@
*** Settings ***
Documentation Integration tests for CI Dockerfile.server security scanning
... Validates that the CI workflow includes the Trivy security scan step
... configured to fail on HIGH/CRITICAL vulnerabilities (issue #1927).
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
*** Test Cases ***
CI Workflow Contains Trivy Security Scan Step
[Documentation] Verify the docker CI job includes the Trivy security scan step
[Tags] security ci dockerfile
${content}= Get File ${WORKSPACE}/.forgejo/workflows/ci.yml
Should Contain ${content} Security scan Dockerfile.server image with Trivy
CI Workflow Targets Dockerfile Server Image
[Documentation] Verify the CI scan targets the Dockerfile.server image specifically
[Tags] security ci dockerfile
${content}= Get File ${WORKSPACE}/.forgejo/workflows/ci.yml
Should Contain ${content} Dockerfile.server
Should Contain ${content} cleveragents-server:test
CI Trivy Configured To Fail On High Severity
[Documentation] Verify Trivy exits non-zero on HIGH or CRITICAL findings
[Tags] security ci dockerfile
${content}= Get File ${WORKSPACE}/.forgejo/workflows/ci.yml
Should Contain ${content} --severity HIGH,CRITICAL
Should Contain ${content} --exit-code 1
CI Trivy Uses Pinned Version
[Documentation] Verify Trivy installation is pinned to a specific version
[Tags] security ci dockerfile
${content}= Get File ${WORKSPACE}/.forgejo/workflows/ci.yml
Should Contain ${content} TRIVY_VERSION=
CI Trivy Install Verifies Checksum
[Documentation] Verify Trivy install downloads and validates the checksum file
[Tags] security ci dockerfile
${content}= Get File ${WORKSPACE}/.forgejo/workflows/ci.yml
Should Contain ${content} checksums.txt
Should Contain ${content} sha256sum -c
Dockerfile Server Exists In Repository
[Documentation] Verify Dockerfile.server exists at the expected repository location
[Tags] security dockerfile
File Should Exist ${WORKSPACE}/Dockerfile.server