[AUTO-INF-5] Add pre-release quality gate to release.yml before artifact publishing #10233

New issue

Open

opened 2026-04-17 09:10:49 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-17 09:10:49 +00:00

Owner

Summary

release.yml triggers on tag pushes (v*) and immediately starts building and publishing artifacts — with no quality checks (lint, typecheck, security scan, or tests) before the release is created. A broken or untested commit could be tagged and published to users.

Problem

The current release.yml job graph is:

build-wheel  →  build-docker  →  create-release

There are no quality gate jobs. The workflow goes directly from checkout to nox -s build and then publishes the wheel and Docker image to the registry and creates a Forgejo release — all without running:

Lint / format checks
Type checking (Pyright)
Security scan (Bandit)
Unit tests (Behave)
Integration or E2E tests

This means a developer can push a v* tag on any commit — including one that has never passed CI — and a broken release artifact will be published.

Proposed Fix

Add a quality-gate job at the start of release.yml that runs the essential checks before any build or publish step:

jobs:
    quality-gate:
        runs-on: docker
        container:
            image: python:3.13-slim
        steps:
            - uses: actions/checkout@v4
            - name: Install uv and nox
              run: pip install -q uv==0.8.0 nox
            - name: Run lint
              run: nox -s lint
            - name: Run typecheck
              run: nox -s typecheck
            - name: Run security scan
              run: nox -s security_scan
            - name: Run unit tests
              run: nox -s unit_tests

    build-wheel:
        needs: [quality-gate]   # ← gate added
        ...

Alternatively, enforce that the tag can only be pushed after ci.yml has passed (branch protection + required status checks).

Impact

Prevents publishing broken release artifacts to users
Ensures every published version has passed at minimum lint, typecheck, security, and unit tests
Aligns release pipeline with the quality standards enforced on PRs by ci.yml

Duplicate Check

The following existing issues were reviewed before filing this issue:

#10197: Add job-level timeout-minutes to all CI jobs in ci.yml — about timeouts, not quality gates
#9953: Nightly quality workflow omits integration_tests and e2e_tests — about the nightly workflow, not release
#9951: Fix rentention-days typo in master.yml benchmark jobs — about a typo
#9943: Add job-level timeout-minutes to all CI jobs in ci.yml — about timeouts
#9890: Improve Docker caching, template DB reuse, and release SBOMs — about Docker caching and SBOMs in release, not quality gates before publishing
#9778: Stabilize Behave/Robot test layers to cut CI flake — about test stability
#9697: Harden CI quality gates — about coverage parity and nightly enforcement in ci.yml, not release.yml
#9128: CI Pipeline Design: Guard integration/e2e jobs when LLM secrets unavailable — about secret-gating in ci.yml
#9767: Harden CI workflow reliability — about runner setup and secret-dependent jobs
#10067: Add missing job dependencies in CI pipeline — about missing needs: between existing jobs in ci.yml, not about adding a quality gate to release.yml
#10068: Add timeout configurations and path filters to CI workflows — about timeouts and path filters

None of the above issues address the missing pre-release quality gate in release.yml. This is a new, distinct finding.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

## Summary `release.yml` triggers on tag pushes (`v*`) and immediately starts building and publishing artifacts — with **no quality checks** (lint, typecheck, security scan, or tests) before the release is created. A broken or untested commit could be tagged and published to users. ## Problem The current `release.yml` job graph is: ``` build-wheel → build-docker → create-release ``` There are no quality gate jobs. The workflow goes directly from checkout to `nox -s build` and then publishes the wheel and Docker image to the registry and creates a Forgejo release — all without running: - Lint / format checks - Type checking (Pyright) - Security scan (Bandit) - Unit tests (Behave) - Integration or E2E tests This means a developer can push a `v*` tag on any commit — including one that has never passed CI — and a broken release artifact will be published. ## Proposed Fix Add a `quality-gate` job at the start of `release.yml` that runs the essential checks before any build or publish step: ```yaml jobs: quality-gate: runs-on: docker container: image: python:3.13-slim steps: - uses: actions/checkout@v4 - name: Install uv and nox run: pip install -q uv==0.8.0 nox - name: Run lint run: nox -s lint - name: Run typecheck run: nox -s typecheck - name: Run security scan run: nox -s security_scan - name: Run unit tests run: nox -s unit_tests build-wheel: needs: [quality-gate] # ← gate added ... ``` Alternatively, enforce that the tag can only be pushed after `ci.yml` has passed (branch protection + required status checks). ## Impact - Prevents publishing broken release artifacts to users - Ensures every published version has passed at minimum lint, typecheck, security, and unit tests - Aligns release pipeline with the quality standards enforced on PRs by `ci.yml` ### Duplicate Check The following existing issues were reviewed before filing this issue: - #10197: Add job-level timeout-minutes to all CI jobs in ci.yml — about timeouts, not quality gates - #9953: Nightly quality workflow omits integration_tests and e2e_tests — about the nightly workflow, not release - #9951: Fix rentention-days typo in master.yml benchmark jobs — about a typo - #9943: Add job-level timeout-minutes to all CI jobs in ci.yml — about timeouts - #9890: Improve Docker caching, template DB reuse, and release SBOMs — about Docker caching and SBOMs in release, not quality gates before publishing - #9778: Stabilize Behave/Robot test layers to cut CI flake — about test stability - #9697: Harden CI quality gates — about coverage parity and nightly enforcement in ci.yml, not release.yml - #9128: CI Pipeline Design: Guard integration/e2e jobs when LLM secrets unavailable — about secret-gating in ci.yml - #9767: Harden CI workflow reliability — about runner setup and secret-dependent jobs - #10067: Add missing job dependencies in CI pipeline — about missing `needs:` between existing jobs in ci.yml, not about adding a quality gate to release.yml - #10068: Add timeout configurations and path filters to CI workflows — about timeouts and path filters **None of the above issues address the missing pre-release quality gate in release.yml.** This is a new, distinct finding. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

HAL9000 added the

labels

2026-04-17 09:13:36 +00:00

HAL9000 referenced this issue

2026-04-17 11:22:32 +00:00

[AUTO-INF-POOL] Status: Test Infrastructure Pool (Cycle 4) #10206