[AUTO-INF-5] `release.yml` has no pre-release quality gate — broken releases can be published without running any tests or quality checks #10241

New issue

Open

opened 2026-04-17 10:12:49 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-17 10:12:49 +00:00

Owner

Metadata

Commit message: fix(ci): add pre-release quality gate job to release.yml before build and publish
Branch name: fix/auto-inf-5-release-pre-quality-gate

Background and Context

The release workflow (.forgejo/workflows/release.yml) triggers on tag push and immediately proceeds to build and publish artifacts (wheel + Docker image) without running any tests or quality checks. There is no gate to prevent publishing a broken release.

Current state in release.yml:

on:
    push:
        tags:
            - 'v*'

jobs:
    build-wheel:
        # Immediately builds and publishes — no quality gate
        runs-on: docker
        steps:
            - uses: actions/checkout@v4
            - name: Build wheel
              run: ...
            - name: Publish to PyPI
              run: ...

This means:

A tag pushed to code that fails lint will still publish a broken wheel to PyPI
A tag pushed to code that fails unit tests will still publish a broken Docker image
There is no automated safety net between "tag created" and "artifact published"
Downstream consumers of the package/image have no guarantee that the release passed basic quality checks

This is a supply-chain reliability gap: the CI pipeline enforces quality gates on PRs but not on the final release artifact publication step.

Expected Behavior

The release workflow should run a pre-release validation job before any build or publish steps:

jobs:
    pre-release-check:
        runs-on: docker
        container:
            image: python:3.13-slim
        steps:
            - uses: actions/checkout@v4
            - name: Install uv and nox
              run: pip install -q uv==0.8.0 nox
            - name: Run lint
              run: nox -s lint
            - name: Run typecheck
              run: nox -s typecheck
            - name: Run unit tests
              run: nox -s unit_tests

    build-wheel:
        needs: [pre-release-check]  # Gate on quality check passing
        # ... existing steps unchanged

    build-docker:
        needs: [pre-release-check]  # Gate on quality check passing
        # ... existing steps unchanged

This ensures that no release artifact is published unless the code passes lint, typecheck, and unit tests.

Acceptance Criteria

A pre-release-check job exists in .forgejo/workflows/release.yml that runs at minimum: lint, typecheck, and unit_tests nox sessions
All build/publish jobs in release.yml declare needs: [pre-release-check]
A release tag pushed to code that fails lint causes the build/publish jobs to be skipped
A release tag pushed to code that fails unit tests causes the build/publish jobs to be skipped
A release tag pushed to passing code still results in artifacts being published correctly
The pre-release-check job uses the same Python version (3.13) as the main CI pipeline
Failure of pre-release-check produces a clear error message indicating which check failed

Subtasks

Audit .forgejo/workflows/release.yml to identify all build and publish jobs
Add a pre-release-check job that runs nox -s lint, nox -s typecheck, and nox -s unit_tests
Add needs: [pre-release-check] to all build and publish jobs in release.yml
Verify the pre-release-check job uses the correct Python version and uv/nox versions consistent with ci.yml
Test the fix by creating a test tag on a branch with a deliberate lint failure and confirming publish is blocked
Consider whether security and quality nox sessions should also be included in pre-release-check
Update CONTRIBUTING.md or docs/development/ci-cd.md to document the pre-release quality gate behavior

Definition of Done

This issue should be closed when:

A pre-release-check job exists in .forgejo/workflows/release.yml and runs lint, typecheck, and unit tests
All build and publish jobs in release.yml depend on pre-release-check passing
A failing quality check provably blocks artifact publication
The fix is merged to the main branch via a PR that passes all required CI checks
Documentation reflects the new pre-release quality gate behavior

Duplicate Check

Check	Query / Source	Result
Check 1	Open issues pages 1–7 searched for `pre-release`, `release quality`, `release gate`, `release.yml`	No existing issue addresses running tests/quality checks before release publication
Check 2	Closed issues pages 1–6 searched for same keywords	No matches found
Check 3	Cross-area: #9890 "Improve Docker caching, template DB reuse, and release SBOMs"	Covers release.yml for SBOM generation, vulnerability scanning, and signing — does NOT address running tests/lint before publish
Check 4	Cross-area: #9783 "Reduce CI execution time"	Mentions `release.yml` only in context of pre-baked CI runner image — does not address quality gates before release
Check 5	Known issues list: all 9 known existing issues reviewed	None address pre-release quality gates in `release.yml`

Conclusion: No duplicate found. This is a genuinely new finding. The closest existing issue (#9890) covers release supply-chain security (SBOMs, signing) but not the fundamental gap of running tests before publishing.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

## Metadata - **Commit message:** `fix(ci): add pre-release quality gate job to release.yml before build and publish` - **Branch name:** `fix/auto-inf-5-release-pre-quality-gate` ## Background and Context The release workflow (`.forgejo/workflows/release.yml`) triggers on tag push and immediately proceeds to build and publish artifacts (wheel + Docker image) without running any tests or quality checks. There is no gate to prevent publishing a broken release. **Current state in release.yml:** ```yaml on: push: tags: - 'v*' jobs: build-wheel: # Immediately builds and publishes — no quality gate runs-on: docker steps: - uses: actions/checkout@v4 - name: Build wheel run: ... - name: Publish to PyPI run: ... ``` This means: - A tag pushed to code that fails lint will still publish a broken wheel to PyPI - A tag pushed to code that fails unit tests will still publish a broken Docker image - There is no automated safety net between "tag created" and "artifact published" - Downstream consumers of the package/image have no guarantee that the release passed basic quality checks This is a supply-chain reliability gap: the CI pipeline enforces quality gates on PRs but not on the final release artifact publication step. ## Expected Behavior The release workflow should run a pre-release validation job before any build or publish steps: ```yaml jobs: pre-release-check: runs-on: docker container: image: python:3.13-slim steps: - uses: actions/checkout@v4 - name: Install uv and nox run: pip install -q uv==0.8.0 nox - name: Run lint run: nox -s lint - name: Run typecheck run: nox -s typecheck - name: Run unit tests run: nox -s unit_tests build-wheel: needs: [pre-release-check] # Gate on quality check passing # ... existing steps unchanged build-docker: needs: [pre-release-check] # Gate on quality check passing # ... existing steps unchanged ``` This ensures that no release artifact is published unless the code passes lint, typecheck, and unit tests. ## Acceptance Criteria - [ ] A `pre-release-check` job exists in `.forgejo/workflows/release.yml` that runs at minimum: `lint`, `typecheck`, and `unit_tests` nox sessions - [ ] All build/publish jobs in `release.yml` declare `needs: [pre-release-check]` - [ ] A release tag pushed to code that fails lint causes the build/publish jobs to be skipped - [ ] A release tag pushed to code that fails unit tests causes the build/publish jobs to be skipped - [ ] A release tag pushed to passing code still results in artifacts being published correctly - [ ] The `pre-release-check` job uses the same Python version (3.13) as the main CI pipeline - [ ] Failure of `pre-release-check` produces a clear error message indicating which check failed ## Subtasks - [ ] Audit `.forgejo/workflows/release.yml` to identify all build and publish jobs - [ ] Add a `pre-release-check` job that runs `nox -s lint`, `nox -s typecheck`, and `nox -s unit_tests` - [ ] Add `needs: [pre-release-check]` to all build and publish jobs in `release.yml` - [ ] Verify the `pre-release-check` job uses the correct Python version and uv/nox versions consistent with `ci.yml` - [ ] Test the fix by creating a test tag on a branch with a deliberate lint failure and confirming publish is blocked - [ ] Consider whether `security` and `quality` nox sessions should also be included in `pre-release-check` - [ ] Update `CONTRIBUTING.md` or `docs/development/ci-cd.md` to document the pre-release quality gate behavior ## Definition of Done This issue should be closed when: 1. A `pre-release-check` job exists in `.forgejo/workflows/release.yml` and runs lint, typecheck, and unit tests 2. All build and publish jobs in `release.yml` depend on `pre-release-check` passing 3. A failing quality check provably blocks artifact publication 4. The fix is merged to the main branch via a PR that passes all required CI checks 5. Documentation reflects the new pre-release quality gate behavior ### Duplicate Check | Check | Query / Source | Result | |-------|---------------|--------| | Check 1 | Open issues pages 1–7 searched for `pre-release`, `release quality`, `release gate`, `release.yml` | No existing issue addresses running tests/quality checks before release publication | | Check 2 | Closed issues pages 1–6 searched for same keywords | No matches found | | Check 3 | Cross-area: #9890 "Improve Docker caching, template DB reuse, and release SBOMs" | Covers release.yml for SBOM generation, vulnerability scanning, and signing — does NOT address running tests/lint before publish | | Check 4 | Cross-area: #9783 "Reduce CI execution time" | Mentions `release.yml` only in context of pre-baked CI runner image — does not address quality gates before release | | Check 5 | Known issues list: all 9 known existing issues reviewed | None address pre-release quality gates in `release.yml` | **Conclusion:** No duplicate found. This is a genuinely new finding. The closest existing issue (#9890) covers release supply-chain security (SBOMs, signing) but not the fundamental gap of running tests before publishing. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

HAL9000 added the

labels

2026-04-17 10:27:51 +00:00

HAL9000 referenced this issue

2026-04-17 11:22:32 +00:00

[AUTO-INF-POOL] Status: Test Infrastructure Pool (Cycle 4) #10206