TEST-INFRA: [ci-execution-time] Share .coverage artifact from unit_tests to coverage job to eliminate duplicate test execution #1704

New issue

Closed

opened 2026-04-02 23:31:32 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:31:32 +00:00

Owner

Metadata

Branch: fix/ci-coverage-artifact-sharing
Commit Message: fix(ci): share .coverage artifact between unit_tests and coverage jobs to avoid re-running tests
Milestone: v3.8.0
Parent Epic: #1678

Problem

The coverage job in .forgejo/workflows/ci.yml runs nox -s coverage_report, which re-executes the entire test suite in order to generate coverage data. The unit_tests job already ran the same tests earlier in the same CI run.

This means every CI run executes the full Behave test suite twice: once in unit_tests and once in coverage. Since both jobs run independently (no needs: relationship between them), they also run in parallel, doubling the runner load for test execution.

Estimated waste: If the unit test suite takes ~10 minutes, the coverage job wastes ~10 minutes of runner time per CI run re-running tests that already passed.

Solution

In unit_tests: Run nox -s unit_tests with coverage enabled (set COVERAGE_FILE env var or pass --cov flags), then upload the resulting .coverage file as an artifact.
In coverage: Add needs: [unit_tests], download the .coverage artifact, then run only the report generation step (e.g., nox -s coverage_report -- --no-rerun or a dedicated nox -s coverage_report_only session that skips test execution and only calls coverage report/coverage xml).
In noxfile.py: Add a coverage_report_only nox session (or a --no-rerun flag to the existing coverage_report session) that generates the HTML/XML/JSON reports from an existing .coverage file without re-running tests.

Artifact configuration

# In unit_tests job — after running tests:
- name: Upload coverage data
  uses: actions/upload-artifact@v3
  with:
    name: coverage-data
    path: .coverage
    retention-days: 1

# In coverage job:
needs: [lint, typecheck, unit_tests]
steps:
  - name: Download coverage data
    uses: actions/download-artifact@v3
    with:
      name: coverage-data

Subtasks

Add a coverage_report_only nox session (or --no-rerun flag) to noxfile.py that generates reports from an existing .coverage file
Update unit_tests job to upload .coverage as an artifact after test run
Update coverage job to add needs: [unit_tests], download the artifact, and call the report-only session
Add a Behave scenario in features/ci_workflow_validation.feature asserting the artifact upload/download steps are present
Verify nox -s unit_tests still passes
Verify nox -s coverage_report still passes end-to-end
Verify all nox stages pass

Definition of Done

noxfile.py has a session that generates coverage reports from an existing .coverage file without re-running tests
unit_tests CI job uploads .coverage as an artifact
coverage CI job downloads the artifact and uses it instead of re-running tests
coverage CI job has needs: [unit_tests] (in addition to existing needs: [lint, typecheck])
features/ci_workflow_validation.feature asserts the artifact sharing steps are present
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/ci-coverage-artifact-sharing` - **Commit Message**: `fix(ci): share .coverage artifact between unit_tests and coverage jobs to avoid re-running tests` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Problem The `coverage` job in `.forgejo/workflows/ci.yml` runs `nox -s coverage_report`, which re-executes the entire test suite in order to generate coverage data. The `unit_tests` job already ran the same tests earlier in the same CI run. This means every CI run executes the full Behave test suite **twice**: once in `unit_tests` and once in `coverage`. Since both jobs run independently (no `needs:` relationship between them), they also run in parallel, doubling the runner load for test execution. **Estimated waste**: If the unit test suite takes ~10 minutes, the `coverage` job wastes ~10 minutes of runner time per CI run re-running tests that already passed. ## Solution 1. **In `unit_tests`**: Run `nox -s unit_tests` with coverage enabled (set `COVERAGE_FILE` env var or pass `--cov` flags), then upload the resulting `.coverage` file as an artifact. 2. **In `coverage`**: Add `needs: [unit_tests]`, download the `.coverage` artifact, then run only the report generation step (e.g., `nox -s coverage_report -- --no-rerun` or a dedicated `nox -s coverage_report_only` session that skips test execution and only calls `coverage report`/`coverage xml`). 3. **In `noxfile.py`**: Add a `coverage_report_only` nox session (or a `--no-rerun` flag to the existing `coverage_report` session) that generates the HTML/XML/JSON reports from an existing `.coverage` file without re-running tests. ### Artifact configuration ```yaml # In unit_tests job — after running tests: - name: Upload coverage data uses: actions/upload-artifact@v3 with: name: coverage-data path: .coverage retention-days: 1 # In coverage job: needs: [lint, typecheck, unit_tests] steps: - name: Download coverage data uses: actions/download-artifact@v3 with: name: coverage-data ``` ## Subtasks - [ ] Add a `coverage_report_only` nox session (or `--no-rerun` flag) to `noxfile.py` that generates reports from an existing `.coverage` file - [ ] Update `unit_tests` job to upload `.coverage` as an artifact after test run - [ ] Update `coverage` job to add `needs: [unit_tests]`, download the artifact, and call the report-only session - [ ] Add a Behave scenario in `features/ci_workflow_validation.feature` asserting the artifact upload/download steps are present - [ ] Verify `nox -s unit_tests` still passes - [ ] Verify `nox -s coverage_report` still passes end-to-end - [ ] Verify all nox stages pass ## Definition of Done - [ ] `noxfile.py` has a session that generates coverage reports from an existing `.coverage` file without re-running tests - [ ] `unit_tests` CI job uploads `.coverage` as an artifact - [ ] `coverage` CI job downloads the artifact and uses it instead of re-running tests - [ ] `coverage` CI job has `needs: [unit_tests]` (in addition to existing `needs: [lint, typecheck]`) - [ ] `features/ci_workflow_validation.feature` asserts the artifact sharing steps are present - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-02 23:32:45 +00:00

freemo added a new dependency

2026-04-02 23:36:09 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-02 23:41:39 +00:00

Author

Owner

Closing as duplicate of #1589 (dependency caching improvement).

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #1589 (dependency caching improvement). --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo

2026-04-02 23:41:40 +00:00