cleveragents/cleveragents-core

Fork 3

UAT: `coverage` CI job missing dependency on `unit_tests` — coverage can run when tests are failing #3887

New issue

Open

opened 2026-04-06 07:09:07 +00:00 by freemo · 1 comment

freemo commented

2026-04-06 07:09:07 +00:00

Owner

Metadata

Branch: fix/ci-coverage-missing-unit-tests-dep
Commit Message: fix(ci): add unit_tests to coverage job needs to prevent coverage running on test failure
Milestone: None (backlog — see note below)
Parent Epic: #1678

Backlog note: This issue was discovered during autonomous operation
on milestone v3.7.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Background and Context

In .forgejo/workflows/ci.yml, the coverage job's needs: declaration is missing unit_tests. This means the coverage job can be triggered and run in parallel with — or even after the failure of — the unit_tests job. Coverage data is only meaningful when the full test suite passes; running coverage when tests are failing wastes CI resources and can produce misleading coverage reports.

Current Behavior

The coverage job in .forgejo/workflows/ci.yml declares:

coverage:
    needs: [lint, typecheck, security, quality]

unit_tests is absent from the needs: list. This means:

The coverage job can start running in parallel with unit_tests
If unit_tests fails (tests are broken), coverage will still run the full Behave test suite under slipcover
The coverage job may produce a passing result even when unit_tests fails (if the test failures are non-critical or if slipcover's success_codes=[0, 1] masks failures)
CI resources are wasted running coverage when tests are already known to fail

Code Location: .forgejo/workflows/ci.yml — coverage job, needs: declaration

Steps to Reproduce:

Open .forgejo/workflows/ci.yml
Find the coverage job
Observe needs: [lint, typecheck, security, quality] — unit_tests is missing
Compare with the status-check job which correctly requires both unit_tests and coverage to succeed

Expected Behavior

The coverage CI job should only run after unit_tests has successfully completed. Coverage data is only meaningful when the test suite passes.

Suggested Fix: Add unit_tests to the coverage job's needs: list:

coverage:
    needs: [lint, typecheck, security, quality, unit_tests]

Acceptance Criteria

The coverage job in .forgejo/workflows/ci.yml lists unit_tests in its needs: declaration
The coverage job does not run when unit_tests fails
The status-check job continues to require both unit_tests and coverage
features/ci_workflow_validation.feature is updated to assert the unit_tests dependency on the coverage job

Supporting Information

Impact: Medium — The status-check job still requires both unit_tests and coverage to succeed, so the overall CI will fail if either fails. However, CI resources are wasted running coverage when tests are already failing, and there is a risk of misleading coverage results.
What Was Tested: Code-level analysis of .forgejo/workflows/ci.yml — the coverage job's needs: declaration.
Related Epic: #1678 (CI Execution Time Optimization) — that Epic also discusses the coverage job's relationship with unit_tests and artifact sharing.

Subtasks

Add unit_tests to the coverage job's needs: list in .forgejo/workflows/ci.yml
Update features/ci_workflow_validation.feature to assert unit_tests is in the coverage job's needs: list
Run nox (all default sessions), fix any errors
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/ci-coverage-missing-unit-tests-dep` - **Commit Message**: `fix(ci): add unit_tests to coverage job needs to prevent coverage running on test failure` - **Milestone**: None (backlog — see note below) - **Parent Epic**: #1678 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.7.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Background and Context In `.forgejo/workflows/ci.yml`, the `coverage` job's `needs:` declaration is missing `unit_tests`. This means the `coverage` job can be triggered and run in parallel with — or even after the failure of — the `unit_tests` job. Coverage data is only meaningful when the full test suite passes; running coverage when tests are failing wastes CI resources and can produce misleading coverage reports. ## Current Behavior The `coverage` job in `.forgejo/workflows/ci.yml` declares: ```yaml coverage: needs: [lint, typecheck, security, quality] ``` `unit_tests` is absent from the `needs:` list. This means: 1. The `coverage` job can start running in parallel with `unit_tests` 2. If `unit_tests` fails (tests are broken), `coverage` will still run the full Behave test suite under slipcover 3. The `coverage` job may produce a passing result even when `unit_tests` fails (if the test failures are non-critical or if slipcover's `success_codes=[0, 1]` masks failures) 4. CI resources are wasted running coverage when tests are already known to fail **Code Location:** `.forgejo/workflows/ci.yml` — `coverage` job, `needs:` declaration **Steps to Reproduce:** 1. Open `.forgejo/workflows/ci.yml` 2. Find the `coverage` job 3. Observe `needs: [lint, typecheck, security, quality]` — `unit_tests` is missing 4. Compare with the `status-check` job which correctly requires both `unit_tests` and `coverage` to succeed ## Expected Behavior The `coverage` CI job should only run after `unit_tests` has successfully completed. Coverage data is only meaningful when the test suite passes. **Suggested Fix:** Add `unit_tests` to the `coverage` job's `needs:` list: ```yaml coverage: needs: [lint, typecheck, security, quality, unit_tests] ``` ## Acceptance Criteria - [ ] The `coverage` job in `.forgejo/workflows/ci.yml` lists `unit_tests` in its `needs:` declaration - [ ] The `coverage` job does not run when `unit_tests` fails - [ ] The `status-check` job continues to require both `unit_tests` and `coverage` - [ ] `features/ci_workflow_validation.feature` is updated to assert the `unit_tests` dependency on the `coverage` job ## Supporting Information - **Impact:** Medium — The `status-check` job still requires both `unit_tests` and `coverage` to succeed, so the overall CI will fail if either fails. However, CI resources are wasted running coverage when tests are already failing, and there is a risk of misleading coverage results. - **What Was Tested:** Code-level analysis of `.forgejo/workflows/ci.yml` — the `coverage` job's `needs:` declaration. - **Related Epic:** #1678 (CI Execution Time Optimization) — that Epic also discusses the `coverage` job's relationship with `unit_tests` and artifact sharing. ## Subtasks - [ ] Add `unit_tests` to the `coverage` job's `needs:` list in `.forgejo/workflows/ci.yml` - [ ] Update `features/ci_workflow_validation.feature` to assert `unit_tests` is in the `coverage` job's `needs:` list - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added a new dependency

2026-04-06 07:09:25 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo referenced this issue

2026-04-06 07:09:57 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:10:37 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo added the

Points

label

2026-04-06 07:15:12 +00:00

freemo referenced this issue

2026-04-06 07:16:14 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:21:26 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:22:18 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:28:07 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:33:38 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 10:35:53 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

HAL9000 commented

2026-04-08 14:03:27 +00:00

Owner

Label compliance fix applied:

Removed conflicting label: State/In Progress
Kept: State/Verified
Reason: Issue had two conflicting State/* labels. Per CONTRIBUTING.md, only one State/* label is allowed. Keeping the more advanced state (State/Verified).

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Removed conflicting label: `State/In Progress` - Kept: `State/Verified` - Reason: Issue had two conflicting State/* labels. Per CONTRIBUTING.md, only one State/* label is allowed. Keeping the more advanced state (`State/Verified`). --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer