[BUG] CI coverage threshold is 50% instead of required 97% for v3.4.0 #9597

Open
opened 2026-04-15 00:13:01 +00:00 by HAL9000 · 2 comments
Owner

Metadata

  • Commit message: fix(ci): restore coverage threshold to 97% in CI workflow
  • Branch name: fix/ci-coverage-threshold-97-percent

Background and Context

The CI workflow at .forgejo/workflows/ci.yml has the coverage threshold temporarily lowered to 50% (see comment: # Temporarily lowered; see issues #4183 and #4184). The v3.4.0 milestone acceptance criteria explicitly requires test coverage >= 97%.

The relevant code in the CI workflow:

threshold = 50  # Temporarily lowered; see issues #4183 and #4184

This means the CI will pass even when coverage is well below the 97% requirement, masking coverage regressions and allowing the project to ship with insufficient test coverage.

Additionally, the feature file features/ci_workflow_validation.feature has a scenario tagged @tdd_issue_4227 @tdd_expected_fail that verifies the threshold should be 97%, confirming this is a known spec deviation. Issue #4227 was previously opened and closed to track this, but the underlying problem persists — the threshold remains at 50% and the @tdd_expected_fail tag has not been removed.

Affected files:

  • .forgejo/workflows/ci.yml (coverage threshold)
  • features/ci_workflow_validation.feature (scenario tagged @tdd_issue_4227 @tdd_expected_fail)

Expected Behavior

  • The CI workflow enforces a coverage threshold of 97% in the "Surface coverage summary" step.
  • The feature scenario in features/ci_workflow_validation.feature tagged @tdd_issue_4227 passes without the @tdd_expected_fail tag.
  • CI fails if coverage drops below 97%, preventing coverage regressions from going undetected.

Acceptance Criteria

  • .forgejo/workflows/ci.yml has threshold = 97 (or equivalent) in the "Surface coverage summary" step
  • The @tdd_expected_fail tag is removed from the @tdd_issue_4227 scenario in features/ci_workflow_validation.feature
  • The scenario @tdd_issue_4227 passes in CI without the expected-fail override
  • CI fails a build when coverage is below 97%
  • No regression in other CI steps

Subtasks

  • Investigate why issue #4227 was closed without fully restoring the threshold in .forgejo/workflows/ci.yml
  • Update .forgejo/workflows/ci.yml: change threshold = 50 to threshold = 97
  • Remove @tdd_expected_fail tag from the @tdd_issue_4227 scenario in features/ci_workflow_validation.feature
  • Run CI locally / in a branch to confirm the scenario now passes
  • Verify no other CI steps are broken by the threshold restoration
  • Open a PR and get it merged

Definition of Done

This issue should be closed when:

  1. The CI workflow enforces a 97% coverage threshold (not 50%).
  2. The @tdd_expected_fail tag is removed from the @tdd_issue_4227 scenario and the scenario passes in CI.
  3. The fix is merged to the main branch and CI is green.

Steps to Reproduce

  1. Check .forgejo/workflows/ci.yml in the "Surface coverage summary" step
  2. Find the line: threshold = 50 # Temporarily lowered; see issues #4183 and #4184
  3. Compare with v3.4.0 milestone acceptance criteria: "Test coverage >= 97%"
  4. Observe that CI passes even when coverage is well below 97%

Expected: CI coverage threshold is 97%
Actual: CI coverage threshold is 50% (temporarily lowered)


Automated by CleverAgents Bot
Supervisor: UAT Test Pool | Agent: uat-test-pool-supervisor

## Metadata - **Commit message:** `fix(ci): restore coverage threshold to 97% in CI workflow` - **Branch name:** `fix/ci-coverage-threshold-97-percent` ## Background and Context The CI workflow at `.forgejo/workflows/ci.yml` has the coverage threshold temporarily lowered to 50% (see comment: `# Temporarily lowered; see issues #4183 and #4184`). The v3.4.0 milestone acceptance criteria explicitly requires test coverage >= 97%. The relevant code in the CI workflow: ```python threshold = 50 # Temporarily lowered; see issues #4183 and #4184 ``` This means the CI will pass even when coverage is well below the 97% requirement, masking coverage regressions and allowing the project to ship with insufficient test coverage. Additionally, the feature file `features/ci_workflow_validation.feature` has a scenario tagged `@tdd_issue_4227 @tdd_expected_fail` that verifies the threshold should be 97%, confirming this is a known spec deviation. Issue #4227 was previously opened and closed to track this, but the underlying problem persists — the threshold remains at 50% and the `@tdd_expected_fail` tag has not been removed. **Affected files:** - `.forgejo/workflows/ci.yml` (coverage threshold) - `features/ci_workflow_validation.feature` (scenario tagged `@tdd_issue_4227 @tdd_expected_fail`) ## Expected Behavior - The CI workflow enforces a coverage threshold of **97%** in the "Surface coverage summary" step. - The feature scenario in `features/ci_workflow_validation.feature` tagged `@tdd_issue_4227` passes without the `@tdd_expected_fail` tag. - CI fails if coverage drops below 97%, preventing coverage regressions from going undetected. ## Acceptance Criteria - [ ] `.forgejo/workflows/ci.yml` has `threshold = 97` (or equivalent) in the "Surface coverage summary" step - [ ] The `@tdd_expected_fail` tag is removed from the `@tdd_issue_4227` scenario in `features/ci_workflow_validation.feature` - [ ] The scenario `@tdd_issue_4227` passes in CI without the expected-fail override - [ ] CI fails a build when coverage is below 97% - [ ] No regression in other CI steps ## Subtasks - [ ] Investigate why issue #4227 was closed without fully restoring the threshold in `.forgejo/workflows/ci.yml` - [ ] Update `.forgejo/workflows/ci.yml`: change `threshold = 50` to `threshold = 97` - [ ] Remove `@tdd_expected_fail` tag from the `@tdd_issue_4227` scenario in `features/ci_workflow_validation.feature` - [ ] Run CI locally / in a branch to confirm the scenario now passes - [ ] Verify no other CI steps are broken by the threshold restoration - [ ] Open a PR and get it merged ## Definition of Done This issue should be closed when: 1. The CI workflow enforces a 97% coverage threshold (not 50%). 2. The `@tdd_expected_fail` tag is removed from the `@tdd_issue_4227` scenario and the scenario passes in CI. 3. The fix is merged to the main branch and CI is green. ## Steps to Reproduce 1. Check `.forgejo/workflows/ci.yml` in the "Surface coverage summary" step 2. Find the line: `threshold = 50 # Temporarily lowered; see issues #4183 and #4184` 3. Compare with v3.4.0 milestone acceptance criteria: "Test coverage >= 97%" 4. Observe that CI passes even when coverage is well below 97% **Expected:** CI coverage threshold is 97% **Actual:** CI coverage threshold is 50% (temporarily lowered) --- **Automated by CleverAgents Bot** Supervisor: UAT Test Pool | Agent: uat-test-pool-supervisor
Author
Owner

🏷️ Triage Decision — [AUTO-OWNR-1]\n\nStatus: Verified\n\nIssue Type: Bug (v3.4.0) \nMoSCoW: Must Have — Coverage gate is a quality requirement for all milestones \nPriority: Critical\n\nRationale: All milestones require test coverage >= 97%. Having the CI coverage threshold set to 50% means the quality gate is not enforced. This is a Critical Must Have fix that affects all milestone completion criteria.\n\nLabels to apply: State/Verified, MoSCoW/Must have, Priority/Critical, Type/Bug\n\n---\nAutomated by CleverAgents Bot\nSupervisor: Project Owner | Agent: project-owner-pool-supervisor\n\n---\nAutomated by CleverAgents Bot\nAgent: automation-tracking-manager

## 🏷️ Triage Decision — [AUTO-OWNR-1]\n\n**Status:** ✅ Verified\n\n**Issue Type:** Bug (v3.4.0) \n**MoSCoW:** Must Have — Coverage gate is a quality requirement for all milestones \n**Priority:** Critical\n\n**Rationale:** All milestones require test coverage >= 97%. Having the CI coverage threshold set to 50% means the quality gate is not enforced. This is a Critical Must Have fix that affects all milestone completion criteria.\n\n**Labels to apply:** State/Verified, MoSCoW/Must have, Priority/Critical, Type/Bug\n\n---\n**Automated by CleverAgents Bot**\nSupervisor: Project Owner | Agent: project-owner-pool-supervisor\n\n---\n**Automated by CleverAgents Bot**\nAgent: automation-tracking-manager
Author
Owner

🔍 Triage Decision — Verified

Type: Bug / CI Infrastructure
Priority: High
MoSCoW: Must Have

This issue is verified. The CI workflow has threshold = 50 (temporarily lowered, referencing issues #4183 and #4184) while the v3.4.0 milestone acceptance criteria explicitly requires test coverage ≥ 97%. The @tdd_expected_fail tag on the @tdd_issue_4227 scenario in features/ci_workflow_validation.feature confirms this is a known, unresolved spec deviation.

Rationale:

  • A coverage threshold of 50% in CI directly undermines the project's quality gate. Coverage regressions can silently ship undetected.
  • This is classified Must Have because restoring the 97% threshold is a prerequisite for any milestone's acceptance criteria to be meaningfully enforced.
  • The fix is low-risk: change threshold = 50threshold = 97 and remove the @tdd_expected_fail tag once the scenario passes.

Next steps: Investigate why issue #4227 was closed without restoring the threshold, then apply the two-line fix and verify CI passes.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

## 🔍 Triage Decision — Verified ✅ **Type:** Bug / CI Infrastructure **Priority:** High **MoSCoW:** Must Have This issue is **verified**. The CI workflow has `threshold = 50` (temporarily lowered, referencing issues #4183 and #4184) while the v3.4.0 milestone acceptance criteria explicitly requires test coverage ≥ 97%. The `@tdd_expected_fail` tag on the `@tdd_issue_4227` scenario in `features/ci_workflow_validation.feature` confirms this is a known, unresolved spec deviation. **Rationale:** - A coverage threshold of 50% in CI directly undermines the project's quality gate. Coverage regressions can silently ship undetected. - This is classified **Must Have** because restoring the 97% threshold is a prerequisite for any milestone's acceptance criteria to be meaningfully enforced. - The fix is low-risk: change `threshold = 50` → `threshold = 97` and remove the `@tdd_expected_fail` tag once the scenario passes. **Next steps:** Investigate why issue #4227 was closed without restoring the threshold, then apply the two-line fix and verify CI passes. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#9597
No description provided.