TEST-INFRA: [ci-pipeline-design] Redundant Nightly Quality Job #2221

Open
opened 2026-04-03 09:37:28 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: task/ci-pipeline-design-redundant-nightly-quality-job
  • Commit Message: refactor(ci): remove redundant checks from nightly-quality workflow and add extensive tests
  • Milestone: v3.6.0
  • Parent Epic: #376

Background

The nightly-quality.yml workflow appears to run the same checks as the ci.yml workflow. This is redundant. The nightly build should be used for more extensive, long-running tests that are not practical to run on every PR, such as performance benchmarks or extensive integration tests. The current setup just re-runs the same quality checks.

Recommendation:

  • Remove the redundant checks from the nightly-quality.yml workflow.
  • Add more extensive tests to the nightly build, such as performance benchmarks (using ASV), end-to-end tests, or tests against a wider range of dependencies.

Subtasks

  • Audit nightly-quality.yml and ci.yml to identify all duplicated steps/jobs
  • Remove duplicated quality checks from nightly-quality.yml
  • Add ASV performance benchmark job to nightly-quality.yml
  • Add end-to-end / extended integration test job to nightly-quality.yml
  • Add multi-dependency-version matrix testing to nightly-quality.yml
  • Verify ci.yml remains unchanged and still covers all standard quality gates
  • Update CI documentation / comments to clarify the purpose of each workflow

Definition of Done

  • nightly-quality.yml contains no steps that duplicate those already in ci.yml
  • nightly-quality.yml includes at least one long-running or extended test not present in ci.yml (e.g., ASV benchmarks, extended integration tests, or dependency matrix)
  • Both workflows pass successfully in CI
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-pipeline-design-redundant-nightly-quality-job` - **Commit Message**: `refactor(ci): remove redundant checks from nightly-quality workflow and add extensive tests` - **Milestone**: v3.6.0 - **Parent Epic**: #376 ## Background The `nightly-quality.yml` workflow appears to run the same checks as the `ci.yml` workflow. This is redundant. The nightly build should be used for more extensive, long-running tests that are not practical to run on every PR, such as performance benchmarks or extensive integration tests. The current setup just re-runs the same quality checks. **Recommendation:** - Remove the redundant checks from the `nightly-quality.yml` workflow. - Add more extensive tests to the nightly build, such as performance benchmarks (using ASV), end-to-end tests, or tests against a wider range of dependencies. ## Subtasks - [ ] Audit `nightly-quality.yml` and `ci.yml` to identify all duplicated steps/jobs - [ ] Remove duplicated quality checks from `nightly-quality.yml` - [ ] Add ASV performance benchmark job to `nightly-quality.yml` - [ ] Add end-to-end / extended integration test job to `nightly-quality.yml` - [ ] Add multi-dependency-version matrix testing to `nightly-quality.yml` - [ ] Verify `ci.yml` remains unchanged and still covers all standard quality gates - [ ] Update CI documentation / comments to clarify the purpose of each workflow ## Definition of Done - [ ] `nightly-quality.yml` contains no steps that duplicate those already in `ci.yml` - [ ] `nightly-quality.yml` includes at least one long-running or extended test not present in `ci.yml` (e.g., ASV benchmarks, extended integration tests, or dependency matrix) - [ ] Both workflows pass successfully in CI - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.6.0 milestone 2026-04-03 09:37:38 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium (unchanged)
  • Milestone: v3.6.0 (already assigned)
  • MoSCoW: Could Have — Removing a redundant nightly quality job is a CI optimization, not a functional requirement. The spec does not mandate any specific CI pipeline structure. This is desirable for efficiency but not essential for the v3.6.0 milestone.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified ✅ - **Priority**: Medium (unchanged) - **Milestone**: v3.6.0 (already assigned) - **MoSCoW**: Could Have — Removing a redundant nightly quality job is a CI optimization, not a functional requirement. The spec does not mandate any specific CI pipeline structure. This is desirable for efficiency but not essential for the v3.6.0 milestone. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium
  • Milestone: v3.6.0
  • MoSCoW: Could Have — removing a redundant nightly quality job is a CI optimization, not blocking any functional work

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium - **Milestone**: v3.6.0 - **MoSCoW**: Could Have — removing a redundant nightly quality job is a CI optimization, not blocking any functional work --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
Reference
cleveragents/cleveragents-core#2221
No description provided.