TEST-INFRA: [ci-execution-time] Optimize coverage report generation to avoid a separate sequential test run #2299

Open
opened 2026-04-03 13:25:15 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-coverage-report-no-sequential-rerun
  • Commit Message: perf(ci): eliminate sequential test re-run from coverage_report nox session
  • Milestone: v3.8.0
  • Parent Epic: #1678

Description

The coverage_report nox session currently re-runs all unit tests in a sequential manner to generate a code coverage report. This is inefficient because the unit_tests session already runs these tests in parallel. A separate, sequential run for coverage significantly increases the total CI execution time.

Subtasks

  • Investigate methods to collect coverage data from the parallel unit_tests session. This may involve:
    • Configuring slipcover or coverage.py to work with the behave-parallel runner.
    • Generating per-worker coverage files and merging them after the test run.
  • Modify the unit_tests session to generate coverage data.
  • Modify the coverage_report session to only generate the report from the existing data, without re-running the tests.
  • Ensure the generated coverage report is accurate and complete.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

  • The separate, sequential test run in the coverage_report session is eliminated.
  • Coverage data is successfully collected from the parallel unit_tests session.
  • The coverage_report session only generates the report, without re-running tests.
  • The overall CI execution time is reduced without compromising the quality of the coverage report.
  • All nox stages pass.
  • Coverage >= 97%.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-coverage-report-no-sequential-rerun` - **Commit Message**: `perf(ci): eliminate sequential test re-run from coverage_report nox session` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The `coverage_report` nox session currently re-runs all unit tests in a sequential manner to generate a code coverage report. This is inefficient because the `unit_tests` session already runs these tests in parallel. A separate, sequential run for coverage significantly increases the total CI execution time. ## Subtasks - [ ] Investigate methods to collect coverage data from the parallel `unit_tests` session. This may involve: - Configuring `slipcover` or `coverage.py` to work with the `behave-parallel` runner. - Generating per-worker coverage files and merging them after the test run. - [ ] Modify the `unit_tests` session to generate coverage data. - [ ] Modify the `coverage_report` session to only generate the report from the existing data, without re-running the tests. - [ ] Ensure the generated coverage report is accurate and complete. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done - [ ] The separate, sequential test run in the `coverage_report` session is eliminated. - [ ] Coverage data is successfully collected from the parallel `unit_tests` session. - [ ] The `coverage_report` session only generates the report, without re-running tests. - [ ] The overall CI execution time is reduced without compromising the quality of the coverage report. - [ ] All nox stages pass. - [ ] Coverage >= 97%. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 13:25:24 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium (already set, correct)
  • Milestone: v3.8.0 (correct)
  • MoSCoW: Could Have — Coverage report optimization is a nice-to-have, not blocking milestone delivery.
  • Parent Epic: #1678

This is the canonical issue for coverage report optimization. Duplicates #2286, #2340 have been closed and consolidated here.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified ✅ - **Priority**: Medium (already set, correct) - **Milestone**: v3.8.0 (correct) - **MoSCoW**: Could Have — Coverage report optimization is a nice-to-have, not blocking milestone delivery. - **Parent Epic**: #1678 This is the canonical issue for coverage report optimization. Duplicates #2286, #2340 have been closed and consolidated here. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2299
No description provided.