TEST-INFRA: [ci-execution-time] Improve caching for unit tests and coverage reports #6051

New issue

Open

opened 2026-04-09 14:10:29 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-09 14:10:29 +00:00

Owner

Metadata

Branch: test-infra/improve-unit-test-coverage-caching
Commit Message: perf(ci): improve caching for unit tests and coverage reports
Milestone: (backlog — see note below)
Parent Epic: #5924

Summary

The unit_tests and coverage_report nox sessions are run on every pull request and can take a significant amount of time to complete. This issue proposes a strategy for improving the caching mechanism for these sessions to reduce their execution time.

Problem

While the CI pipeline already uses uv and has a caching mechanism for dependencies, there are other artifacts that could be cached to speed up the test runs. Specifically:

Compiled Python files (.pyc) are not currently cached, meaning they are recompiled on every CI run.
The slipcover coverage data could be cached between the unit_tests and coverage_report runs, avoiding the need to run all tests twice.

Proposal

Cache compiled Python files: Configure the CI pipeline to cache __pycache__ directories to avoid recompiling Python source files on every CI run.
Cache slipcover coverage data: Investigate ways to cache the coverage data generated by the unit_tests session and reuse it in the coverage_report session, eliminating the need to run tests twice.
Optimize the uv cache: Review the current uv cache configuration to ensure it is being used effectively (cache key strategy, eviction policy).

Subtasks

Configure the CI pipeline to cache the __pycache__ directories.
Investigate and implement a mechanism for caching slipcover coverage data between unit_tests and coverage_report sessions.
Review and optimize the uv cache configuration (cache key, eviction policy).
Measure the impact of the caching improvements on the execution time of the unit_tests and coverage_report sessions.

Definition of Done

The average execution time of the unit_tests and coverage_report sessions is reduced by at least 20%.
The new caching mechanisms are implemented and documented in the CI configuration.
The CI pipeline is configured to use the new caching mechanisms.
All nox stages pass.
Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `test-infra/improve-unit-test-coverage-caching` - **Commit Message**: `perf(ci): improve caching for unit tests and coverage reports` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #5924 ## Summary The `unit_tests` and `coverage_report` nox sessions are run on every pull request and can take a significant amount of time to complete. This issue proposes a strategy for improving the caching mechanism for these sessions to reduce their execution time. ## Problem While the CI pipeline already uses `uv` and has a caching mechanism for dependencies, there are other artifacts that could be cached to speed up the test runs. Specifically: - Compiled Python files (`.pyc`) are not currently cached, meaning they are recompiled on every CI run. - The `slipcover` coverage data could be cached between the `unit_tests` and `coverage_report` runs, avoiding the need to run all tests twice. ## Proposal 1. **Cache compiled Python files**: Configure the CI pipeline to cache `__pycache__` directories to avoid recompiling Python source files on every CI run. 2. **Cache `slipcover` coverage data**: Investigate ways to cache the coverage data generated by the `unit_tests` session and reuse it in the `coverage_report` session, eliminating the need to run tests twice. 3. **Optimize the `uv` cache**: Review the current `uv` cache configuration to ensure it is being used effectively (cache key strategy, eviction policy). ## Subtasks - [ ] Configure the CI pipeline to cache the `__pycache__` directories. - [ ] Investigate and implement a mechanism for caching `slipcover` coverage data between `unit_tests` and `coverage_report` sessions. - [ ] Review and optimize the `uv` cache configuration (cache key, eviction policy). - [ ] Measure the impact of the caching improvements on the execution time of the `unit_tests` and `coverage_report` sessions. ## Definition of Done - [ ] The average execution time of the `unit_tests` and `coverage_report` sessions is reduced by at least 20%. - [ ] The new caching mechanisms are implemented and documented in the CI configuration. - [ ] The CI pipeline is configured to use the new caching mechanisms. - [ ] All nox stages pass. - [ ] Coverage >= 97%. > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

Rows
Columns