TEST-INFRA: [ci-pipeline-design] Implement Dependency Caching in Nox Sessions #2203

Open
opened 2026-04-03 09:29:09 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-nox-dependency-caching
  • Commit Message: chore(ci): implement dependency caching in nox sessions
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The CI pipeline currently reinstalls all dependencies from scratch for each nox session on every pipeline run. This results in significant redundant work — packages that have not changed are downloaded and installed repeatedly, adding unnecessary time to every CI execution.

Implementing dependency caching in nox sessions would allow the pipeline to reuse previously installed environments when the dependency set has not changed (i.e., when pyproject.toml and lock files are unchanged). This is a standard CI optimization that directly reduces pipeline wall-clock time and resource consumption.

This issue is part of the broader CI Execution Time Optimization effort tracked in Epic #1678.

Expected Behavior

Nox sessions should detect when the virtual environment and installed packages are already up-to-date with the current dependency specification and skip reinstallation. On cache hits, sessions should proceed directly to their primary task (linting, testing, type-checking, etc.) without re-downloading or re-installing packages.

Acceptance Criteria

  • Nox sessions use reuse_venv or equivalent caching strategy so that unchanged dependency sets are not reinstalled on repeated runs
  • Cache invalidation is correctly triggered when pyproject.toml, uv.lock, or other dependency manifests change
  • CI pipeline configuration (e.g., GitHub Actions / Forgejo CI YAML) caches the nox virtual environment directories between runs using the appropriate cache key (hash of dependency files)
  • All existing nox sessions continue to pass after the caching changes are applied
  • Pipeline wall-clock time is measurably reduced on cache-hit runs compared to baseline
  • No stale-cache issues: a changed dependency file always triggers a full reinstall

Supporting Information

Subtasks

  • Audit all current nox sessions and identify which install dependencies (and how)
  • Enable reuse_venv=True (or venv_backend caching) on all applicable nox sessions
  • Add CI cache step to persist and restore nox .nox/ virtual environment directories, keyed on hash of pyproject.toml + uv.lock
  • Validate cache invalidation: modify a dependency and confirm the venv is rebuilt
  • Validate cache hit: run pipeline twice with no dependency changes and confirm second run skips reinstall
  • Tests (Behave/Robot): add or update integration scenarios verifying nox session behaviour with and without cache
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): implement dependency caching in nox sessions), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (chore/ci-nox-dependency-caching).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-nox-dependency-caching` - **Commit Message**: `chore(ci): implement dependency caching in nox sessions` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The CI pipeline currently reinstalls all dependencies from scratch for each nox session on every pipeline run. This results in significant redundant work — packages that have not changed are downloaded and installed repeatedly, adding unnecessary time to every CI execution. Implementing dependency caching in nox sessions would allow the pipeline to reuse previously installed environments when the dependency set has not changed (i.e., when `pyproject.toml` and lock files are unchanged). This is a standard CI optimization that directly reduces pipeline wall-clock time and resource consumption. This issue is part of the broader CI Execution Time Optimization effort tracked in Epic #1678. ## Expected Behavior Nox sessions should detect when the virtual environment and installed packages are already up-to-date with the current dependency specification and skip reinstallation. On cache hits, sessions should proceed directly to their primary task (linting, testing, type-checking, etc.) without re-downloading or re-installing packages. ## Acceptance Criteria - [ ] Nox sessions use `reuse_venv` or equivalent caching strategy so that unchanged dependency sets are not reinstalled on repeated runs - [ ] Cache invalidation is correctly triggered when `pyproject.toml`, `uv.lock`, or other dependency manifests change - [ ] CI pipeline configuration (e.g., GitHub Actions / Forgejo CI YAML) caches the nox virtual environment directories between runs using the appropriate cache key (hash of dependency files) - [ ] All existing nox sessions continue to pass after the caching changes are applied - [ ] Pipeline wall-clock time is measurably reduced on cache-hit runs compared to baseline - [ ] No stale-cache issues: a changed dependency file always triggers a full reinstall ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization - Nox documentation on `reuse_venv`: https://nox.thea.codes/en/stable/config.html#nox.sessions.Session.install - Related: CI concurrency and timeout work tracked in Epic #1678 ## Subtasks - [ ] Audit all current nox sessions and identify which install dependencies (and how) - [ ] Enable `reuse_venv=True` (or `venv_backend` caching) on all applicable nox sessions - [ ] Add CI cache step to persist and restore nox `.nox/` virtual environment directories, keyed on hash of `pyproject.toml` + `uv.lock` - [ ] Validate cache invalidation: modify a dependency and confirm the venv is rebuilt - [ ] Validate cache hit: run pipeline twice with no dependency changes and confirm second run skips reinstall - [ ] Tests (Behave/Robot): add or update integration scenarios verifying nox session behaviour with and without cache - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): implement dependency caching in nox sessions`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`chore/ci-nox-dependency-caching`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 09:29:45 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High (confirmed) — Dependency caching is a foundational CI optimization that benefits all subsequent runs.
  • Milestone: v3.8.0 (confirmed — CI infrastructure improvements)
  • MoSCoW: Could Have — CI dependency caching improves pipeline speed but does not block any milestone deliverables. The pipeline functions correctly without it; this is a performance optimization. Note: This may overlap with #1977 ("Cache nox virtual environments") and #2018 ("Implement dependency caching in CI"). Check for potential duplication.
  • Parent Epic: #1678 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High (confirmed) — Dependency caching is a foundational CI optimization that benefits all subsequent runs. - **Milestone**: v3.8.0 (confirmed — CI infrastructure improvements) - **MoSCoW**: Could Have — CI dependency caching improves pipeline speed but does not block any milestone deliverables. The pipeline functions correctly without it; this is a performance optimization. Note: This may overlap with #1977 ("Cache nox virtual environments") and #2018 ("Implement dependency caching in CI"). Check for potential duplication. - **Parent Epic**: #1678 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2203
No description provided.