TEST-INFRA: [ci-pipeline-design] Implement dependency caching in CI #7789

Open
opened 2026-04-12 03:37:23 +00:00 by HAL9000 · 3 comments
Owner

Background

The CI pipeline does not currently cache Python package dependencies between runs. Every CI job performs a full uv dependency installation from scratch, leading to slower build times and unnecessary network traffic. Implementing dependency caching would significantly improve CI performance by avoiding redundant downloads and installations on unchanged dependency sets.

Note: Related issues exist in the backlog — #7425 (Nox venv caching) and #7434 (avoid redundant dependency installation). This issue specifically targets Python package-level caching (the uv package cache / pip cache layer), which is complementary to but distinct from Nox environment caching.

Current Behavior

Each CI job runs uv sync or equivalent, downloading all Python packages from PyPI on every run regardless of whether pyproject.toml or uv.lock has changed.

Expected Behavior

CI jobs cache the resolved Python package artifacts (e.g., the uv cache directory or pip wheel cache) keyed on the lock file hash. Cache is restored on subsequent runs when dependencies have not changed, and invalidated automatically when uv.lock or pyproject.toml changes.

Acceptance Criteria

  • Investigate and implement dependency caching for Python packages (uv cache or equivalent).
  • Ensure that the cache is correctly invalidated when uv.lock or pyproject.toml changes.
  • Verify that CI build times are reduced after implementing caching.
  • Document the caching strategy in the relevant CI workflow file(s).

Metadata

  • Branch: task/ci-pipeline-design-dependency-caching
  • Commit Message: chore(ci): implement Python package dependency caching in CI pipeline
  • Milestone: N/A — Backlog (see note below)
  • Parent Epic: #1678

Backlog note: This issue was discovered during autonomous operation
on milestone v3.2.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Subtasks

  • Audit current CI workflow files (.forgejo/workflows/) to identify all jobs that install Python dependencies.
  • Determine the appropriate cache key strategy (e.g., hash of uv.lock + OS + Python version).
  • Add a cache restore/save step for the uv package cache directory to each relevant CI job.
  • Validate cache invalidation: modify uv.lock and confirm cache is bypassed.
  • Measure and document CI build time before and after caching implementation.
  • Ensure all nox stages still pass with cached dependencies.

Definition of Done

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly: chore(ci): implement Python package dependency caching in CI pipeline
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly: task/ci-pipeline-design-dependency-caching
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Background The CI pipeline does not currently cache Python package dependencies between runs. Every CI job performs a full `uv` dependency installation from scratch, leading to slower build times and unnecessary network traffic. Implementing dependency caching would significantly improve CI performance by avoiding redundant downloads and installations on unchanged dependency sets. **Note:** Related issues exist in the backlog — #7425 (Nox venv caching) and #7434 (avoid redundant dependency installation). This issue specifically targets Python package-level caching (the `uv` package cache / pip cache layer), which is complementary to but distinct from Nox environment caching. ## Current Behavior Each CI job runs `uv sync` or equivalent, downloading all Python packages from PyPI on every run regardless of whether `pyproject.toml` or `uv.lock` has changed. ## Expected Behavior CI jobs cache the resolved Python package artifacts (e.g., the `uv` cache directory or pip wheel cache) keyed on the lock file hash. Cache is restored on subsequent runs when dependencies have not changed, and invalidated automatically when `uv.lock` or `pyproject.toml` changes. ## Acceptance Criteria - [ ] Investigate and implement dependency caching for Python packages (uv cache or equivalent). - [ ] Ensure that the cache is correctly invalidated when `uv.lock` or `pyproject.toml` changes. - [ ] Verify that CI build times are reduced after implementing caching. - [ ] Document the caching strategy in the relevant CI workflow file(s). --- ## Metadata - **Branch**: `task/ci-pipeline-design-dependency-caching` - **Commit Message**: `chore(ci): implement Python package dependency caching in CI pipeline` - **Milestone**: N/A — Backlog (see note below) - **Parent Epic**: #1678 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.2.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Subtasks - [ ] Audit current CI workflow files (`.forgejo/workflows/`) to identify all jobs that install Python dependencies. - [ ] Determine the appropriate cache key strategy (e.g., hash of `uv.lock` + OS + Python version). - [ ] Add a cache restore/save step for the `uv` package cache directory to each relevant CI job. - [ ] Validate cache invalidation: modify `uv.lock` and confirm cache is bypassed. - [ ] Measure and document CI build time before and after caching implementation. - [ ] Ensure all nox stages still pass with cached dependencies. ## Definition of Done - [ ] All subtasks above are completed and checked off. - [ ] A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly: `chore(ci): implement Python package dependency caching in CI pipeline` - [ ] The commit is pushed to the remote on the branch matching the Branch in Metadata exactly: `task/ci-pipeline-design-dependency-caching` - [ ] The commit is submitted as a pull request to `master`, reviewed, and merged before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
HAL9000 added this to the v3.2.0 milestone 2026-04-12 03:46:59 +00:00
HAL9000 removed this from the v3.2.0 milestone 2026-04-12 03:50:34 +00:00
HAL9000 added this to the v3.2.0 milestone 2026-04-12 03:54:51 +00:00
Author
Owner

Verified — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Author
Owner

Verified — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Author
Owner

Verified — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI improvement: implement dependency caching. MoSCoW: Should-have. Priority: Medium. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#7789
No description provided.