ci-pipeline-design: Use a shared cache for all CI jobs #1812

Open
opened 2026-04-02 23:54:36 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-shared-dependency-cache
  • Commit Message: chore(ci): configure shared dependency cache for all CI jobs
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The current CI pipeline in .forgejo/workflows/ci.yml configures a separate dependency cache per job. This is inefficient: each job independently downloads and installs the same Python dependencies, leading to redundant network I/O and installation time on every pipeline run. A single shared cache keyed on pyproject.toml would be restored once and reused across all jobs, reducing overall pipeline wall-clock time and runner resource consumption.

Current Behavior

Each CI job that requires Python dependencies defines its own isolated cache step. When dependencies have not changed, every job still performs a full cache restore independently, and any cache miss causes a full reinstall in every affected job.

Expected Behavior

A single shared cache is configured for all CI jobs. The cache key is derived from the contents of pyproject.toml so that the cache is automatically invalidated when dependencies change. All jobs restore from this shared cache, eliminating redundant installs.

Acceptance Criteria

  • A single, shared cache entry is defined in .forgejo/workflows/ci.yml and referenced by all jobs that require Python dependencies.
  • The cache key is based on a hash of pyproject.toml.
  • No job performs a redundant, independent cache configuration when the shared cache is available.
  • The CI pipeline runs successfully end-to-end with the shared cache in place.
  • The features/ci_workflow_validation.feature BDD scenarios are updated to assert the shared cache configuration is present.

Supporting Information

  • Parent Epic: #1678 — "Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing"
  • Related CI workflow file: .forgejo/workflows/ci.yml

Subtasks

  • Audit all jobs in .forgejo/workflows/ci.yml that currently define their own cache steps
  • Design a shared cache configuration keyed on hashFiles('pyproject.toml')
  • Replace per-job cache definitions with references to the shared cache
  • Verify all jobs can correctly restore and use the shared cached dependencies
  • Update features/ci_workflow_validation.feature to assert shared cache configuration
  • Run nox (all default sessions) and fix any errors
  • Verify coverage ≥ 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All per-job cache definitions in .forgejo/workflows/ci.yml are replaced with a single shared cache configuration.
  • The shared cache key is based on hashFiles('pyproject.toml').
  • All CI jobs successfully restore dependencies from the shared cache.
  • features/ci_workflow_validation.feature is updated to cover the shared cache configuration.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-shared-dependency-cache` - **Commit Message**: `chore(ci): configure shared dependency cache for all CI jobs` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline in `.forgejo/workflows/ci.yml` configures a separate dependency cache per job. This is inefficient: each job independently downloads and installs the same Python dependencies, leading to redundant network I/O and installation time on every pipeline run. A single shared cache keyed on `pyproject.toml` would be restored once and reused across all jobs, reducing overall pipeline wall-clock time and runner resource consumption. ## Current Behavior Each CI job that requires Python dependencies defines its own isolated cache step. When dependencies have not changed, every job still performs a full cache restore independently, and any cache miss causes a full reinstall in every affected job. ## Expected Behavior A single shared cache is configured for all CI jobs. The cache key is derived from the contents of `pyproject.toml` so that the cache is automatically invalidated when dependencies change. All jobs restore from this shared cache, eliminating redundant installs. ## Acceptance Criteria - A single, shared cache entry is defined in `.forgejo/workflows/ci.yml` and referenced by all jobs that require Python dependencies. - The cache key is based on a hash of `pyproject.toml`. - No job performs a redundant, independent cache configuration when the shared cache is available. - The CI pipeline runs successfully end-to-end with the shared cache in place. - The `features/ci_workflow_validation.feature` BDD scenarios are updated to assert the shared cache configuration is present. ## Supporting Information - Parent Epic: #1678 — "Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing" - Related CI workflow file: `.forgejo/workflows/ci.yml` ## Subtasks - [ ] Audit all jobs in `.forgejo/workflows/ci.yml` that currently define their own cache steps - [ ] Design a shared cache configuration keyed on `hashFiles('pyproject.toml')` - [ ] Replace per-job cache definitions with references to the shared cache - [ ] Verify all jobs can correctly restore and use the shared cached dependencies - [ ] Update `features/ci_workflow_validation.feature` to assert shared cache configuration - [ ] Run `nox` (all default sessions) and fix any errors - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - [ ] All per-job cache definitions in `.forgejo/workflows/ci.yml` are replaced with a single shared cache configuration. - [ ] The shared cache key is based on `hashFiles('pyproject.toml')`. - [ ] All CI jobs successfully restore dependencies from the shared cache. - [ ] `features/ci_workflow_validation.feature` is updated to cover the shared cache configuration. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:54:50 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1812
No description provided.