feat(ci): introduce custom Docker image to pre-bake CI dependencies #1760

Open
opened 2026-04-02 23:44:39 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/ci-custom-docker-image
  • Commit Message: feat(ci): introduce custom Docker image to pre-bake CI dependencies
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

Every CI run currently installs all Python dependencies from scratch via uv sync (or pip install) on a bare runner image. This cold-install step adds significant wall-clock time to every pull-request check, even when the dependency set has not changed. A custom Docker image that pre-bakes the full dependency tree — including uv, the project's virtual environment, and any system-level packages required by the test suite — eliminates this overhead entirely for the common case.

This work is a natural complement to the concurrency and parallelism improvements tracked in Epic #1678: once jobs run faster in parallel, the per-job startup cost becomes the next dominant bottleneck.

Current Behavior

  • CI jobs pull a generic Python base image and run uv sync (or equivalent) on every invocation.
  • Cold dependency installation takes a non-trivial amount of time per job, multiplied across all parallel jobs.
  • There is no published, versioned CI image for this repository; runner caches are the only mitigation and they are unreliable across runner restarts.

Expected Behavior

  • A Dockerfile (or equivalent build spec) lives in the repository under ci/ and defines a versioned CI image.
  • The image is built and pushed to the project's container registry automatically when pyproject.toml, uv.lock, or the Dockerfile itself changes.
  • All CI jobs reference the custom image by digest or tag, eliminating the cold-install step.
  • The image version is pinned in the workflow files so builds are reproducible.
  • A rebuild workflow (or Forgejo Actions trigger) exists to refresh the image on dependency updates.

Acceptance Criteria

  • A ci/Dockerfile (or ci/docker/Dockerfile) exists that produces a working CI image containing Python, uv, and all project dependencies pre-installed.
  • A Forgejo Actions workflow (e.g., .forgejo/workflows/build-ci-image.yml) builds and pushes the image to the registry on changes to pyproject.toml, uv.lock, or the Dockerfile.
  • All existing CI workflow jobs are updated to use the new custom image instead of the bare Python base image.
  • The image tag/digest is pinned in workflow files; floating latest tags are not used.
  • CI wall-clock time for dependency installation is reduced to near-zero for jobs using the pre-baked image.
  • The image build workflow itself passes all CI checks (lint, security scan).
  • Documentation in docs/ or a ci/README.md describes how to rebuild the image locally and how the versioning scheme works.

Supporting Information

  • Related Epic: #1678 — CI Execution Time Optimization
  • Existing CI pipeline: .forgejo/workflows/
  • Dependency manifest: pyproject.toml, uv.lock
  • The nox task runner is used for all test and quality sessions; the image should support nox invocations directly.

Subtasks

  • Audit current CI base image and all system-level packages installed during CI runs
  • Write ci/Dockerfile (multi-stage if appropriate) that pre-installs uv, the venv, and system deps
  • Write .forgejo/workflows/build-ci-image.yml to build and push on dependency file changes
  • Update all existing CI workflow files to reference the new image
  • Pin image by digest or immutable tag in workflow files
  • Add ci/README.md documenting image versioning, local rebuild steps, and update process
  • Tests (Behave): verify CI image build workflow triggers correctly on relevant file changes
  • Verify coverage ≥ 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (feat(ci): introduce custom Docker image to pre-bake CI dependencies), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-custom-docker-image).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-custom-docker-image` - **Commit Message**: `feat(ci): introduce custom Docker image to pre-bake CI dependencies` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context Every CI run currently installs all Python dependencies from scratch via `uv sync` (or `pip install`) on a bare runner image. This cold-install step adds significant wall-clock time to every pull-request check, even when the dependency set has not changed. A custom Docker image that pre-bakes the full dependency tree — including `uv`, the project's virtual environment, and any system-level packages required by the test suite — eliminates this overhead entirely for the common case. This work is a natural complement to the concurrency and parallelism improvements tracked in Epic #1678: once jobs run faster in parallel, the per-job startup cost becomes the next dominant bottleneck. ## Current Behavior - CI jobs pull a generic Python base image and run `uv sync` (or equivalent) on every invocation. - Cold dependency installation takes a non-trivial amount of time per job, multiplied across all parallel jobs. - There is no published, versioned CI image for this repository; runner caches are the only mitigation and they are unreliable across runner restarts. ## Expected Behavior - A `Dockerfile` (or equivalent build spec) lives in the repository under `ci/` and defines a versioned CI image. - The image is built and pushed to the project's container registry automatically when `pyproject.toml`, `uv.lock`, or the `Dockerfile` itself changes. - All CI jobs reference the custom image by digest or tag, eliminating the cold-install step. - The image version is pinned in the workflow files so builds are reproducible. - A rebuild workflow (or Forgejo Actions trigger) exists to refresh the image on dependency updates. ## Acceptance Criteria - [ ] A `ci/Dockerfile` (or `ci/docker/Dockerfile`) exists that produces a working CI image containing Python, `uv`, and all project dependencies pre-installed. - [ ] A Forgejo Actions workflow (e.g., `.forgejo/workflows/build-ci-image.yml`) builds and pushes the image to the registry on changes to `pyproject.toml`, `uv.lock`, or the Dockerfile. - [ ] All existing CI workflow jobs are updated to use the new custom image instead of the bare Python base image. - [ ] The image tag/digest is pinned in workflow files; floating `latest` tags are not used. - [ ] CI wall-clock time for dependency installation is reduced to near-zero for jobs using the pre-baked image. - [ ] The image build workflow itself passes all CI checks (lint, security scan). - [ ] Documentation in `docs/` or a `ci/README.md` describes how to rebuild the image locally and how the versioning scheme works. ## Supporting Information - Related Epic: #1678 — CI Execution Time Optimization - Existing CI pipeline: `.forgejo/workflows/` - Dependency manifest: `pyproject.toml`, `uv.lock` - The `nox` task runner is used for all test and quality sessions; the image should support `nox` invocations directly. ## Subtasks - [ ] Audit current CI base image and all system-level packages installed during CI runs - [ ] Write `ci/Dockerfile` (multi-stage if appropriate) that pre-installs `uv`, the venv, and system deps - [ ] Write `.forgejo/workflows/build-ci-image.yml` to build and push on dependency file changes - [ ] Update all existing CI workflow files to reference the new image - [ ] Pin image by digest or immutable tag in workflow files - [ ] Add `ci/README.md` documenting image versioning, local rebuild steps, and update process - [ ] Tests (Behave): verify CI image build workflow triggers correctly on relevant file changes - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`feat(ci): introduce custom Docker image to pre-bake CI dependencies`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-custom-docker-image`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:44:58 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1760
No description provided.