TEST-INFRA: [ci-pipeline-design] Implement matrix builds for all supported Python versions #3337

Open
opened 2026-04-05 10:10:06 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test-infra/ci-pipeline-design/matrix-builds-python-versions
  • Commit Message: chore(ci): implement matrix builds for all supported Python versions
  • Milestone: (backlog — see note below)
  • Parent Epic: #3212

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Background and Context

The test suite is only run against Python 3.13. We should run it against all supported Python versions in a matrix build to ensure compatibility.

Running tests against a single Python version risks shipping code that silently breaks on other supported interpreter versions. A CI matrix build catches version-specific regressions early, before they reach users.

Expected Behavior

The CI pipeline should run the full test suite against all Python versions that the project officially supports (as declared in pyproject.toml), using a matrix strategy so each version is tested in parallel.

Acceptance Criteria

  • The CI pipeline configuration defines a matrix of all supported Python versions
  • The full test suite (all nox sessions) runs for each Python version in the matrix
  • CI fails if any version fails, and the failing version is clearly identified in the output
  • The matrix is driven by the versions declared in pyproject.toml (or equivalent source of truth) to avoid drift
  • All nox stages pass on all supported Python versions
  • Coverage ≥ 97% on all supported Python versions

Supporting Information

  • Related issue: #3299 (Add dependency caching for uv — a complementary CI pipeline improvement)
  • Parent analysis issue: #3212 (Analyze CI pipeline design)
  • Parent Epic: #3204 (Parent issue for test infrastructure improvements)

Subtasks

  • Identify all Python versions officially supported by the project (check pyproject.toml requires-python and classifiers)
  • Review the current CI pipeline configuration to understand how jobs are defined
  • Add a matrix strategy to the CI pipeline for all supported Python versions
  • Ensure each matrix job installs the correct Python version before running nox
  • Verify that the matrix build runs correctly in CI (all versions pass)
  • Confirm that a failure in one matrix version causes the overall CI run to fail
  • Update CI documentation if applicable
  • Run nox (all default sessions) on each supported Python version locally if possible, fix any errors
  • Verify coverage ≥ 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): implement matrix builds for all supported Python versions), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test-infra/ci-pipeline-design/matrix-builds-python-versions).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test-infra/ci-pipeline-design/matrix-builds-python-versions` - **Commit Message**: `chore(ci): implement matrix builds for all supported Python versions` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #3212 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Background and Context The test suite is only run against Python 3.13. We should run it against all supported Python versions in a matrix build to ensure compatibility. Running tests against a single Python version risks shipping code that silently breaks on other supported interpreter versions. A CI matrix build catches version-specific regressions early, before they reach users. ## Expected Behavior The CI pipeline should run the full test suite against all Python versions that the project officially supports (as declared in `pyproject.toml`), using a matrix strategy so each version is tested in parallel. ## Acceptance Criteria - [ ] The CI pipeline configuration defines a matrix of all supported Python versions - [ ] The full test suite (all nox sessions) runs for each Python version in the matrix - [ ] CI fails if any version fails, and the failing version is clearly identified in the output - [ ] The matrix is driven by the versions declared in `pyproject.toml` (or equivalent source of truth) to avoid drift - [ ] All nox stages pass on all supported Python versions - [ ] Coverage ≥ 97% on all supported Python versions ## Supporting Information - **Related issue**: #3299 (Add dependency caching for uv — a complementary CI pipeline improvement) - **Parent analysis issue**: #3212 (Analyze CI pipeline design) - **Parent Epic**: #3204 (Parent issue for test infrastructure improvements) ## Subtasks - [ ] Identify all Python versions officially supported by the project (check `pyproject.toml` `requires-python` and classifiers) - [ ] Review the current CI pipeline configuration to understand how jobs are defined - [ ] Add a matrix strategy to the CI pipeline for all supported Python versions - [ ] Ensure each matrix job installs the correct Python version before running nox - [ ] Verify that the matrix build runs correctly in CI (all versions pass) - [ ] Confirm that a failure in one matrix version causes the overall CI run to fail - [ ] Update CI documentation if applicable - [ ] Run `nox` (all default sessions) on each supported Python version locally if possible, fix any errors - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): implement matrix builds for all supported Python versions`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test-infra/ci-pipeline-design/matrix-builds-python-versions`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
Author
Owner

Closing as duplicate of #2031. Both issues address implementing matrix builds for all supported Python versions. #2031 (v3.8.0, MoSCoW/Could Have) is the canonical issue. Also related to previously closed duplicate #1894.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2031. Both issues address implementing matrix builds for all supported Python versions. #2031 (v3.8.0, MoSCoW/Could Have) is the canonical issue. Also related to previously closed duplicate #1894. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#3337
No description provided.