TEST-INFRA: [ci-pipeline-design] Introduce matrix testing for different Python versions #1913

Open
opened 2026-04-03 00:12:09 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-python-version-matrix
  • Commit Message: chore(ci): introduce matrix testing for all supported Python versions
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The CI pipeline currently runs tests against a single Python version (3.13). To ensure compatibility and prevent regressions across the full range of supported Python versions, it is important to run tests against all supported Python versions. This can be achieved by introducing a test matrix in the CI configuration.

Expected Behavior

The CI pipeline should run the full test suite against every supported Python version in parallel, collect results from each, and fail the build if any version reports a test failure.

Acceptance Criteria

  • The CI pipeline runs tests against all supported Python versions.
  • The CI pipeline fails if tests fail on any of the supported Python versions.
  • Test reports are available for each Python version in the matrix.

Subtasks

  • Identify all supported Python versions.
  • Update the CI configuration to run tests in a matrix against all supported Python versions.
  • Ensure that the test results from all Python versions are collected and reported correctly.
  • Verify coverage >= 97% passes across all matrix versions via nox -s coverage_report.
  • Run nox (all default sessions), fix any errors.

Definition of Done

  • The CI pipeline runs tests against all supported Python versions.
  • The CI pipeline fails if tests fail on any of the supported Python versions.
  • Test reports are available for each Python version in the matrix.
  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-python-version-matrix` - **Commit Message**: `chore(ci): introduce matrix testing for all supported Python versions` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The CI pipeline currently runs tests against a single Python version (3.13). To ensure compatibility and prevent regressions across the full range of supported Python versions, it is important to run tests against all supported Python versions. This can be achieved by introducing a test matrix in the CI configuration. ## Expected Behavior The CI pipeline should run the full test suite against every supported Python version in parallel, collect results from each, and fail the build if any version reports a test failure. ## Acceptance Criteria - The CI pipeline runs tests against all supported Python versions. - The CI pipeline fails if tests fail on any of the supported Python versions. - Test reports are available for each Python version in the matrix. ## Subtasks - [ ] Identify all supported Python versions. - [ ] Update the CI configuration to run tests in a matrix against all supported Python versions. - [ ] Ensure that the test results from all Python versions are collected and reported correctly. - [ ] Verify coverage >= 97% passes across all matrix versions via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done - [ ] The CI pipeline runs tests against all supported Python versions. - [ ] The CI pipeline fails if tests fail on any of the supported Python versions. - [ ] Test reports are available for each Python version in the matrix. - [ ] All subtasks above are completed and checked off. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 00:12:18 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1913
No description provided.