TEST-INFRA: [ci-pipeline-design] Introduce a matrix strategy for Python versions #1842

Open
opened 2026-04-02 23:57:08 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/ci-matrix-python-versions
  • Commit Message: chore(ci): introduce matrix strategy for Python version testing in CI pipeline
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The current CI pipeline in .forgejo/workflows/ci.yml only tests against a single Python version (3.13). This does not guarantee that the code is compatible with other Python versions that users or downstream consumers may be running. As the project matures and potentially broadens its supported Python range, this gap will become a source of undetected regressions.

Expected Behavior

The CI pipeline should use a strategy.matrix block to test against multiple Python versions (e.g., 3.11, 3.12, 3.13). The lint, typecheck, security, quality, and test jobs should each run for every Python version in the matrix, ensuring broad compatibility is verified on every CI run.

Acceptance Criteria

  • The .forgejo/workflows/ci.yml file defines a strategy.matrix block with the supported Python versions (at minimum 3.11, 3.12, 3.13).
  • All relevant jobs (lint, typecheck, security, quality, and test jobs) use the matrix-provided Python version via the matrix.python-version variable.
  • All matrix jobs must pass for the CI run to be considered successful.
  • The matrix is consistent with the SUPPORTED_PYTHONS list defined in noxfile.py.

Supporting Information

Subtasks

  • Add a strategy.matrix block with Python versions (3.11, 3.12, 3.13) to the relevant jobs in .forgejo/workflows/ci.yml.
  • Update each job step to reference ${{ matrix.python-version }} when setting up Python.
  • Align the matrix version list with SUPPORTED_PYTHONS in noxfile.py (or derive it from there).
  • Update features/ci_workflow_validation.feature to assert matrix strategy is present in the workflow.
  • Tests (Behave): Add/update BDD scenarios covering the matrix strategy configuration.
  • Verify coverage >= 97% via nox -s coverage_report.
  • Run nox (all default sessions), fix any errors.

Definition of Done

  • All subtasks above are completed and checked off.
  • The CI pipeline tests against multiple Python versions (3.11, 3.12, 3.13 at minimum).
  • The code is verified to be compatible with all Python versions in the matrix.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-matrix-python-versions` - **Commit Message**: `chore(ci): introduce matrix strategy for Python version testing in CI pipeline` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline in `.forgejo/workflows/ci.yml` only tests against a single Python version (3.13). This does not guarantee that the code is compatible with other Python versions that users or downstream consumers may be running. As the project matures and potentially broadens its supported Python range, this gap will become a source of undetected regressions. ## Expected Behavior The CI pipeline should use a `strategy.matrix` block to test against multiple Python versions (e.g., 3.11, 3.12, 3.13). The `lint`, `typecheck`, `security`, `quality`, and test jobs should each run for every Python version in the matrix, ensuring broad compatibility is verified on every CI run. ## Acceptance Criteria - The `.forgejo/workflows/ci.yml` file defines a `strategy.matrix` block with the supported Python versions (at minimum 3.11, 3.12, 3.13). - All relevant jobs (`lint`, `typecheck`, `security`, `quality`, and test jobs) use the matrix-provided Python version via the `matrix.python-version` variable. - All matrix jobs must pass for the CI run to be considered successful. - The matrix is consistent with the `SUPPORTED_PYTHONS` list defined in `noxfile.py`. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing - Related issues: #1794 (matrix strategy — incomplete), #1803 (matrix testing — broad), #1804 (matrix builds in noxfile.py) - Forgejo Actions `strategy.matrix` docs: https://forgejo.org/docs/latest/user/actions/ ## Subtasks - [ ] Add a `strategy.matrix` block with Python versions (3.11, 3.12, 3.13) to the relevant jobs in `.forgejo/workflows/ci.yml`. - [ ] Update each job step to reference `${{ matrix.python-version }}` when setting up Python. - [ ] Align the matrix version list with `SUPPORTED_PYTHONS` in `noxfile.py` (or derive it from there). - [ ] Update `features/ci_workflow_validation.feature` to assert matrix strategy is present in the workflow. - [ ] Tests (Behave): Add/update BDD scenarios covering the matrix strategy configuration. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done - [ ] All subtasks above are completed and checked off. - [ ] The CI pipeline tests against multiple Python versions (3.11, 3.12, 3.13 at minimum). - [ ] The code is verified to be compatible with all Python versions in the matrix. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:57:39 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1842
No description provided.