TEST-INFRA: [ci-pipeline-design] Introduce a matrix testing strategy for multiple Python versions #1672

New issue

Open

opened 2026-04-02 23:27:21 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:27:21 +00:00

Owner

Metadata

Branch: task/ci-matrix-testing-python-versions
Commit Message: chore(ci): introduce matrix testing strategy for multiple Python versions
Milestone: v3.8.0
Parent Epic: (orphan — needs manual linking; no open Epic found for ci-pipeline-design)

Background and Context

The noxfile.py currently only lists Python 3.13 in SUPPORTED_PYTHONS. If the project is intended to support multiple Python versions, the current CI setup does not automatically test against all of them. This can lead to compatibility issues being missed early in the development cycle.

A matrix testing strategy in the CI pipeline will run the test suite against all supported Python versions in parallel, ensuring that the project remains compatible with all supported versions and that any version-specific issues are caught early.

Proposed Solution

Introduce a matrix strategy in the .forgejo/workflows/ CI configuration to run the unit_tests and integration_tests nox sessions for each Python version listed in SUPPORTED_PYTHONS, in parallel.

Subtasks

Identify all Python versions that the project should support and update SUPPORTED_PYTHONS in noxfile.py accordingly.
Configure the CI pipeline (.forgejo/workflows/) to use a matrix strategy to run the unit_tests and integration_tests sessions for each Python version in SUPPORTED_PYTHONS.
Ensure that the CI pipeline runs the tests in parallel for each Python version.
Update docs/development/ci-cd.md to document the new matrix testing strategy.
Tests (Behave): Verify no regressions in existing unit test scenarios.
Verify coverage >= 97% via nox -s coverage_report.
Run nox (all default sessions), fix any errors.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): introduce matrix testing strategy for multiple Python versions), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-matrix-testing-python-versions).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
The CI pipeline is configured to run tests against all supported Python versions in a matrix.
The SUPPORTED_PYTHONS list in noxfile.py is up-to-date.
The CI pipeline provides clear feedback on the test results for each Python version.
The matrix testing strategy is documented in docs/development/ci-cd.md.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-matrix-testing-python-versions` - **Commit Message**: `chore(ci): introduce matrix testing strategy for multiple Python versions` - **Milestone**: v3.8.0 - **Parent Epic**: *(orphan — needs manual linking; no open Epic found for ci-pipeline-design)* ## Background and Context The `noxfile.py` currently only lists Python 3.13 in `SUPPORTED_PYTHONS`. If the project is intended to support multiple Python versions, the current CI setup does not automatically test against all of them. This can lead to compatibility issues being missed early in the development cycle. A matrix testing strategy in the CI pipeline will run the test suite against all supported Python versions in parallel, ensuring that the project remains compatible with all supported versions and that any version-specific issues are caught early. ## Proposed Solution Introduce a matrix strategy in the `.forgejo/workflows/` CI configuration to run the `unit_tests` and `integration_tests` nox sessions for each Python version listed in `SUPPORTED_PYTHONS`, in parallel. ## Subtasks - [ ] Identify all Python versions that the project should support and update `SUPPORTED_PYTHONS` in `noxfile.py` accordingly. - [ ] Configure the CI pipeline (`.forgejo/workflows/`) to use a matrix strategy to run the `unit_tests` and `integration_tests` sessions for each Python version in `SUPPORTED_PYTHONS`. - [ ] Ensure that the CI pipeline runs the tests in parallel for each Python version. - [ ] Update `docs/development/ci-cd.md` to document the new matrix testing strategy. - [ ] Tests (Behave): Verify no regressions in existing unit test scenarios. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): introduce matrix testing strategy for multiple Python versions`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-matrix-testing-python-versions`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - The CI pipeline is configured to run tests against all supported Python versions in a matrix. - The `SUPPORTED_PYTHONS` list in `noxfile.py` is up-to-date. - The CI pipeline provides clear feedback on the test results for each Python version. - The matrix testing strategy is documented in `docs/development/ci-cd.md`. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-02 23:27:41 +00:00

freemo added the

labels

2026-04-02 23:28:58 +00:00

freemo commented

2026-04-02 23:29:38 +00:00

Author

Owner

⚠️ Orphan Issue — Manual Linking Required

This issue was created without a parent Epic because no open Type/Epic issue was found in the repository for the ci-pipeline-design focus area.

Per CONTRIBUTING.md, orphan issues are not permitted. A project owner must:

Identify or create an appropriate parent Epic for CI pipeline design work.
Link this issue as a dependency of that Epic (this issue blocks the parent Epic).

Note: A closely related issue already exists: #1539 "TEST-INFRA: [ci-pipeline-design] Use Matrix Strategy for Python Versions" (currently State/In progress). The project owner may wish to evaluate whether this issue duplicates #1539 or represents distinct scope before proceeding.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

⚠️ **Orphan Issue — Manual Linking Required** This issue was created without a parent Epic because no open `Type/Epic` issue was found in the repository for the `ci-pipeline-design` focus area. Per `CONTRIBUTING.md`, orphan issues are **not permitted**. A project owner must: 1. Identify or create an appropriate parent Epic for CI pipeline design work. 2. Link this issue as a dependency of that Epic (this issue **blocks** the parent Epic). **Note**: A closely related issue already exists: #1539 "TEST-INFRA: [ci-pipeline-design] Use Matrix Strategy for Python Versions" (currently `State/In progress`). The project owner may wish to evaluate whether this issue duplicates #1539 or represents distinct scope before proceeding. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo referenced this issue

2026-04-02 23:42:25 +00:00

TEST-INFRA: [ci-pipeline-design] Implement matrix testing strategy for different Python versions #1744

freemo referenced this issue

2026-04-02 23:52:50 +00:00

Proposal: improve ca-test-infra-improver — prevent massive duplicate issue creation #1802

freemo removed the