TEST-INFRA: [ci-pipeline-design] Implement matrix testing strategy for different Python versions #1744

New issue

Open

opened 2026-04-02 23:40:17 +00:00 by freemo · 2 comments

freemo commented

2026-04-02 23:40:17 +00:00

Owner

Metadata

Branch: task/ci-matrix-testing-strategy-python-versions
Commit Message: chore(ci): implement matrix testing strategy for different Python versions
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The current CI pipeline runs tests against all supported Python versions sequentially. This is inefficient and can lead to longer build times.

We should implement a matrix testing strategy to run the tests in parallel across different Python versions. This will significantly reduce the overall CI execution time and provide faster feedback to developers.

Expected Behavior

The CI workflow should use a matrix strategy to run the unit_tests, integration_tests, and slow_integration_tests nox sessions in parallel across all supported Python versions, significantly reducing overall CI execution time.

Acceptance Criteria

The unit_tests, integration_tests, and slow_integration_tests sessions are run in parallel across all supported Python versions via a matrix strategy.
The overall CI build time is measurably reduced compared to the sequential baseline.
The matrix is driven by the SUPPORTED_PYTHONS list defined in noxfile.py.

Subtasks

Investigate how to implement a matrix testing strategy in our CI provider (Forgejo Actions).
Update .forgejo/workflows/ci.yml to use a matrix strategy (e.g., strategy.matrix.python-version) for the unit_tests, integration_tests, and slow_integration_tests sessions across all supported Python versions.
Ensure the matrix is driven by the SUPPORTED_PYTHONS list defined in noxfile.py.
Update job names to include the Python version (e.g., unit_tests (3.13)).
Verify that the tests are running in parallel across Python versions in CI.
Measure and document the impact on CI build time.
Update features/ci_workflow_validation.feature to assert the matrix strategy is present.
Run nox (all default sessions), fix any errors.
Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): implement matrix testing strategy for different Python versions), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/ci-matrix-testing-strategy-python-versions).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
The unit_tests, integration_tests, and slow_integration_tests sessions are run in parallel across all supported Python versions.
The CI build time is measurably reduced.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-matrix-testing-strategy-python-versions` - **Commit Message**: `chore(ci): implement matrix testing strategy for different Python versions` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline runs tests against all supported Python versions sequentially. This is inefficient and can lead to longer build times. We should implement a matrix testing strategy to run the tests in parallel across different Python versions. This will significantly reduce the overall CI execution time and provide faster feedback to developers. ## Expected Behavior The CI workflow should use a matrix strategy to run the `unit_tests`, `integration_tests`, and `slow_integration_tests` nox sessions in parallel across all supported Python versions, significantly reducing overall CI execution time. ## Acceptance Criteria - The `unit_tests`, `integration_tests`, and `slow_integration_tests` sessions are run in parallel across all supported Python versions via a matrix strategy. - The overall CI build time is measurably reduced compared to the sequential baseline. - The matrix is driven by the `SUPPORTED_PYTHONS` list defined in `noxfile.py`. ## Subtasks - [ ] Investigate how to implement a matrix testing strategy in our CI provider (Forgejo Actions). - [ ] Update `.forgejo/workflows/ci.yml` to use a matrix strategy (e.g., `strategy.matrix.python-version`) for the `unit_tests`, `integration_tests`, and `slow_integration_tests` sessions across all supported Python versions. - [ ] Ensure the matrix is driven by the `SUPPORTED_PYTHONS` list defined in `noxfile.py`. - [ ] Update job names to include the Python version (e.g., `unit_tests (3.13)`). - [ ] Verify that the tests are running in parallel across Python versions in CI. - [ ] Measure and document the impact on CI build time. - [ ] Update `features/ci_workflow_validation.feature` to assert the matrix strategy is present. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): implement matrix testing strategy for different Python versions`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/ci-matrix-testing-strategy-python-versions`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - The `unit_tests`, `integration_tests`, and `slow_integration_tests` sessions are run in parallel across all supported Python versions. - The CI build time is measurably reduced. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-02 23:40:44 +00:00

freemo commented

2026-04-02 23:42:25 +00:00

Author

Owner

⚠️ Potential Duplicate Notice

This issue covers the same scope as the following existing issues and may be a duplicate. A maintainer should review and consolidate if appropriate:

#1712 — "TEST-INFRA: [ci-pipeline-design] Use matrix strategy for parallel Python version testing" (most complete — has proper parent Epic #1678 link, milestone v3.8.0, full subtasks)
#1676 — "TEST-INFRA: [ci-pipeline-design] Use matrix strategy for tests" (v3.8.0, orphan — no parent Epic linked)
#1672 — "TEST-INFRA: [ci-pipeline-design] Introduce a matrix testing strategy for multiple Python versions" (v3.8.0, orphan — no parent Epic linked)

The most complete prior issue is #1712, which already has the correct parent Epic (#1678) and milestone assigned. If this issue is confirmed as a duplicate, it should be closed in favour of #1712.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

⚠️ **Potential Duplicate Notice** This issue covers the same scope as the following existing issues and may be a duplicate. A maintainer should review and consolidate if appropriate: - **#1712** — "TEST-INFRA: [ci-pipeline-design] Use matrix strategy for parallel Python version testing" *(most complete — has proper parent Epic #1678 link, milestone v3.8.0, full subtasks)* - **#1676** — "TEST-INFRA: [ci-pipeline-design] Use matrix strategy for tests" *(v3.8.0, orphan — no parent Epic linked)* - **#1672** — "TEST-INFRA: [ci-pipeline-design] Introduce a matrix testing strategy for multiple Python versions" *(v3.8.0, orphan — no parent Epic linked)* The most complete prior issue is **#1712**, which already has the correct parent Epic (#1678) and milestone assigned. If this issue is confirmed as a duplicate, it should be closed in favour of #1712. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-02 23:49:12 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-02 23:51:03 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Could have

label

2026-04-02 23:51:03 +00:00