TEST-INFRA: [ci-pipeline-design] Implement conditional test execution #1831

New issue

Open

opened 2026-04-02 23:56:06 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:56:06 +00:00

Owner

Metadata

Branch: chore/ci-conditional-test-execution
Commit Message: chore(ci): implement conditional test execution based on changed files
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The current CI pipeline runs all tests on every commit, which is inefficient and time-consuming. As the test suite grows, this approach creates unnecessary delays and wastes CI runner capacity on tests that are unaffected by a given change.

Implementing a conditional test execution strategy — where only the tests affected by the changed files in a pull request are run — will significantly reduce CI execution time and provide faster feedback to developers.

Expected Behavior

Only the tests that are affected by the changes in a pull request are run. The build time is significantly reduced compared to running the full test suite on every commit.

Acceptance Criteria

A mechanism exists to determine which files changed in a pull request.
Changed files are mapped to their relevant test suites (unit, integration, e2e).
The CI configuration only triggers the affected test sessions.
Unaffected test sessions are skipped, reducing total CI wall-clock time.
The full test suite still runs on merges to master to ensure no regressions slip through.

Supporting Information

Related issues already tracking similar work:

#1793 TEST-INFRA: [ci-pipeline-design] Implement conditional test execution based on code changes
#1758 CI Pipeline: Implement conditional test execution
#1715 TEST-INFRA: [ci-pipeline-design] Introduce Conditional Execution for Faster PR Checks
#1702 TEST-INFRA: [ci-pipeline-design] Implement conditional execution to skip jobs based on file changes
#1745 TEST-INFRA: [ci-pipeline-design] Add conditional nox test execution via Behave tags and Robot Framework filters
#1752 TEST-INFRA: [ci-pipeline-design] Investigate conditional test execution to reduce CI run times

Parent Epic: #1678 — CI Execution Time Optimization

Subtasks

Investigate tools and techniques for determining changed files in a pull request (e.g., git diff, Forgejo API, path filters in workflow YAML)
Design a mapping strategy from changed source paths to relevant nox test sessions
Implement path-filter conditions in .forgejo/workflows/ci.yml to conditionally skip unaffected jobs
Update noxfile.py if needed to support selective session execution
Update features/ci_workflow_validation.feature to assert conditional execution steps are present
Tests (Behave): Add scenarios covering conditional execution logic
Tests (Robot): Add integration smoke test verifying CI skips unaffected sessions
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
Only the tests affected by PR changes are executed on pull request events; the full suite runs on master merges.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-conditional-test-execution` - **Commit Message**: `chore(ci): implement conditional test execution based on changed files` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline runs all tests on every commit, which is inefficient and time-consuming. As the test suite grows, this approach creates unnecessary delays and wastes CI runner capacity on tests that are unaffected by a given change. Implementing a conditional test execution strategy — where only the tests affected by the changed files in a pull request are run — will significantly reduce CI execution time and provide faster feedback to developers. ## Expected Behavior Only the tests that are affected by the changes in a pull request are run. The build time is significantly reduced compared to running the full test suite on every commit. ## Acceptance Criteria - A mechanism exists to determine which files changed in a pull request. - Changed files are mapped to their relevant test suites (unit, integration, e2e). - The CI configuration only triggers the affected test sessions. - Unaffected test sessions are skipped, reducing total CI wall-clock time. - The full test suite still runs on merges to `master` to ensure no regressions slip through. ## Supporting Information Related issues already tracking similar work: - #1793 TEST-INFRA: [ci-pipeline-design] Implement conditional test execution based on code changes - #1758 CI Pipeline: Implement conditional test execution - #1715 TEST-INFRA: [ci-pipeline-design] Introduce Conditional Execution for Faster PR Checks - #1702 TEST-INFRA: [ci-pipeline-design] Implement conditional execution to skip jobs based on file changes - #1745 TEST-INFRA: [ci-pipeline-design] Add conditional nox test execution via Behave tags and Robot Framework filters - #1752 TEST-INFRA: [ci-pipeline-design] Investigate conditional test execution to reduce CI run times Parent Epic: #1678 — CI Execution Time Optimization ## Subtasks - [ ] Investigate tools and techniques for determining changed files in a pull request (e.g., `git diff`, Forgejo API, path filters in workflow YAML) - [ ] Design a mapping strategy from changed source paths to relevant nox test sessions - [ ] Implement path-filter conditions in `.forgejo/workflows/ci.yml` to conditionally skip unaffected jobs - [ ] Update `noxfile.py` if needed to support selective session execution - [ ] Update `features/ci_workflow_validation.feature` to assert conditional execution steps are present - [ ] Tests (Behave): Add scenarios covering conditional execution logic - [ ] Tests (Robot): Add integration smoke test verifying CI skips unaffected sessions - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - Only the tests affected by PR changes are executed on pull request events; the full suite runs on `master` merges. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-02 23:56:14 +00:00

freemo added a new dependency

2026-04-02 23:59:30 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 00:04:50 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Could have

label

2026-04-03 00:04:51 +00:00

freemo referenced this issue

2026-04-03 11:16:59 +00:00

TEST-INFRA: [ci-pipeline-design] Explore Parallel Execution of Nox Sessions #2245