TEST-INFRA: [ci-pipeline-design] Implement Conditional Test Execution #1870

Open
opened 2026-04-03 00:02:50 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: feature/ci-conditional-test-execution
  • Commit Message: feat(ci): implement conditional test execution based on changed files in pull requests
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The CI pipeline currently runs the full test suite on every pull request, regardless of which files were changed. This results in unnecessary build time and resource consumption — for example, a PR that only modifies documentation has no need to execute the entire test suite.

Implementing conditional test execution will allow the CI pipeline to selectively run only the tests that are relevant to the changed files in a given pull request, reducing overall build time and improving developer feedback loops.

Expected Behavior

The CI pipeline should inspect the set of changed files in a pull request and selectively execute only the test sessions that are relevant to those changes. For example:

  • Documentation-only changes should skip the full test suite.
  • Changes to a specific module should only trigger tests for that module and its dependents.

Acceptance Criteria

  • The CI pipeline correctly identifies changed files in a pull request.
  • Only the test sessions relevant to the changed files are executed.
  • Documentation-only PRs do not trigger the full test suite.
  • The overall build time is measurably reduced for PRs that affect only a subset of the codebase.
  • The conditional logic is covered by the features/ci_workflow_validation.feature BDD scenarios.

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing
  • Related tooling to investigate: nox-poetry, Forgejo workflow path filters (paths: / paths-ignore: triggers), and job-level if: conditions.

Subtasks

  • Investigate using Forgejo workflow paths: / paths-ignore: trigger filters to skip entire workflow runs for documentation-only changes.
  • Investigate using a tool like nox-poetry or custom scripting to identify changed files within a pull request and map them to relevant nox sessions.
  • Design the conditional execution strategy and document the decision in an ADR if it constitutes an architectural change.
  • Modify .forgejo/workflows/ci.yml to selectively run the corresponding test sessions based on the changed files.
  • Update features/ci_workflow_validation.feature to assert conditional execution paths are present and correct.
  • Tests (Behave): Add scenarios covering the conditional test execution logic.
  • Verify coverage >= 97% via nox -s coverage_report.
  • Run nox (all default sessions), fix any errors.

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • The CI pipeline only runs the tests that are relevant to the changes in a pull request.
  • The overall build time is reduced for pull requests that only affect a subset of the codebase.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `feature/ci-conditional-test-execution` - **Commit Message**: `feat(ci): implement conditional test execution based on changed files in pull requests` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The CI pipeline currently runs the full test suite on every pull request, regardless of which files were changed. This results in unnecessary build time and resource consumption — for example, a PR that only modifies documentation has no need to execute the entire test suite. Implementing conditional test execution will allow the CI pipeline to selectively run only the tests that are relevant to the changed files in a given pull request, reducing overall build time and improving developer feedback loops. ## Expected Behavior The CI pipeline should inspect the set of changed files in a pull request and selectively execute only the test sessions that are relevant to those changes. For example: - Documentation-only changes should skip the full test suite. - Changes to a specific module should only trigger tests for that module and its dependents. ## Acceptance Criteria - [ ] The CI pipeline correctly identifies changed files in a pull request. - [ ] Only the test sessions relevant to the changed files are executed. - [ ] Documentation-only PRs do not trigger the full test suite. - [ ] The overall build time is measurably reduced for PRs that affect only a subset of the codebase. - [ ] The conditional logic is covered by the `features/ci_workflow_validation.feature` BDD scenarios. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing - Related tooling to investigate: `nox-poetry`, Forgejo workflow path filters (`paths:` / `paths-ignore:` triggers), and job-level `if:` conditions. ## Subtasks - [ ] Investigate using Forgejo workflow `paths:` / `paths-ignore:` trigger filters to skip entire workflow runs for documentation-only changes. - [ ] Investigate using a tool like `nox-poetry` or custom scripting to identify changed files within a pull request and map them to relevant nox sessions. - [ ] Design the conditional execution strategy and document the decision in an ADR if it constitutes an architectural change. - [ ] Modify `.forgejo/workflows/ci.yml` to selectively run the corresponding test sessions based on the changed files. - [ ] Update `features/ci_workflow_validation.feature` to assert conditional execution paths are present and correct. - [ ] Tests (Behave): Add scenarios covering the conditional test execution logic. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - The CI pipeline only runs the tests that are relevant to the changes in a pull request. - The overall build time is reduced for pull requests that only affect a subset of the codebase. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 00:03:00 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1870
No description provided.