TEST-INFRA: [ci-pipeline-design] Implement conditional test execution based on code changes #1793

Open
opened 2026-04-02 23:51:34 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/v3.8.0-conditional-test-execution
  • Commit Message: feat(ci): implement conditional test execution based on changed files
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The current CI pipeline runs all tests on every commit, regardless of which files were changed. This is inefficient and results in unnecessarily long CI build times, especially for small, isolated changes. Implementing a strategy to selectively run only the tests relevant to the changed code will significantly reduce CI feedback time and runner resource usage.

Expected Behavior

The CI workflow should analyse the git diff on each run, map changed source files to their corresponding test files, and execute only the relevant subset of tests. For changes that cannot be mapped (e.g. configuration files, CI workflow files), the full test suite should still run as a safe fallback.

Acceptance Criteria

  • The CI workflow only runs tests for the code that has changed.
  • The CI build time is significantly reduced for small, isolated changes.
  • A safe fallback to the full test suite is in place for unmapped or cross-cutting changes.

Supporting Information

  • Related Epic: #1678 (CI Execution Time Optimization)
  • Related issues: #1745 (conditional nox test execution via Behave tags), #1752 (investigate conditional test execution)

Subtasks

  • Implement a script to analyse the git diff and identify the changed source files.
  • Implement a mechanism to map changed source files to their relevant test files.
  • Modify the CI workflow to invoke this mechanism and run only the relevant tests.
  • Add a fallback to run the full test suite when no mapping can be determined.
  • Verify that the correct tests are being run for representative change sets.
  • Measure and document the impact on CI build time.
  • Update features/ci_workflow_validation.feature to assert conditional execution behaviour.
  • Run all nox sessions and fix any errors.
  • Verify coverage >= 97%.

Definition of Done

  • A script exists that analyses the git diff and maps changed files to test files.
  • The CI workflow uses this mapping to run only the relevant tests.
  • A full-suite fallback is implemented and tested.
  • features/ci_workflow_validation.feature covers the new conditional execution steps.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/v3.8.0-conditional-test-execution` - **Commit Message**: `feat(ci): implement conditional test execution based on changed files` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline runs all tests on every commit, regardless of which files were changed. This is inefficient and results in unnecessarily long CI build times, especially for small, isolated changes. Implementing a strategy to selectively run only the tests relevant to the changed code will significantly reduce CI feedback time and runner resource usage. ## Expected Behavior The CI workflow should analyse the git diff on each run, map changed source files to their corresponding test files, and execute only the relevant subset of tests. For changes that cannot be mapped (e.g. configuration files, CI workflow files), the full test suite should still run as a safe fallback. ## Acceptance Criteria - [ ] The CI workflow only runs tests for the code that has changed. - [ ] The CI build time is significantly reduced for small, isolated changes. - [ ] A safe fallback to the full test suite is in place for unmapped or cross-cutting changes. ## Supporting Information - Related Epic: #1678 (CI Execution Time Optimization) - Related issues: #1745 (conditional nox test execution via Behave tags), #1752 (investigate conditional test execution) ## Subtasks - [ ] Implement a script to analyse the git diff and identify the changed source files. - [ ] Implement a mechanism to map changed source files to their relevant test files. - [ ] Modify the CI workflow to invoke this mechanism and run only the relevant tests. - [ ] Add a fallback to run the full test suite when no mapping can be determined. - [ ] Verify that the correct tests are being run for representative change sets. - [ ] Measure and document the impact on CI build time. - [ ] Update `features/ci_workflow_validation.feature` to assert conditional execution behaviour. - [ ] Run all `nox` sessions and fix any errors. - [ ] Verify coverage >= 97%. ## Definition of Done - [ ] A script exists that analyses the git diff and maps changed files to test files. - [ ] The CI workflow uses this mapping to run only the relevant tests. - [ ] A full-suite fallback is implemented and tested. - [ ] `features/ci_workflow_validation.feature` covers the new conditional execution steps. - [ ] All nox stages pass. - [ ] Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:51:52 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1793
No description provided.