TEST-INFRA: [ci-pipeline-design] Implement conditional test execution #1984

New issue

Open

opened 2026-04-03 00:30:56 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 00:30:56 +00:00

Owner

Metadata

Branch: task/ci-conditional-test-execution
Commit Message: chore(ci): implement conditional test execution based on changed file paths
Milestone: v3.8.0
Parent Epic: #1678

Background and Context

The current CI pipeline runs the full test suite on every push and pull request, regardless of which files were changed. This means that a documentation-only change, a configuration tweak, or a minor comment fix triggers the same expensive end-to-end test run as a core logic change. As the project grows and the test suite expands, this unconditional execution model increasingly wastes CI runner time and slows developer feedback loops.

Implementing conditional test execution — where specific test jobs are skipped or scoped based on the paths of changed files — will significantly reduce unnecessary CI load while preserving full coverage for changes that actually affect tested code.

Expected Behavior

The CI pipeline should inspect the set of changed files in each push or pull request and use that information to conditionally skip or restrict test jobs:

Changes confined to documentation files (e.g., docs/**, *.md) should skip unit, integration, and e2e test jobs.
Changes confined to CI configuration files (e.g., .forgejo/workflows/**) should trigger only a CI-validation job, not the full test suite.
Changes to source code (src/**) or test files (tests/**, features/**) should always trigger the full test suite.
A fallback rule must ensure that when path detection is ambiguous or unavailable (e.g., on a forced push or initial commit), the full test suite runs.

Acceptance Criteria

The CI workflow uses path filters or a path-detection step to determine which categories of files changed.
At least the following conditional skip rules are implemented:
- Documentation-only changes skip unit_tests, integration_tests, and e2e_tests.
- CI-config-only changes skip unit_tests, integration_tests, and e2e_tests but run ci_validation.
The fallback (full suite) rule is in place and documented in the workflow file via comments.
No existing test job is permanently removed — all jobs remain present but may be conditionally skipped.
The status_check job (or equivalent gate job) correctly reflects skipped jobs as passing so that PRs are not blocked by intentionally skipped jobs.

Supporting Information

Related Epic: #1678 (CI Execution Time Optimization)
Related issue: #1973 (Implement dependency caching) — both target CI execution time reduction
Forgejo Actions supports paths and paths-ignore filters on workflow triggers, as well as if: conditions on individual jobs using github.event.head_commit path data or a dedicated path-filter action.
The status_check job pattern used in .forgejo/workflows/ci.yml must be updated to treat skipped jobs as success.

Subtasks

Audit .forgejo/workflows/ci.yml to identify all jobs that can be conditionally skipped.
Add a paths-filter step (or equivalent) at the start of the workflow to detect changed file categories.
Implement if: conditions on unit_tests, integration_tests, and e2e_tests jobs to skip when only docs or CI config changed.
Update the status_check (gate) job to treat skipped dependent jobs as passing.
Add comments in the workflow file documenting the skip logic and fallback rules.
Update features/ci_workflow_validation.feature to assert that conditional skip conditions are present in the workflow.
Run nox (all default sessions) and fix any errors.
Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
Conditional test execution is active in the CI pipeline and verified with at least one documentation-only push that skips the test jobs.
The status_check gate passes correctly for both full-suite and skipped-suite runs.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-conditional-test-execution` - **Commit Message**: `chore(ci): implement conditional test execution based on changed file paths` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The current CI pipeline runs the full test suite on every push and pull request, regardless of which files were changed. This means that a documentation-only change, a configuration tweak, or a minor comment fix triggers the same expensive end-to-end test run as a core logic change. As the project grows and the test suite expands, this unconditional execution model increasingly wastes CI runner time and slows developer feedback loops. Implementing conditional test execution — where specific test jobs are skipped or scoped based on the paths of changed files — will significantly reduce unnecessary CI load while preserving full coverage for changes that actually affect tested code. ## Expected Behavior The CI pipeline should inspect the set of changed files in each push or pull request and use that information to conditionally skip or restrict test jobs: - Changes confined to documentation files (e.g., `docs/**`, `*.md`) should skip unit, integration, and e2e test jobs. - Changes confined to CI configuration files (e.g., `.forgejo/workflows/**`) should trigger only a CI-validation job, not the full test suite. - Changes to source code (`src/**`) or test files (`tests/**`, `features/**`) should always trigger the full test suite. - A fallback rule must ensure that when path detection is ambiguous or unavailable (e.g., on a forced push or initial commit), the full test suite runs. ## Acceptance Criteria - The CI workflow uses path filters or a path-detection step to determine which categories of files changed. - At least the following conditional skip rules are implemented: - Documentation-only changes skip `unit_tests`, `integration_tests`, and `e2e_tests`. - CI-config-only changes skip `unit_tests`, `integration_tests`, and `e2e_tests` but run `ci_validation`. - The fallback (full suite) rule is in place and documented in the workflow file via comments. - No existing test job is permanently removed — all jobs remain present but may be conditionally skipped. - The `status_check` job (or equivalent gate job) correctly reflects skipped jobs as passing so that PRs are not blocked by intentionally skipped jobs. ## Supporting Information - Related Epic: #1678 (CI Execution Time Optimization) - Related issue: #1973 (Implement dependency caching) — both target CI execution time reduction - Forgejo Actions supports `paths` and `paths-ignore` filters on workflow triggers, as well as `if:` conditions on individual jobs using `github.event.head_commit` path data or a dedicated path-filter action. - The `status_check` job pattern used in `.forgejo/workflows/ci.yml` must be updated to treat skipped jobs as success. ## Subtasks - [ ] Audit `.forgejo/workflows/ci.yml` to identify all jobs that can be conditionally skipped. - [ ] Add a `paths-filter` step (or equivalent) at the start of the workflow to detect changed file categories. - [ ] Implement `if:` conditions on `unit_tests`, `integration_tests`, and `e2e_tests` jobs to skip when only docs or CI config changed. - [ ] Update the `status_check` (gate) job to treat skipped dependent jobs as passing. - [ ] Add comments in the workflow file documenting the skip logic and fallback rules. - [ ] Update `features/ci_workflow_validation.feature` to assert that conditional skip conditions are present in the workflow. - [ ] Run `nox` (all default sessions) and fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - Conditional test execution is active in the CI pipeline and verified with at least one documentation-only push that skips the test jobs. - The `status_check` gate passes correctly for both full-suite and skipped-suite runs. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 00:31:11 +00:00

freemo added a new dependency

2026-04-03 00:32:57 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 02:42:52 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: High — Running the full test suite on documentation-only changes wastes significant CI resources and slows developer feedback. This is a high-impact infrastructure improvement.
Milestone: v3.8.0 (Server Implementation — already assigned. CI infrastructure improvements are correctly scoped here.)
MoSCoW: Could Have — Conditional test execution is a CI optimization that improves developer experience but does not block any milestone deliverables. The full test suite still runs correctly; this just avoids unnecessary runs. Should be addressed when there is bandwidth.
Parent Epic: #1678 (CI Execution Time Optimization — already linked, confirmed correct)

Well-documented with clear skip rules, fallback behavior, and status_check gate considerations.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High — Running the full test suite on documentation-only changes wastes significant CI resources and slows developer feedback. This is a high-impact infrastructure improvement. - **Milestone**: v3.8.0 (Server Implementation — already assigned. CI infrastructure improvements are correctly scoped here.) - **MoSCoW**: Could Have — Conditional test execution is a CI optimization that improves developer experience but does not block any milestone deliverables. The full test suite still runs correctly; this just avoids unnecessary runs. Should be addressed when there is bandwidth. - **Parent Epic**: #1678 (CI Execution Time Optimization — already linked, confirmed correct) Well-documented with clear skip rules, fallback behavior, and status_check gate considerations. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Could have

label

2026-04-03 02:43:13 +00:00

freemo referenced this issue

2026-04-03 10:06:42 +00:00

TEST-INFRA: [ci-pipeline-design] Implement Conditional Test Execution to Run Only Affected Tests #2232

freemo referenced this issue

2026-04-03 10:12:33 +00:00

[Automated] Product Build Session State — v3.7.0 Production Sprint #2006

freemo referenced this issue