TEST-INFRA: [ci-pipeline-design] Conditionally execute e2e_tests #1929

Open
opened 2026-04-03 00:15:05 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: task/ci-conditionally-execute-e2e-tests
  • Commit Message: chore(ci): conditionally execute e2e_tests job only when src/ or robot/e2e files change
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The e2e_tests job in .forgejo/workflows/ci.yml is run on every pull request, regardless of which files were changed. End-to-end tests are expensive in terms of CI execution time. When a pull request only touches documentation, configuration, or unrelated modules, running the full e2e suite provides no additional signal and wastes runner capacity.

By adding path-based conditions to the e2e_tests job, the CI pipeline can skip the job entirely when no relevant files have changed, reducing wall-clock time for the majority of pull requests.

Expected Behavior

The e2e_tests job should only execute when files in the src/ or robot/e2e directories are modified in a pull request. All other pull requests should skip the job without failing the pipeline.

Acceptance Criteria

  • The e2e_tests job in .forgejo/workflows/ci.yml is configured with path-based conditions (e.g., paths: filter or an if: expression using github.event.pull_request.changed_files or equivalent Forgejo CI mechanism).
  • The job runs when files under src/ are changed.
  • The job runs when files under robot/e2e are changed.
  • The job is skipped (not failed) when neither of the above paths are changed.
  • features/ci_workflow_validation.feature is updated to assert the conditional execution behaviour.
  • No regression is introduced to the existing CI pipeline behaviour.

Subtasks

  • Investigate the Forgejo CI mechanism for path-based job conditions (e.g., paths: trigger filter or if: expression with changed-files detection)
  • Update the e2e_tests job in .forgejo/workflows/ci.yml to run only when files in src/ or robot/e2e are changed
  • Verify the job is correctly skipped when unrelated files are changed and correctly triggered when relevant files are changed
  • Update features/ci_workflow_validation.feature to assert the new conditional execution behaviour
  • Run nox (all default sessions) locally and fix any errors introduced
  • Verify coverage >= 97% via nox -s coverage_report

Definition of Done

  • The e2e_tests job in .forgejo/workflows/ci.yml has path-based conditional execution configured
  • The job runs when src/ or robot/e2e files are changed, and is skipped otherwise
  • features/ci_workflow_validation.feature is updated to cover the conditional execution logic
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-conditionally-execute-e2e-tests` - **Commit Message**: `chore(ci): conditionally execute e2e_tests job only when src/ or robot/e2e files change` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `e2e_tests` job in `.forgejo/workflows/ci.yml` is run on every pull request, regardless of which files were changed. End-to-end tests are expensive in terms of CI execution time. When a pull request only touches documentation, configuration, or unrelated modules, running the full e2e suite provides no additional signal and wastes runner capacity. By adding path-based conditions to the `e2e_tests` job, the CI pipeline can skip the job entirely when no relevant files have changed, reducing wall-clock time for the majority of pull requests. ## Expected Behavior The `e2e_tests` job should only execute when files in the `src/` or `robot/e2e` directories are modified in a pull request. All other pull requests should skip the job without failing the pipeline. ## Acceptance Criteria - The `e2e_tests` job in `.forgejo/workflows/ci.yml` is configured with path-based conditions (e.g., `paths:` filter or an `if:` expression using `github.event.pull_request.changed_files` or equivalent Forgejo CI mechanism). - The job runs when files under `src/` are changed. - The job runs when files under `robot/e2e` are changed. - The job is skipped (not failed) when neither of the above paths are changed. - `features/ci_workflow_validation.feature` is updated to assert the conditional execution behaviour. - No regression is introduced to the existing CI pipeline behaviour. ## Subtasks - [ ] Investigate the Forgejo CI mechanism for path-based job conditions (e.g., `paths:` trigger filter or `if:` expression with changed-files detection) - [ ] Update the `e2e_tests` job in `.forgejo/workflows/ci.yml` to run only when files in `src/` or `robot/e2e` are changed - [ ] Verify the job is correctly skipped when unrelated files are changed and correctly triggered when relevant files are changed - [ ] Update `features/ci_workflow_validation.feature` to assert the new conditional execution behaviour - [ ] Run `nox` (all default sessions) locally and fix any errors introduced - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done - [ ] The `e2e_tests` job in `.forgejo/workflows/ci.yml` has path-based conditional execution configured - [ ] The job runs when `src/` or `robot/e2e` files are changed, and is skipped otherwise - [ ] `features/ci_workflow_validation.feature` is updated to cover the conditional execution logic - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 00:15:16 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1929
No description provided.