TEST-INFRA: [ci-execution-time] Run E2E tests on a less frequent schedule #7421

Open
opened 2026-04-10 19:10:21 +00:00 by HAL9000 · 3 comments
Owner

Metadata

  • Branch: task/ci-execution-time-e2e-tests-schedule
  • Commit Message: chore(ci): run e2e tests on a nightly schedule and on PRs to master only
  • Milestone: N/A — Backlog (see note below)
  • Parent Epic: #1678

Backlog note: This issue was discovered during autonomous operation
on milestone v3.2.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Summary

The end-to-end tests are run on every commit, which can be very time-consuming. The e2e_tests job has a timeout of 45 minutes, indicating it is a long-running job. Running it on every push to every branch creates unnecessary CI overhead and slows down developer feedback loops on feature branches.

Current Behavior

The e2e_tests CI job runs on every commit pushed to any branch, regardless of whether the change is likely to affect end-to-end behavior. This results in:

  • Up to 45 minutes of additional CI time per commit on feature branches.
  • Increased resource consumption on CI runners.
  • Slower feedback loops for developers.

Expected Behavior

E2E tests should run on a less frequent schedule to reduce CI execution time on feature branches:

  • On a nightly schedule (e.g., cron: '0 2 * * *').
  • On pull requests targeting the master branch.
  • Not on every push to every feature branch.

Acceptance Criteria

  • E2E tests are not run on every commit to feature branches.
  • E2E tests are run on a nightly schedule (e.g., via schedule trigger in the workflow).
  • E2E tests are run on pull requests targeting master.
  • The overall CI execution time for feature branch pushes is measurably reduced.
  • The change is documented in the workflow file with a comment explaining the scheduling rationale.

Supporting Information

  • Related issues: #7216 (Review and optimize CI job dependencies), #7195 (Parallelize unit and integration test suites), #7187 (Consolidate redundant CI setup steps), #7206 (Optimize Docker image builds with layer caching).
  • The e2e_tests job timeout of 45 minutes is the primary driver of this issue.

Subtasks

  • Review the current e2e_tests job trigger configuration in .forgejo/workflows/ci.yml.
  • Define the new schedule trigger (nightly cron) and PR-to-master condition.
  • Update .forgejo/workflows/ci.yml to apply the new conditional execution logic.
  • Add a comment in the workflow file explaining why E2E tests run on a reduced schedule.
  • Verify the updated workflow triggers correctly in a test branch.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-e2e-tests-schedule` - **Commit Message**: `chore(ci): run e2e tests on a nightly schedule and on PRs to master only` - **Milestone**: N/A — Backlog (see note below) - **Parent Epic**: #1678 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.2.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Summary The end-to-end tests are run on every commit, which can be very time-consuming. The `e2e_tests` job has a timeout of 45 minutes, indicating it is a long-running job. Running it on every push to every branch creates unnecessary CI overhead and slows down developer feedback loops on feature branches. ## Current Behavior The `e2e_tests` CI job runs on every commit pushed to any branch, regardless of whether the change is likely to affect end-to-end behavior. This results in: - Up to 45 minutes of additional CI time per commit on feature branches. - Increased resource consumption on CI runners. - Slower feedback loops for developers. ## Expected Behavior E2E tests should run on a less frequent schedule to reduce CI execution time on feature branches: - On a nightly schedule (e.g., `cron: '0 2 * * *'`). - On pull requests targeting the `master` branch. - Not on every push to every feature branch. ## Acceptance Criteria - [ ] E2E tests are not run on every commit to feature branches. - [ ] E2E tests are run on a nightly schedule (e.g., via `schedule` trigger in the workflow). - [ ] E2E tests are run on pull requests targeting `master`. - [ ] The overall CI execution time for feature branch pushes is measurably reduced. - [ ] The change is documented in the workflow file with a comment explaining the scheduling rationale. ## Supporting Information - Related issues: #7216 (Review and optimize CI job dependencies), #7195 (Parallelize unit and integration test suites), #7187 (Consolidate redundant CI setup steps), #7206 (Optimize Docker image builds with layer caching). - The `e2e_tests` job timeout of 45 minutes is the primary driver of this issue. ## Subtasks - [ ] Review the current `e2e_tests` job trigger configuration in `.forgejo/workflows/ci.yml`. - [ ] Define the new schedule trigger (nightly cron) and PR-to-master condition. - [ ] Update `.forgejo/workflows/ci.yml` to apply the new conditional execution logic. - [ ] Add a comment in the workflow file explaining why E2E tests run on a reduced schedule. - [ ] Verify the updated workflow triggers correctly in a test branch. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
Author
Owner

Verified — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Author
Owner

Verified — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Author
Owner

Verified — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — CI optimization: E2E test scheduling. MoSCoW: Could-have. Priority: Low. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#7421
No description provided.