TEST-INFRA: [ci-execution-time] Optimize the default nox session execution #2305

Closed
opened 2026-04-03 13:33:34 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: chore/ci-nox-session-stages
  • Commit Message: chore(ci): split nox sessions into targeted CI pipeline stages
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The default nox command in noxfile.py executes a large number of sessions: lint, format, typecheck, security_scan, dead_code, unit_tests, integration_tests, docs, build, benchmark, and coverage_report. While comprehensive, running all of these sessions on every commit leads to long CI execution times and slow developer feedback loops.

Current Behavior

Running the default nox command triggers every registered session sequentially on every commit, regardless of the nature of the change or the stage of the CI/CD pipeline. This means a simple documentation fix triggers the same full suite as a production code change, including benchmarks, security scans, and full integration tests.

Expected Behavior

The default nox sessions and the CI configuration should be restructured so that different session groups run at the appropriate stage of the pipeline:

  • Core Checks (run on every commit): lint, typecheck, unit_tests — fast feedback for all contributors.
  • Integration and E2E Checks (run on PRs or merges to master): integration_tests, e2e_tests — validate cross-component behaviour before merging.
  • Build and Deploy Checks (run on release branches): build, docs — only relevant when preparing a release.
  • Scheduled or On-demand Checks: benchmark, security_scan, dead_code, coverage_report — expensive or infrequent checks that do not need to block every commit.

Acceptance Criteria

  • The default nox sessions in noxfile.py are updated to include only the core checks (lint, typecheck, unit_tests).
  • The CI configuration is updated to run different nox session groups at the appropriate pipeline stages (per-commit, PR/merge, release, scheduled).
  • Documentation is updated to reflect the new CI pipeline structure and explain which sessions run at each stage.
  • All existing sessions remain available and can still be invoked individually or as a group.
  • CI wall-clock time for per-commit runs is measurably reduced.

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization
  • Related issue exploring session parallelisation: #2282
  • Sibling CI time optimisation issues: #2289, #2293, #2294, #2297

Subtasks

  • Audit all current default nox sessions and classify each into the appropriate pipeline stage (per-commit / PR / release / scheduled)
  • Update noxfile.py to set the default session list to only core checks (lint, typecheck, unit_tests)
  • Update the CI configuration (e.g., .forgejo/workflows/) to invoke the correct nox session groups at each pipeline stage
  • Add documentation (e.g., in docs/ or README) describing the new CI pipeline structure and session groupings
  • Update features/ci_workflow_validation.feature if the CI workflow file is modified
  • Run nox (all default sessions), fix any errors
  • Verify coverage ≥ 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (chore(ci): split nox sessions into targeted CI pipeline stages), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (chore/ci-nox-session-stages).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-nox-session-stages` - **Commit Message**: `chore(ci): split nox sessions into targeted CI pipeline stages` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The default `nox` command in `noxfile.py` executes a large number of sessions: `lint`, `format`, `typecheck`, `security_scan`, `dead_code`, `unit_tests`, `integration_tests`, `docs`, `build`, `benchmark`, and `coverage_report`. While comprehensive, running all of these sessions on every commit leads to long CI execution times and slow developer feedback loops. ## Current Behavior Running the default `nox` command triggers every registered session sequentially on every commit, regardless of the nature of the change or the stage of the CI/CD pipeline. This means a simple documentation fix triggers the same full suite as a production code change, including benchmarks, security scans, and full integration tests. ## Expected Behavior The default `nox` sessions and the CI configuration should be restructured so that different session groups run at the appropriate stage of the pipeline: - **Core Checks (run on every commit):** `lint`, `typecheck`, `unit_tests` — fast feedback for all contributors. - **Integration and E2E Checks (run on PRs or merges to master):** `integration_tests`, `e2e_tests` — validate cross-component behaviour before merging. - **Build and Deploy Checks (run on release branches):** `build`, `docs` — only relevant when preparing a release. - **Scheduled or On-demand Checks:** `benchmark`, `security_scan`, `dead_code`, `coverage_report` — expensive or infrequent checks that do not need to block every commit. ## Acceptance Criteria - [ ] The default `nox` sessions in `noxfile.py` are updated to include only the core checks (`lint`, `typecheck`, `unit_tests`). - [ ] The CI configuration is updated to run different `nox` session groups at the appropriate pipeline stages (per-commit, PR/merge, release, scheduled). - [ ] Documentation is updated to reflect the new CI pipeline structure and explain which sessions run at each stage. - [ ] All existing sessions remain available and can still be invoked individually or as a group. - [ ] CI wall-clock time for per-commit runs is measurably reduced. ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization - Related issue exploring session parallelisation: #2282 - Sibling CI time optimisation issues: #2289, #2293, #2294, #2297 ## Subtasks - [ ] Audit all current default nox sessions and classify each into the appropriate pipeline stage (per-commit / PR / release / scheduled) - [ ] Update `noxfile.py` to set the default session list to only core checks (`lint`, `typecheck`, `unit_tests`) - [ ] Update the CI configuration (e.g., `.forgejo/workflows/`) to invoke the correct nox session groups at each pipeline stage - [ ] Add documentation (e.g., in `docs/` or `README`) describing the new CI pipeline structure and session groupings - [ ] Update `features/ci_workflow_validation.feature` if the CI workflow file is modified - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`chore(ci): split nox sessions into targeted CI pipeline stages`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`chore/ci-nox-session-stages`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 13:33:53 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High (confirmed) — Restructuring nox sessions into pipeline stages is a significant CI optimization.
  • Milestone: v3.8.0 (confirmed — CI infrastructure)
  • MoSCoW: Could Have — CI pipeline stage optimization. Not blocking any deliverables but would significantly improve developer feedback loops.
  • Parent Epic: #1678 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High (confirmed) — Restructuring nox sessions into pipeline stages is a significant CI optimization. - **Milestone**: v3.8.0 (confirmed — CI infrastructure) - **MoSCoW**: Could Have — CI pipeline stage optimization. Not blocking any deliverables but would significantly improve developer feedback loops. - **Parent Epic**: #1678 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Closing as duplicate of #2293.

This issue ("Optimize the default nox session execution") overlaps significantly with #2293 ("Parallelize Nox Sessions in CI"). The primary optimization for nox session execution is parallelization, which #2293 covers. Consolidating to avoid duplicate work.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2293. This issue ("Optimize the default nox session execution") overlaps significantly with #2293 ("Parallelize Nox Sessions in CI"). The primary optimization for nox session execution is parallelization, which #2293 covers. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2305
No description provided.