TEST-INFRA: [ci-pipeline-design] Consolidate integration_tests and slow_integration_tests nox sessions #1777

Open
opened 2026-04-02 23:47:44 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-consolidate-integration-test-sessions
  • Commit Message: chore(nox): consolidate integration_tests and slow_integration_tests into a single configurable session
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The noxfile.py contains two separate nox sessions for integration tests: integration_tests and slow_integration_tests. The slow_integration_tests session is a subset of the integration_tests session, with the only difference being that it doesn't exclude the slow tag. This can be simplified by having a single integration_tests session that can be configured to include or exclude slow tests via a command-line argument.

Consolidating these sessions reduces maintenance overhead, eliminates duplication in the nox configuration, and makes the CI pipeline easier to reason about.

Current Behavior

Two separate nox sessions exist:

  • integration_tests — runs integration tests excluding those tagged slow
  • slow_integration_tests — runs integration tests including those tagged slow

These sessions are nearly identical, differing only in whether the slow tag is excluded.

Expected Behavior

A single integration_tests nox session exists that accepts a --runslow (or similar) command-line flag to control whether slow tests are included. The CI configuration is updated to invoke the session with the appropriate flag when slow tests should be run.

Acceptance Criteria

  • The slow_integration_tests nox session is removed from noxfile.py.
  • The integration_tests session accepts a --runslow (or similar) argument to include slow tests.
  • The CI configuration correctly invokes integration_tests with and without the slow flag as appropriate.
  • All existing integration tests (both fast and slow) continue to pass.
  • No regressions are introduced in test coverage or CI behaviour.

Supporting Information

  • Related Epic: #1678 (CI Execution Time Optimization)
  • See also: noxfile.py in the repository root for the current session definitions.

Subtasks

  • Merge the slow_integration_tests session into the integration_tests session.
  • Add a --runslow (or similar) command-line argument to the integration_tests session to control the inclusion of slow tests.
  • Update the CI configuration to use the new command-line argument for running slow tests.
  • Update any documentation or developer guides that reference slow_integration_tests.
  • Verify coverage >= 97% via nox -s coverage_report.
  • Run nox (all default sessions) and fix any errors.

Definition of Done

  • The slow_integration_tests nox session is removed.
  • The integration_tests session can run both fast and slow tests based on a command-line argument.
  • The CI pipeline is updated to use the new consolidated session.
  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-consolidate-integration-test-sessions` - **Commit Message**: `chore(nox): consolidate integration_tests and slow_integration_tests into a single configurable session` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `noxfile.py` contains two separate nox sessions for integration tests: `integration_tests` and `slow_integration_tests`. The `slow_integration_tests` session is a subset of the `integration_tests` session, with the only difference being that it doesn't exclude the `slow` tag. This can be simplified by having a single `integration_tests` session that can be configured to include or exclude slow tests via a command-line argument. Consolidating these sessions reduces maintenance overhead, eliminates duplication in the nox configuration, and makes the CI pipeline easier to reason about. ## Current Behavior Two separate nox sessions exist: - `integration_tests` — runs integration tests excluding those tagged `slow` - `slow_integration_tests` — runs integration tests including those tagged `slow` These sessions are nearly identical, differing only in whether the `slow` tag is excluded. ## Expected Behavior A single `integration_tests` nox session exists that accepts a `--runslow` (or similar) command-line flag to control whether slow tests are included. The CI configuration is updated to invoke the session with the appropriate flag when slow tests should be run. ## Acceptance Criteria - The `slow_integration_tests` nox session is removed from `noxfile.py`. - The `integration_tests` session accepts a `--runslow` (or similar) argument to include slow tests. - The CI configuration correctly invokes `integration_tests` with and without the slow flag as appropriate. - All existing integration tests (both fast and slow) continue to pass. - No regressions are introduced in test coverage or CI behaviour. ## Supporting Information - Related Epic: #1678 (CI Execution Time Optimization) - See also: `noxfile.py` in the repository root for the current session definitions. ## Subtasks - [ ] Merge the `slow_integration_tests` session into the `integration_tests` session. - [ ] Add a `--runslow` (or similar) command-line argument to the `integration_tests` session to control the inclusion of slow tests. - [ ] Update the CI configuration to use the new command-line argument for running slow tests. - [ ] Update any documentation or developer guides that reference `slow_integration_tests`. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions) and fix any errors. ## Definition of Done - [ ] The `slow_integration_tests` nox session is removed. - [ ] The `integration_tests` session can run both fast and slow tests based on a command-line argument. - [ ] The CI pipeline is updated to use the new consolidated session. - [ ] All subtasks above are completed and checked off. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All nox stages pass. - [ ] Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:49:23 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1777
No description provided.