TEST-INFRA: [ci-execution-time] Investigate and split 'Cli' suite #3488

Open
opened 2026-04-05 18:34:46 +00:00 by freemo · 0 comments
Owner

The 'Cli' suite takes over 68 seconds to run, making it one of the slowest suites in the CI pipeline. This slows down the feedback loop for developers.

Proposal:

  • Investigate the tests within the 'Cli' suite to identify the slowest ones.
  • Split the suite into smaller, more focused suites to improve parallelization and reduce overall execution time.

Metadata

  • Branch: test/ci-execution-time-split-cli-suite
  • Commit Message: test(ci): split slow Cli suite into focused sub-suites to reduce execution time
  • Milestone: (backlog — see note below)
  • Parent Epic: #1678

Subtasks

  • Profile the 'Cli' suite to identify the slowest individual tests and their root causes
  • Group tests by functional area (e.g., argument parsing, command dispatch, output formatting)
  • Create new focused sub-suites in features/ following Behave conventions
  • Migrate existing Cli tests into the appropriate new sub-suites
  • Verify no test coverage is lost (all scenarios still present and passing)
  • Confirm each new sub-suite runs in under 20 seconds independently
  • Update nox session configuration if suite discovery paths need adjustment
  • Update CI pipeline configuration to run new sub-suites in parallel

Definition of Done

  • The original 'Cli' suite has been split into at least two focused sub-suites
  • Each new sub-suite runs in under 20 seconds
  • Total execution time for all Cli-related tests is significantly reduced vs. the original 68+ seconds
  • All new sub-suites are well-structured and named to reflect their functional focus
  • No regression in test coverage — coverage remains >= 97%
  • All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e coverage_report)
  • Coverage >= 97%
  • A PR has been merged resolving this issue

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

The 'Cli' suite takes over 68 seconds to run, making it one of the slowest suites in the CI pipeline. This slows down the feedback loop for developers. **Proposal:** - Investigate the tests within the 'Cli' suite to identify the slowest ones. - Split the suite into smaller, more focused suites to improve parallelization and reduce overall execution time. ## Metadata - **Branch**: `test/ci-execution-time-split-cli-suite` - **Commit Message**: `test(ci): split slow Cli suite into focused sub-suites to reduce execution time` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #1678 ## Subtasks - [ ] Profile the 'Cli' suite to identify the slowest individual tests and their root causes - [ ] Group tests by functional area (e.g., argument parsing, command dispatch, output formatting) - [ ] Create new focused sub-suites in `features/` following Behave conventions - [ ] Migrate existing Cli tests into the appropriate new sub-suites - [ ] Verify no test coverage is lost (all scenarios still present and passing) - [ ] Confirm each new sub-suite runs in under 20 seconds independently - [ ] Update `nox` session configuration if suite discovery paths need adjustment - [ ] Update CI pipeline configuration to run new sub-suites in parallel ## Definition of Done - [ ] The original 'Cli' suite has been split into at least two focused sub-suites - [ ] Each new sub-suite runs in under 20 seconds - [ ] Total execution time for all Cli-related tests is significantly reduced vs. the original 68+ seconds - [ ] All new sub-suites are well-structured and named to reflect their functional focus - [ ] No regression in test coverage — coverage remains >= 97% - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e coverage_report`) - [ ] Coverage >= 97% - [ ] A PR has been merged resolving this issue > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-05 20:42:47 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#3488
No description provided.