TEST-INFRA: [ci-execution-time] Investigate and optimize slow test suite: m3-e2e-verification #3071

Open
opened 2026-04-05 04:54:45 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/m3-e2e-verification-optimize
  • Commit Message: test(ci): investigate and optimize slow m3-e2e-verification test suite
  • Milestone: v3.6.0
  • Parent Epic: #1678

Background

The test suite m3-e2e-verification is taking an excessive amount of time to run (9952 seconds, approximately 2 hours 45 minutes). This is significantly impacting the CI/CD pipeline performance and blocking timely feedback for developers. The root cause must be identified and addressed to restore acceptable CI execution times.

Subtasks

  • Profile the m3-e2e-verification test suite to identify the slowest tests and bottlenecks
  • Investigate whether the slowness is due to test logic, external dependencies, or infrastructure issues
  • Identify opportunities for parallelisation or test splitting
  • Implement optimisations (e.g., mocking slow I/O, reducing redundant setup/teardown, parallelising where safe)
  • Validate that optimised suite still provides equivalent coverage and correctness guarantees
  • Update CI configuration if concurrency or timeout settings need adjustment
  • Document findings and changes made

Definition of Done

  • All subtasks above are completed and checked off
  • The m3-e2e-verification suite execution time is reduced to an acceptable threshold (target: under 1800 seconds)
  • No regression in test coverage (coverage remains >= 97%)
  • All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e integration_tests, nox -e coverage_report)
  • Coverage >= 97%
  • The correctly formatted commit (test(ci): investigate and optimize slow m3-e2e-verification test suite) is pushed to branch test/m3-e2e-verification-optimize
  • Pull request is merged and this issue is closed

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/m3-e2e-verification-optimize` - **Commit Message**: `test(ci): investigate and optimize slow m3-e2e-verification test suite` - **Milestone**: v3.6.0 - **Parent Epic**: #1678 ## Background The test suite `m3-e2e-verification` is taking an excessive amount of time to run (**9952 seconds**, approximately 2 hours 45 minutes). This is significantly impacting the CI/CD pipeline performance and blocking timely feedback for developers. The root cause must be identified and addressed to restore acceptable CI execution times. ## Subtasks - [ ] Profile the `m3-e2e-verification` test suite to identify the slowest tests and bottlenecks - [ ] Investigate whether the slowness is due to test logic, external dependencies, or infrastructure issues - [ ] Identify opportunities for parallelisation or test splitting - [ ] Implement optimisations (e.g., mocking slow I/O, reducing redundant setup/teardown, parallelising where safe) - [ ] Validate that optimised suite still provides equivalent coverage and correctness guarantees - [ ] Update CI configuration if concurrency or timeout settings need adjustment - [ ] Document findings and changes made ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] The `m3-e2e-verification` suite execution time is reduced to an acceptable threshold (target: under 1800 seconds) - [ ] No regression in test coverage (coverage remains >= 97%) - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`, `nox -e coverage_report`) - [ ] Coverage >= 97% - [ ] The correctly formatted commit (`test(ci): investigate and optimize slow m3-e2e-verification test suite`) is pushed to branch `test/m3-e2e-verification-optimize` - [ ] Pull request is merged and this issue is closed --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.6.0 milestone 2026-04-05 04:55:06 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High (keeping existing — 2h45m CI time is severely impacting developer productivity)
  • Milestone: v3.6.0 (already assigned, keeping)
  • MoSCoW: Should Have — CI execution time directly impacts development velocity. A 2h45m test suite blocks timely feedback and slows the entire project. Reducing to <30 minutes is important for the team's ability to ship milestones.
  • Parent Epic: #1678 (CI Execution Time Optimization)

The target of <1800 seconds (30 minutes) is reasonable. This should be prioritized alongside feature work since it's a force multiplier for all other development.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High (keeping existing — 2h45m CI time is severely impacting developer productivity) - **Milestone**: v3.6.0 (already assigned, keeping) - **MoSCoW**: Should Have — CI execution time directly impacts development velocity. A 2h45m test suite blocks timely feedback and slows the entire project. Reducing to <30 minutes is important for the team's ability to ship milestones. - **Parent Epic**: #1678 (CI Execution Time Optimization) The target of <1800 seconds (30 minutes) is reasonable. This should be prioritized alongside feature work since it's a force multiplier for all other development. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#3071
No description provided.