Identify and flag slow tests across the test suite #2858

Closed
opened 2026-04-04 21:01:07 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/identify-and-flag-slow-tests
  • Commit Message: test: identify and flag slow tests across the test suite
  • Milestone: v3.7.0
  • Parent Epic: #1678

Subtasks

  • Audit all Behave unit tests in features/ and record per-test execution times
  • Audit all Robot Framework integration tests in robot/ and record per-test execution times
  • Audit all ASV benchmarks in benchmarks/ for anomalous runtimes
  • Define a threshold (e.g. >1 s for unit tests, >10 s for integration tests) for what constitutes a "slow" test
  • Tag or annotate all tests that exceed the defined threshold (e.g. via Behave tags @slow or Robot Framework tags)
  • Add a nox session (or extend an existing one) to report slow tests during CI runs
  • Document the slow-test policy and threshold values in CONTRIBUTING.md
  • Ensure the new nox session is included in the default nox run and does not break any existing quality gates

Definition of Done

  • All tests exceeding the defined slow-test threshold are tagged/annotated
  • A nox session exists that surfaces slow tests during CI and fails if any untagged test exceeds the threshold
  • Slow-test policy and thresholds are documented in CONTRIBUTING.md
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/identify-and-flag-slow-tests` - **Commit Message**: `test: identify and flag slow tests across the test suite` - **Milestone**: v3.7.0 - **Parent Epic**: #1678 ## Subtasks - [ ] Audit all Behave unit tests in `features/` and record per-test execution times - [ ] Audit all Robot Framework integration tests in `robot/` and record per-test execution times - [ ] Audit all ASV benchmarks in `benchmarks/` for anomalous runtimes - [ ] Define a threshold (e.g. >1 s for unit tests, >10 s for integration tests) for what constitutes a "slow" test - [ ] Tag or annotate all tests that exceed the defined threshold (e.g. via Behave tags `@slow` or Robot Framework tags) - [ ] Add a nox session (or extend an existing one) to report slow tests during CI runs - [ ] Document the slow-test policy and threshold values in `CONTRIBUTING.md` - [ ] Ensure the new nox session is included in the default `nox` run and does not break any existing quality gates ## Definition of Done - [ ] All tests exceeding the defined slow-test threshold are tagged/annotated - [ ] A nox session exists that surfaces slow tests during CI and fails if any untagged test exceeds the threshold - [ ] Slow-test policy and thresholds are documented in `CONTRIBUTING.md` - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-04 21:01:15 +00:00
Owner

State label reconciliation:

  • Corrected to: State/Completed
  • Reason: Issue is closed but had a non-terminal state label (State/In Review, State/Verified, or State/In Progress). CONTRIBUTING.md requires closed issues to have State/Completed or State/Wont Do.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

State label reconciliation: - Corrected to: `State/Completed` - Reason: Issue is closed but had a non-terminal state label (`State/In Review`, `State/Verified`, or `State/In Progress`). CONTRIBUTING.md requires closed issues to have `State/Completed` or `State/Wont Do`. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2858
No description provided.