TEST-INFRA: [ci-execution-time] Implement reporting for slowest Behave and Robot tests #2276

Closed
opened 2026-04-03 12:38:36 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: chore/ci-execution-time-slowest-test-reporting
  • Commit Message: chore(ci): implement slowest test reporting for Behave and Robot suites
  • Milestone: v3.2.0
  • Parent Epic: #1678

Background and Context

The current test infrastructure does not provide a report of the slowest running tests. This makes it difficult to identify and optimize performance bottlenecks within the Behave (unit) and Robot (integration) test suites.

Finding Type: Test Infrastructure
Area: Test Architecture
File: noxfile.py

Expected Behavior

The CI pipeline should generate and publish a report of the top 20 slowest Behave and Robot tests as a downloadable build artifact, enabling developers to quickly identify and address performance bottlenecks in the test suites.

Acceptance Criteria

  • A report of the top 20 slowest Behave tests is generated during the CI run.
  • A report of the top 20 slowest Robot tests is generated during the CI run.
  • Both reports are available as downloadable artifacts in the CI pipeline.
  • The process for accessing and interpreting the reports is documented.

Subtasks

  • Investigate options for generating test timing reports from behave and robot. This may involve using a custom reporter or a third-party library.
  • Modify the unit_tests and integration_tests nox sessions to generate a report of the top 20 slowest tests.
  • Configure the CI pipeline to publish this report as a build artifact for easy access.
  • Document the process for accessing and interpreting the slowest test report.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

  • A report of the slowest Behave and Robot tests is generated during the CI run.
  • The report is available as a downloadable artifact in the CI pipeline.
  • The process for using the report is documented for developers.
  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `chore/ci-execution-time-slowest-test-reporting` - **Commit Message**: `chore(ci): implement slowest test reporting for Behave and Robot suites` - **Milestone**: v3.2.0 - **Parent Epic**: #1678 ## Background and Context The current test infrastructure does not provide a report of the slowest running tests. This makes it difficult to identify and optimize performance bottlenecks within the Behave (unit) and Robot (integration) test suites. **Finding Type**: Test Infrastructure **Area**: Test Architecture **File**: `noxfile.py` ## Expected Behavior The CI pipeline should generate and publish a report of the top 20 slowest Behave and Robot tests as a downloadable build artifact, enabling developers to quickly identify and address performance bottlenecks in the test suites. ## Acceptance Criteria - A report of the top 20 slowest Behave tests is generated during the CI run. - A report of the top 20 slowest Robot tests is generated during the CI run. - Both reports are available as downloadable artifacts in the CI pipeline. - The process for accessing and interpreting the reports is documented. ## Subtasks - [ ] Investigate options for generating test timing reports from `behave` and `robot`. This may involve using a custom reporter or a third-party library. - [ ] Modify the `unit_tests` and `integration_tests` nox sessions to generate a report of the top 20 slowest tests. - [ ] Configure the CI pipeline to publish this report as a build artifact for easy access. - [ ] Document the process for accessing and interpreting the slowest test report. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done - [ ] A report of the slowest Behave and Robot tests is generated during the CI run. - [ ] The report is available as a downloadable artifact in the CI pipeline. - [ ] The process for using the report is documented for developers. - [ ] All subtasks above are completed and checked off. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.2.0 milestone 2026-04-03 12:39:01 +00:00
Author
Owner

Closing as duplicate of #2315.

This issue ("Implement reporting for slowest Behave and Robot tests") is a subtask of #2315 ("Investigate and optimize slow integration_tests suite"), which already includes profiling as its first subtask. Consolidating to avoid duplicate work.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2315. This issue ("Implement reporting for slowest Behave and Robot tests") is a subtask of #2315 ("Investigate and optimize slow integration_tests suite"), which already includes profiling as its first subtask. Consolidating to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2276
No description provided.