TEST-INFRA: [test-architecture] Consolidate benchmark tests into a single directory #5464

Open
opened 2026-04-09 06:55:44 +00:00 by HAL9000 · 1 comment
Owner

Metadata

  • Branch: refactor/test-infra/consolidate-benchmark-dirs
  • Commit Message: refactor(tests): consolidate benchmark tests into single benchmarks/ directory
  • Milestone: Backlog (no milestone — see backlog note below)
  • Parent Epic: #5407

Background

Currently, there are two separate directories containing benchmark tests:

  • benchmarks/ — the primary benchmark directory (referenced by asv.conf.json)
  • asv/benchmarks/ — a secondary benchmark directory containing additional benchmark files

This split is confusing and makes it difficult to manage, discover, and run performance tests. The asv.conf.json already points to benchmarks/ as the benchmark_dir, meaning the asv/benchmarks/ files are not being picked up by the ASV runner.

Proposed Solution

Consolidate all benchmark tests into the single benchmarks/ directory:

  1. Move all files from asv/benchmarks/ into benchmarks/
  2. Remove the now-empty asv/benchmarks/ directory (and asv/ if it becomes empty)
  3. Verify asv.conf.json still correctly points to benchmarks/ (it already does)
  4. Ensure no duplicate benchmark names exist after consolidation

Subtasks

  • Audit asv/benchmarks/ for files not already present in benchmarks/
  • Move unique benchmark files from asv/benchmarks/ into benchmarks/
  • Resolve any naming conflicts between the two directories
  • Remove the asv/benchmarks/ directory (and asv/ if empty)
  • Verify asv.conf.json benchmark_dir still points to benchmarks/
  • Run nox -e benchmarks (or equivalent) to confirm all benchmarks are discovered and pass
  • Update any documentation or CI references to asv/benchmarks/

Definition of Done

  • Only one benchmark directory exists: benchmarks/
  • All benchmark files previously in asv/benchmarks/ are present in benchmarks/
  • asv/benchmarks/ directory no longer exists
  • asv.conf.json correctly references benchmarks/ as benchmark_dir
  • All nox stages pass
  • Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone Cross-cutting. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `refactor/test-infra/consolidate-benchmark-dirs` - **Commit Message**: `refactor(tests): consolidate benchmark tests into single benchmarks/ directory` - **Milestone**: Backlog (no milestone — see backlog note below) - **Parent Epic**: #5407 ## Background Currently, there are two separate directories containing benchmark tests: - `benchmarks/` — the primary benchmark directory (referenced by `asv.conf.json`) - `asv/benchmarks/` — a secondary benchmark directory containing additional benchmark files This split is confusing and makes it difficult to manage, discover, and run performance tests. The `asv.conf.json` already points to `benchmarks/` as the `benchmark_dir`, meaning the `asv/benchmarks/` files are not being picked up by the ASV runner. ## Proposed Solution Consolidate all benchmark tests into the single `benchmarks/` directory: 1. Move all files from `asv/benchmarks/` into `benchmarks/` 2. Remove the now-empty `asv/benchmarks/` directory (and `asv/` if it becomes empty) 3. Verify `asv.conf.json` still correctly points to `benchmarks/` (it already does) 4. Ensure no duplicate benchmark names exist after consolidation ## Subtasks - [ ] Audit `asv/benchmarks/` for files not already present in `benchmarks/` - [ ] Move unique benchmark files from `asv/benchmarks/` into `benchmarks/` - [ ] Resolve any naming conflicts between the two directories - [ ] Remove the `asv/benchmarks/` directory (and `asv/` if empty) - [ ] Verify `asv.conf.json` `benchmark_dir` still points to `benchmarks/` - [ ] Run `nox -e benchmarks` (or equivalent) to confirm all benchmarks are discovered and pass - [ ] Update any documentation or CI references to `asv/benchmarks/` ## Definition of Done - [ ] Only one benchmark directory exists: `benchmarks/` - [ ] All benchmark files previously in `asv/benchmarks/` are present in `benchmarks/` - [ ] `asv/benchmarks/` directory no longer exists - [ ] `asv.conf.json` correctly references `benchmarks/` as `benchmark_dir` - [ ] All nox stages pass - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone Cross-cutting. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
Author
Owner

Label compliance fix applied:

  • Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#5464
No description provided.