TEST-INFRA: [test-architecture] Improve organization of Robot and Benchmark tests #4436

Open
opened 2026-04-08 12:35:56 +00:00 by HAL9000 · 0 comments
Owner

Metadata

  • Branch: task/test-architecture-reorganize-robot-benchmark-tests
  • Commit Message: refactor(tests): reorganize Robot and ASV benchmark tests into feature/milestone subdirectories
  • Milestone: N/A — backlog (see note below)
  • Parent Epic: #1678

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Description

The current organization of Robot Framework tests in robot/ and ASV benchmark tests in benchmarks/ is a flat list of files. This makes it difficult to navigate the codebase and find the tests related to a specific feature, milestone, or workflow.

Proposed Improvements

  1. Group Robot tests by milestone and workflow: Create subdirectories within robot/e2e for each milestone and workflow. For example, robot/e2e/milestone1/, robot/e2e/milestone2/, robot/e2e/workflows/wf04/, etc.
  2. Group benchmark tests by feature: Create subdirectories within benchmarks that correspond to the major feature areas, similar to the proposed improvement for the Behave step definitions.

Subtasks

  • Analyze the existing Robot tests to identify logical groupings based on milestones and workflows.
  • Create a hierarchical directory structure within robot/ and robot/e2e/.
  • Move the existing Robot test files into the appropriate subdirectories.
  • Analyze the existing benchmark tests to identify logical groupings based on features.
  • Create a hierarchical directory structure within benchmarks/.
  • Move the existing benchmark test files into the appropriate subdirectories.
  • Update the test runner configurations (noxfile.py, CI workflow) to ensure the new test organization is correctly recognized.
  • Run the full test suite to verify that all tests still pass after the refactoring.

Definition of Done

  • All Robot tests are organized into subdirectories that correspond to their milestones and workflows.
  • All benchmark tests are organized into subdirectories that correspond to their feature areas.
  • The test suite runs successfully with the new test organization (nox -e integration_tests and nox pass).
  • The CONTRIBUTING.md document is updated to reflect the new guidelines for organizing Robot and benchmark tests.
  • All nox stages pass.
  • Coverage >= 97%

Duplicate Check

  • Search queries used: "robot test organization", "benchmark test organization", "refactor robot tests"
  • Number of results found for each query: 0
  • Why none of the existing issues cover this specific finding: No existing issues address the organization of Robot or benchmark tests.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `task/test-architecture-reorganize-robot-benchmark-tests` - **Commit Message**: `refactor(tests): reorganize Robot and ASV benchmark tests into feature/milestone subdirectories` - **Milestone**: N/A — backlog (see note below) - **Parent Epic**: #1678 > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. ## Description The current organization of Robot Framework tests in `robot/` and ASV benchmark tests in `benchmarks/` is a flat list of files. This makes it difficult to navigate the codebase and find the tests related to a specific feature, milestone, or workflow. ## Proposed Improvements 1. **Group Robot tests by milestone and workflow:** Create subdirectories within `robot/e2e` for each milestone and workflow. For example, `robot/e2e/milestone1/`, `robot/e2e/milestone2/`, `robot/e2e/workflows/wf04/`, etc. 2. **Group benchmark tests by feature:** Create subdirectories within `benchmarks` that correspond to the major feature areas, similar to the proposed improvement for the Behave step definitions. ## Subtasks - [ ] Analyze the existing Robot tests to identify logical groupings based on milestones and workflows. - [ ] Create a hierarchical directory structure within `robot/` and `robot/e2e/`. - [ ] Move the existing Robot test files into the appropriate subdirectories. - [ ] Analyze the existing benchmark tests to identify logical groupings based on features. - [ ] Create a hierarchical directory structure within `benchmarks/`. - [ ] Move the existing benchmark test files into the appropriate subdirectories. - [ ] Update the test runner configurations (noxfile.py, CI workflow) to ensure the new test organization is correctly recognized. - [ ] Run the full test suite to verify that all tests still pass after the refactoring. ## Definition of Done - [ ] All Robot tests are organized into subdirectories that correspond to their milestones and workflows. - [ ] All benchmark tests are organized into subdirectories that correspond to their feature areas. - [ ] The test suite runs successfully with the new test organization (`nox -e integration_tests` and `nox` pass). - [ ] The `CONTRIBUTING.md` document is updated to reflect the new guidelines for organizing Robot and benchmark tests. - [ ] All nox stages pass. - [ ] Coverage >= 97% ## Duplicate Check - Search queries used: "robot test organization", "benchmark test organization", "refactor robot tests" - Number of results found for each query: 0 - Why none of the existing issues cover this specific finding: No existing issues address the organization of Robot or benchmark tests. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator
HAL9000 self-assigned this 2026-04-08 12:38:45 +00:00
HAL9000 added this to the v3.5.0 milestone 2026-04-08 17:42:27 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#4436
No description provided.