TEST-INFRA: [test-architecture] Reorganize test suite for modularity and clarity #1670

New issue

Open

opened 2026-04-02 23:27:15 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:27:15 +00:00

Owner

Metadata

Branch: refactor/test-suite-modular-reorganization
Commit Message: refactor(tests): reorganize test suite into per-module subdirectories for modularity
Milestone: v3.7.0
Parent Epic: (none identified — see orphan notice below)

Background and Context

The current test suite is not organized in a modular fashion, making it difficult to assess test coverage for individual modules. The features/, robot/, and benchmarks/ directories contain a flat list of test files with no clear mapping to the modules they test in src/cleveragents/.

This lack of organization hinders our ability to identify and address gaps in our testing strategy. It also makes the test suite harder to maintain and navigate, and conflicts with the project's modular architecture as described in docs/specification.md.

Current Behavior

Test files in features/, robot/, and benchmarks/ are organized as a flat list with no per-module subdirectory structure. There is no clear mapping between a test file and the src/cleveragents module it exercises.

Expected Behavior

Each module under src/cleveragents/ has a corresponding subdirectory in features/, robot/, and benchmarks/. All test files are co-located under their respective module's subdirectory, making coverage gaps immediately visible and the suite easy to navigate.

Subtasks

Audit all existing test files in features/, robot/, and benchmarks/ and map each to its corresponding module in src/cleveragents/.
Create per-module subdirectories in features/, robot/, and benchmarks/ for each module in src/cleveragents/.
Move all existing test files into their corresponding module subdirectory.
Update behave.ini, robot.cfg, and asv.conf.json (and any other relevant configuration) to reflect the new directory structure.
Update features/mocks/ references as needed to remain consistent with the new layout.
Verify all tests still pass: nox -e unit_tests && nox -e integration_tests.
Verify coverage remains ≥ 97%: nox -e coverage_report.
Update CONTRIBUTING.md to document the new test organization guidelines and per-module subdirectory convention.
Run nox (all default sessions) and fix any errors.

Definition of Done

This issue is complete when:

The test suite is reorganized into a modular structure, with each module in src/cleveragents/ having its own subdirectory in features/, robot/, and benchmarks/.
All existing tests are moved to their appropriate subdirectories with no tests deleted or skipped.
All tool configurations (behave.ini, robot.cfg, asv.conf.json) are updated and functional.
CONTRIBUTING.md is updated to document the new test organization convention.
The CI/CD pipeline runs successfully with the new test suite structure.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox default sessions pass.
Coverage ≥ 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `refactor/test-suite-modular-reorganization` - **Commit Message**: `refactor(tests): reorganize test suite into per-module subdirectories for modularity` - **Milestone**: v3.7.0 - **Parent Epic**: _(none identified — see orphan notice below)_ ## Background and Context The current test suite is not organized in a modular fashion, making it difficult to assess test coverage for individual modules. The `features/`, `robot/`, and `benchmarks/` directories contain a flat list of test files with no clear mapping to the modules they test in `src/cleveragents/`. This lack of organization hinders our ability to identify and address gaps in our testing strategy. It also makes the test suite harder to maintain and navigate, and conflicts with the project's modular architecture as described in `docs/specification.md`. ## Current Behavior Test files in `features/`, `robot/`, and `benchmarks/` are organized as a flat list with no per-module subdirectory structure. There is no clear mapping between a test file and the `src/cleveragents` module it exercises. ## Expected Behavior Each module under `src/cleveragents/` has a corresponding subdirectory in `features/`, `robot/`, and `benchmarks/`. All test files are co-located under their respective module's subdirectory, making coverage gaps immediately visible and the suite easy to navigate. ## Subtasks - [ ] Audit all existing test files in `features/`, `robot/`, and `benchmarks/` and map each to its corresponding module in `src/cleveragents/`. - [ ] Create per-module subdirectories in `features/`, `robot/`, and `benchmarks/` for each module in `src/cleveragents/`. - [ ] Move all existing test files into their corresponding module subdirectory. - [ ] Update `behave.ini`, `robot.cfg`, and `asv.conf.json` (and any other relevant configuration) to reflect the new directory structure. - [ ] Update `features/mocks/` references as needed to remain consistent with the new layout. - [ ] Verify all tests still pass: `nox -e unit_tests && nox -e integration_tests`. - [ ] Verify coverage remains ≥ 97%: `nox -e coverage_report`. - [ ] Update `CONTRIBUTING.md` to document the new test organization guidelines and per-module subdirectory convention. - [ ] Run `nox` (all default sessions) and fix any errors. ## Definition of Done This issue is complete when: - [ ] The test suite is reorganized into a modular structure, with each module in `src/cleveragents/` having its own subdirectory in `features/`, `robot/`, and `benchmarks/`. - [ ] All existing tests are moved to their appropriate subdirectories with no tests deleted or skipped. - [ ] All tool configurations (`behave.ini`, `robot.cfg`, `asv.conf.json`) are updated and functional. - [ ] `CONTRIBUTING.md` is updated to document the new test organization convention. - [ ] The CI/CD pipeline runs successfully with the new test suite structure. - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - [ ] All `nox` default sessions pass. - [ ] Coverage ≥ 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-02 23:28:04 +00:00

freemo commented

2026-04-02 23:28:50 +00:00

Author

Owner

⚠️ Orphan Issue — Manual Linking Required

This issue was created automatically and no parent Epic with Type/Epic label was found in the repository for the test-architecture / missing-test-levels focus area.

Per CONTRIBUTING.md, all issues (except Epics and Legendaries) must be linked to a parent Epic. A project maintainer should either:

Identify an existing Epic that covers test suite architecture/organization and link this issue to it as a child (this issue blocks the parent Epic), or
Create a new Type/Epic issue for "TEST-INFRA: Test Suite Architecture & Modularization" and link this issue as a child of that Epic.

To create the dependency link once a parent Epic is identified:

curl -s -X POST "https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/issues/1670/blocks" \
  -H "Authorization: token <FORGEJO_PAT>" \
  -H "Content-Type: application/json" \
  -d '{"dependency_id": <PARENT_EPIC_NUMBER>}'

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

⚠️ **Orphan Issue — Manual Linking Required** This issue was created automatically and no parent Epic with `Type/Epic` label was found in the repository for the `test-architecture` / `missing-test-levels` focus area. Per `CONTRIBUTING.md`, all issues (except Epics and Legendaries) **must** be linked to a parent Epic. A project maintainer should either: 1. Identify an existing Epic that covers test suite architecture/organization and link this issue to it as a child (this issue **blocks** the parent Epic), **or** 2. Create a new `Type/Epic` issue for "TEST-INFRA: Test Suite Architecture & Modularization" and link this issue as a child of that Epic. To create the dependency link once a parent Epic is identified: ```bash curl -s -X POST "https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/issues/1670/blocks" \ -H "Authorization: token <FORGEJO_PAT>" \ -H "Content-Type: application/json" \ -d '{"dependency_id": <PARENT_EPIC_NUMBER>}' ``` --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo referenced this issue

2026-04-02 23:34:23 +00:00

TEST-INFRA: [test-architecture] Organize Robot Framework tests into subdirectories #1717

freemo referenced this issue

2026-04-02 23:37:09 +00:00

TEST-INFRA: [test-architecture] Organize Robot Framework tests into subdirectories #1717