TEST-INFRA: [missing-test-levels] Add Robot and ASV tests for the core module #1836

Open
opened 2026-04-02 23:56:37 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/core-robot-asv-tests
  • Commit Message: test(core): add Robot Framework integration tests and ASV performance benchmarks for core module
  • Milestone: v3.8.0
  • Parent Epic: #934

Background and Context

The core module is missing two of the three required test levels mandated by CONTRIBUTING.md's Multi-Level Testing Mandate: Robot Framework integration tests and ASV performance benchmarks. Every coding task must include tests at multiple levels — unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task.

Current Behavior

The core module currently lacks:

  • Robot Framework integration tests in the robot/ directory
  • ASV performance benchmarks in the benchmarks/ directory

Only unit-level BDD (Behave/Gherkin) tests exist for this module.

Expected Behavior

The core module should have full multi-level test coverage:

  • Robot Framework integration tests under robot/ exercising real component interactions
  • ASV performance benchmarks under benchmarks/ measuring key operations

Acceptance Criteria

  • Robot Framework integration tests exist under robot/ for the core module and pass via nox -s integration_tests
  • ASV performance benchmarks exist under benchmarks/ for the core module and run via the project's benchmark session
  • Test coverage for the core module is at or above 97%
  • All new tests are integrated into the CI pipeline

Supporting Information

  • Similar issues for other modules: #1780 (lsp Robot tests), #1773 (infrastructure Robot/ASV tests)
  • CONTRIBUTING.md Multi-Level Testing Mandate requires unit, integration, and performance benchmarks for all modules
  • Robot tests must exercise real component interactions — no mocking of integration-level behavior

Subtasks

  • Identify all integration-level behaviors in the core module that lack Robot Framework coverage
  • Write Robot Framework test suites under robot/ covering the core module's integration scenarios
  • Implement any required keywords and resource files under robot/
  • Identify key operations in the core module suitable for ASV performance benchmarking
  • Write ASV benchmark classes under benchmarks/ for the core module
  • Verify nox -s integration_tests passes with the new Robot suites
  • Verify nox -s coverage_report shows core module at ≥97% coverage
  • Verify nox -s typecheck passes
  • Verify nox -s lint passes
  • Run full nox (all default sessions) and fix any errors

Definition of Done

  • All subtasks above are checked off
  • All new Robot tests are written as Robot Framework integration tests under robot/
  • All new ASV benchmarks are written under benchmarks/ and conform to the ASV benchmark interface
  • No Behave/Gherkin or pytest-style tests introduced in robot/
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
  • A pull request is created, reviewed, and merged
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/core-robot-asv-tests` - **Commit Message**: `test(core): add Robot Framework integration tests and ASV performance benchmarks for core module` - **Milestone**: v3.8.0 - **Parent Epic**: #934 ## Background and Context The `core` module is missing two of the three required test levels mandated by CONTRIBUTING.md's Multi-Level Testing Mandate: Robot Framework integration tests and ASV performance benchmarks. Every coding task must include tests at multiple levels — unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task. ## Current Behavior The `core` module currently lacks: - Robot Framework integration tests in the `robot/` directory - ASV performance benchmarks in the `benchmarks/` directory Only unit-level BDD (Behave/Gherkin) tests exist for this module. ## Expected Behavior The `core` module should have full multi-level test coverage: - Robot Framework integration tests under `robot/` exercising real component interactions - ASV performance benchmarks under `benchmarks/` measuring key operations ## Acceptance Criteria - Robot Framework integration tests exist under `robot/` for the `core` module and pass via `nox -s integration_tests` - ASV performance benchmarks exist under `benchmarks/` for the `core` module and run via the project's benchmark session - Test coverage for the `core` module is at or above 97% - All new tests are integrated into the CI pipeline ## Supporting Information - Similar issues for other modules: #1780 (lsp Robot tests), #1773 (infrastructure Robot/ASV tests) - CONTRIBUTING.md Multi-Level Testing Mandate requires unit, integration, and performance benchmarks for all modules - Robot tests must exercise real component interactions — no mocking of integration-level behavior ## Subtasks - [ ] Identify all integration-level behaviors in the `core` module that lack Robot Framework coverage - [ ] Write Robot Framework test suites under `robot/` covering the `core` module's integration scenarios - [ ] Implement any required keywords and resource files under `robot/` - [ ] Identify key operations in the `core` module suitable for ASV performance benchmarking - [ ] Write ASV benchmark classes under `benchmarks/` for the `core` module - [ ] Verify `nox -s integration_tests` passes with the new Robot suites - [ ] Verify `nox -s coverage_report` shows `core` module at ≥97% coverage - [ ] Verify `nox -s typecheck` passes - [ ] Verify `nox -s lint` passes - [ ] Run full `nox` (all default sessions) and fix any errors ## Definition of Done - [ ] All subtasks above are checked off - [ ] All new Robot tests are written as Robot Framework integration tests under `robot/` - [ ] All new ASV benchmarks are written under `benchmarks/` and conform to the ASV benchmark interface - [ ] No Behave/Gherkin or pytest-style tests introduced in `robot/` - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] A pull request is created, reviewed, and merged - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:56:43 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1836
No description provided.