TEST-INFRA: [missing-test-levels] Add Robot and ASV tests for the agents module #1829

Open
opened 2026-04-02 23:56:01 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/agents-robot-asv-tests
  • Commit Message: test(agents): add Robot integration tests and ASV benchmarks for agents module
  • Milestone: v3.8.0
  • Parent Epic: #1678

Description

The agents module in src/cleveragents/agents is missing the following test levels:

  • Robot integration tests (in robot/) — integration tests using real services and real interactions between components, with no mocks permitted per CONTRIBUTING.md
  • ASV performance benchmarks (in benchmarks/) — performance regression tracking for key agents module operations

This violates the project's Multi-Level Testing Mandate as defined in CONTRIBUTING.md:

"Every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task."

The agents module is the core entry point and command dispatch layer of CleverAgents — it provides the CLI command structure, orchestrates plan lifecycle operations, and coordinates all top-level user-facing interactions. Comprehensive test coverage at all levels is essential to ensure reliability and catch performance regressions.

Expected Behavior

The agents module has complete test coverage at all required levels:

  1. Robot Framework integration test suites in robot/ exercising real agents module interactions (no mocks)
  2. ASV performance benchmarks in benchmarks/ tracking performance of key agents module operations

Acceptance Criteria

  • Robot Framework integration test suites exist in robot/ covering the agents module's integration scenarios
  • All Robot step definitions and keyword files are fully implemented (no placeholder steps)
  • ASV performance benchmarks exist in benchmarks/ for key agents module operations (where applicable)
  • All new tests pass via the project's nox task runner
  • Test coverage for the agents module is >= 97%
  • All nox sessions pass (unit, integration, typecheck, lint, coverage)

Subtasks

  • Identify all integration-level behaviors in the agents module that lack Robot Framework coverage
  • Write Robot Framework test suites under robot/ covering the agents module's integration scenarios
  • Implement any required keywords and resource files under robot/
  • Write ASV performance benchmarks under benchmarks/ for key agents module operations
  • Verify nox -s integration_tests passes with the new Robot suites
  • Verify nox -s coverage_report shows agents module at ≥97% coverage
  • Verify nox -s typecheck passes
  • Verify nox -s lint passes
  • Run full nox (all default sessions) and fix any errors

Definition of Done

  • All subtasks above are checked off
  • All new integration tests are written as Robot Framework tests under robot/ — no Behave/Gherkin or pytest-style tests introduced in robot/
  • All new benchmarks are written as ASV benchmarks under benchmarks/
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details
  • The commit footer includes ISSUES CLOSED: #<this issue number>
  • The commit is pushed to the branch specified in the Metadata section
  • A pull request is submitted, reviewed, and merged
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/agents-robot-asv-tests` - **Commit Message**: `test(agents): add Robot integration tests and ASV benchmarks for agents module` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Description The `agents` module in `src/cleveragents/agents` is missing the following test levels: - **Robot** integration tests (in `robot/`) — integration tests using real services and real interactions between components, with no mocks permitted per CONTRIBUTING.md - **ASV** performance benchmarks (in `benchmarks/`) — performance regression tracking for key `agents` module operations This violates the project's Multi-Level Testing Mandate as defined in CONTRIBUTING.md: > *"Every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task."* The `agents` module is the core entry point and command dispatch layer of CleverAgents — it provides the CLI command structure, orchestrates plan lifecycle operations, and coordinates all top-level user-facing interactions. Comprehensive test coverage at all levels is essential to ensure reliability and catch performance regressions. ## Expected Behavior The `agents` module has complete test coverage at all required levels: 1. Robot Framework integration test suites in `robot/` exercising real `agents` module interactions (no mocks) 2. ASV performance benchmarks in `benchmarks/` tracking performance of key `agents` module operations ## Acceptance Criteria - [ ] Robot Framework integration test suites exist in `robot/` covering the `agents` module's integration scenarios - [ ] All Robot step definitions and keyword files are fully implemented (no placeholder steps) - [ ] ASV performance benchmarks exist in `benchmarks/` for key `agents` module operations (where applicable) - [ ] All new tests pass via the project's nox task runner - [ ] Test coverage for the `agents` module is >= 97% - [ ] All nox sessions pass (unit, integration, typecheck, lint, coverage) ## Subtasks - [ ] Identify all integration-level behaviors in the `agents` module that lack Robot Framework coverage - [ ] Write Robot Framework test suites under `robot/` covering the `agents` module's integration scenarios - [ ] Implement any required keywords and resource files under `robot/` - [ ] Write ASV performance benchmarks under `benchmarks/` for key `agents` module operations - [ ] Verify `nox -s integration_tests` passes with the new Robot suites - [ ] Verify `nox -s coverage_report` shows `agents` module at ≥97% coverage - [ ] Verify `nox -s typecheck` passes - [ ] Verify `nox -s lint` passes - [ ] Run full `nox` (all default sessions) and fix any errors ## Definition of Done - [ ] All subtasks above are checked off - [ ] All new integration tests are written as Robot Framework tests under `robot/` — no Behave/Gherkin or pytest-style tests introduced in `robot/` - [ ] All new benchmarks are written as ASV benchmarks under `benchmarks/` - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details - [ ] The commit footer includes `ISSUES CLOSED: #<this issue number>` - [ ] The commit is pushed to the branch specified in the Metadata section - [ ] A pull request is submitted, reviewed, and merged - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:56:28 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1829
No description provided.