TEST-INFRA: [missing-test-levels] Add Robot and ASV tests for the 'agents' module #2206

Open
opened 2026-04-03 09:31:13 +00:00 by freemo · 4 comments
Owner

Metadata

  • Branch: test/missing-test-levels-agents-robot-asv
  • Commit Message: test(agents): add Robot integration tests and ASV performance benchmarks
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The agents module is missing Robot Framework integration tests and ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in CONTRIBUTING.md, every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task.

The agents module is a core component of the CleverAgents system, responsible for actor/agent lifecycle management, tool-calling runtime, and skill resolution. Without Robot integration tests, there is no automated verification of the module's end-to-end behaviour in a real environment. Without ASV benchmarks, there is no baseline for detecting performance regressions.

Current Behavior

The agents module has no Robot Framework integration test suite and no ASV performance benchmark suite. This leaves a gap in the multi-level test coverage required by the project's testing philosophy.

Expected Behavior

  • A Robot Framework integration test suite exists for the agents module, covering key end-to-end scenarios (e.g., actor registration, skill resolution, tool invocation).
  • ASV benchmark suite exists for the agents module, covering performance-critical paths (e.g., actor loading, tool dispatch latency).
  • All new tests pass in CI and coverage remains ≥ 97%.

Acceptance Criteria

  • Robot Framework .robot test files exist under the appropriate integration test directory for the agents module.
  • ASV benchmark files exist for the agents module covering at least one performance-critical code path.
  • All Robot tests pass when run via the project task runner (nox).
  • All ASV benchmarks run without error.
  • Test coverage remains ≥ 97% after changes.
  • All nox default sessions pass without errors.

Supporting Information

  • Related to the project's Multi-Level Testing Mandate (see CONTRIBUTING.md § Testing Philosophy).
  • The agents module is part of Workstream C (Actors, Skills & Tools — Legendary #374).
  • Similar TEST-INFRA issues: #2202, #2203, #2204.

Subtasks

  • Identify all key integration scenarios for the agents module (actor registration, skill resolution, tool invocation, actor runtime lifecycle).
  • Write Robot Framework .robot feature files for each identified integration scenario.
  • Implement any missing Robot step definitions required by the new .robot files.
  • Identify performance-critical paths in the agents module suitable for ASV benchmarking.
  • Write ASV benchmark suite for the identified agents module paths.
  • Run nox -s robot (or equivalent) and confirm all Robot tests pass.
  • Run ASV benchmarks and confirm they execute without error.
  • Verify coverage ≥ 97% via nox -s coverage_report.
  • Run nox (all default sessions) and fix any errors.

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(agents): add Robot integration tests and ASV performance benchmarks), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/missing-test-levels-agents-robot-asv).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/missing-test-levels-agents-robot-asv` - **Commit Message**: `test(agents): add Robot integration tests and ASV performance benchmarks` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `agents` module is missing Robot Framework integration tests and ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in CONTRIBUTING.md, every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task. The `agents` module is a core component of the CleverAgents system, responsible for actor/agent lifecycle management, tool-calling runtime, and skill resolution. Without Robot integration tests, there is no automated verification of the module's end-to-end behaviour in a real environment. Without ASV benchmarks, there is no baseline for detecting performance regressions. ## Current Behavior The `agents` module has no Robot Framework integration test suite and no ASV performance benchmark suite. This leaves a gap in the multi-level test coverage required by the project's testing philosophy. ## Expected Behavior - A Robot Framework integration test suite exists for the `agents` module, covering key end-to-end scenarios (e.g., actor registration, skill resolution, tool invocation). - ASV benchmark suite exists for the `agents` module, covering performance-critical paths (e.g., actor loading, tool dispatch latency). - All new tests pass in CI and coverage remains ≥ 97%. ## Acceptance Criteria - [ ] Robot Framework `.robot` test files exist under the appropriate integration test directory for the `agents` module. - [ ] ASV benchmark files exist for the `agents` module covering at least one performance-critical code path. - [ ] All Robot tests pass when run via the project task runner (`nox`). - [ ] All ASV benchmarks run without error. - [ ] Test coverage remains ≥ 97% after changes. - [ ] All nox default sessions pass without errors. ## Supporting Information - Related to the project's Multi-Level Testing Mandate (see CONTRIBUTING.md § Testing Philosophy). - The `agents` module is part of Workstream C (Actors, Skills & Tools — Legendary #374). - Similar TEST-INFRA issues: #2202, #2203, #2204. ## Subtasks - [ ] Identify all key integration scenarios for the `agents` module (actor registration, skill resolution, tool invocation, actor runtime lifecycle). - [ ] Write Robot Framework `.robot` feature files for each identified integration scenario. - [ ] Implement any missing Robot step definitions required by the new `.robot` files. - [ ] Identify performance-critical paths in the `agents` module suitable for ASV benchmarking. - [ ] Write ASV benchmark suite for the identified `agents` module paths. - [ ] Run `nox -s robot` (or equivalent) and confirm all Robot tests pass. - [ ] Run ASV benchmarks and confirm they execute without error. - [ ] Verify coverage ≥ 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions) and fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(agents): add Robot integration tests and ASV performance benchmarks`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/missing-test-levels-agents-robot-asv`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-03 09:31:23 +00:00
Author
Owner

⚠️ Potential duplicate: Issue #2027 ("TEST-INFRA: [missing-test-levels] Add Robot integration tests for the agents module") covers Robot tests for the agents module. This issue additionally covers ASV benchmarks. Please check if these should be merged or if #2027 should be updated to include ASV benchmarks.


Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

⚠️ **Potential duplicate**: Issue #2027 ("TEST-INFRA: [missing-test-levels] Add Robot integration tests for the `agents` module") covers Robot tests for the agents module. This issue additionally covers ASV benchmarks. Please check if these should be merged or if #2027 should be updated to include ASV benchmarks. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer
freemo self-assigned this 2026-04-03 16:57:58 +00:00
Author
Owner

Label compliance fix applied:

  • Removed conflicting label: State/Unverified
  • Kept: State/Verified
  • Reason: Issue had both State/Unverified and State/Verified simultaneously — contradictory states. Kept the more advanced state (State/Verified) per CONTRIBUTING.md.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

Label compliance fix applied: - Removed conflicting label: `State/Unverified` - Kept: `State/Verified` - Reason: Issue had both `State/Unverified` and `State/Verified` simultaneously — contradictory states. Kept the more advanced state (`State/Verified`) per CONTRIBUTING.md. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer
Author
Owner

Label compliance fix applied:

  • Added missing labels: Priority/Medium, State/Unverified, Type/Testing
  • Reason: Issue was missing all required labels per CONTRIBUTING.md. Inferred Type/Testing from the "TEST-INFRA" prefix and content. Applied Priority/Medium and State/Unverified as defaults.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

Label compliance fix applied: - Added missing labels: `Priority/Medium`, `State/Unverified`, `Type/Testing` - Reason: Issue was missing all required labels per CONTRIBUTING.md. Inferred `Type/Testing` from the "TEST-INFRA" prefix and content. Applied `Priority/Medium` and `State/Unverified` as defaults. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium
  • Milestone: v3.7.0
  • MoSCoW: Should Have — Robot and ASV tests for the agents module improve test coverage breadth

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium - **Milestone**: v3.7.0 - **MoSCoW**: Should Have — Robot and ASV tests for the agents module improve test coverage breadth --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2206
No description provided.