TEST-INFRA: Add Robot and ASV tests for the agents module #1635

Closed
opened 2026-04-02 23:20:36 +00:00 by freemo · 1 comment
Owner

Background and Context

The agents module implements the core Actor architecture of CleverAgents — YAML-configured conversational units that compose LangGraph workflows, reference Skills, and orchestrate Tools. While unit-level Behave scenarios exist, there are currently no Robot Framework integration tests or ASV benchmark suites covering this module.

Integration tests are required to validate end-to-end Actor lifecycle behaviour (instantiation, skill acquisition, tool invocation, graph composition) against real or stubbed infrastructure. ASV benchmarks are required to track performance regressions in Actor construction and execution throughput over time.

Expected Behavior

  • A Robot Framework test suite under robot/ covers the agents module's public API: Actor creation, skill/tool wiring, and graph execution.
  • An ASV benchmark suite under benchmarks/ covers Actor construction time and execution throughput.
  • All new tests pass cleanly under nox -e integration_tests and nox (all default sessions).
  • Coverage remains ≥ 97% (nox -e coverage_report).

Acceptance Criteria

  • At least one Robot Framework .robot file exists under robot/ targeting the agents module.
  • At least one ASV benchmark class exists under benchmarks/ targeting the agents module.
  • All Robot tests pass via nox -e integration_tests.
  • All ASV benchmarks run without error via nox.
  • No regressions in existing Behave unit tests (nox -e unit_tests).
  • Coverage ≥ 97% confirmed via nox -e coverage_report.
  • Pyright type-checks all new files cleanly (nox -e typecheck).

Metadata

  • Branch: test/m7-agents-robot-asv
  • Commit Message: test(agents): add Robot integration tests and ASV benchmarks for agents module
  • Milestone: v3.7.0

Subtasks

  • Audit existing agents module public API surface to identify integration test entry points
  • Write Robot Framework test suite robot/agents/test_agents.robot covering Actor lifecycle
  • Write Robot Framework test suite robot/agents/test_actor_skill_tool.robot covering skill/tool wiring
  • Write ASV benchmark class benchmarks/bench_agents.py covering Actor construction time
  • Write ASV benchmark class benchmarks/bench_agents.py covering Actor execution throughput
  • Run nox -e integration_tests and fix any failures
  • Run nox -e coverage_report and confirm coverage ≥ 97%
  • Run nox -e typecheck and fix any type errors
  • Run nox (all default sessions) and fix any remaining errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Background and Context The `agents` module implements the core Actor architecture of CleverAgents — YAML-configured conversational units that compose LangGraph workflows, reference Skills, and orchestrate Tools. While unit-level Behave scenarios exist, there are currently no Robot Framework integration tests or ASV benchmark suites covering this module. Integration tests are required to validate end-to-end Actor lifecycle behaviour (instantiation, skill acquisition, tool invocation, graph composition) against real or stubbed infrastructure. ASV benchmarks are required to track performance regressions in Actor construction and execution throughput over time. ## Expected Behavior - A Robot Framework test suite under `robot/` covers the agents module's public API: Actor creation, skill/tool wiring, and graph execution. - An ASV benchmark suite under `benchmarks/` covers Actor construction time and execution throughput. - All new tests pass cleanly under `nox -e integration_tests` and `nox` (all default sessions). - Coverage remains ≥ 97% (`nox -e coverage_report`). ## Acceptance Criteria - [ ] At least one Robot Framework `.robot` file exists under `robot/` targeting the agents module. - [ ] At least one ASV benchmark class exists under `benchmarks/` targeting the agents module. - [ ] All Robot tests pass via `nox -e integration_tests`. - [ ] All ASV benchmarks run without error via `nox`. - [ ] No regressions in existing Behave unit tests (`nox -e unit_tests`). - [ ] Coverage ≥ 97% confirmed via `nox -e coverage_report`. - [ ] Pyright type-checks all new files cleanly (`nox -e typecheck`). ## Metadata - **Branch**: `test/m7-agents-robot-asv` - **Commit Message**: `test(agents): add Robot integration tests and ASV benchmarks for agents module` - **Milestone**: v3.7.0 ## Subtasks - [ ] Audit existing agents module public API surface to identify integration test entry points - [ ] Write Robot Framework test suite `robot/agents/test_agents.robot` covering Actor lifecycle - [ ] Write Robot Framework test suite `robot/agents/test_actor_skill_tool.robot` covering skill/tool wiring - [ ] Write ASV benchmark class `benchmarks/bench_agents.py` covering Actor construction time - [ ] Write ASV benchmark class `benchmarks/bench_agents.py` covering Actor execution throughput - [ ] Run `nox -e integration_tests` and fix any failures - [ ] Run `nox -e coverage_report` and confirm coverage ≥ 97% - [ ] Run `nox -e typecheck` and fix any type errors - [ ] Run `nox` (all default sessions) and fix any remaining errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-02 23:21:02 +00:00
Author
Owner

Closing as duplicate of #1627 which covers all three test levels (Behave + Robot + ASV) for the agents module.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #1627 which covers all three test levels (Behave + Robot + ASV) for the agents module. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
freemo 2026-04-02 23:33:47 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1635
No description provided.