Tests (ASV): Add missing ASV benchmark suite for the agents module #2787

Open
opened 2026-04-04 19:32:48 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/missing-asv-benchmarks-agents
  • Commit Message: test(agents): add ASV performance benchmark suite for the agents module
  • Milestone: v3.7.0
  • Parent Epic: #1678

Background and Context

The agents module is missing ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in CONTRIBUTING.md, every module must have tests at all required levels: Behave BDD unit tests, Robot Framework integration tests, and ASV performance benchmarks. The absence of ASV benchmarks for the agents module means there is no automated baseline for detecting performance regressions in one of the system's most critical components.

The agents module is a core component of the CleverAgents architecture, responsible for actor/agent lifecycle management, tool-calling runtime, and skill resolution (see docs/specification.md — Actor Abstraction and Tool & Skill Abstraction). Performance regressions in this module would directly impact the responsiveness and throughput of the entire agent orchestration pipeline.

Current Behaviour

The agents module has no ASV benchmark suite. Running nox (or the benchmark session directly) produces no benchmark results for this module, leaving a gap in the project's performance regression detection coverage.

Expected Behaviour

  • An ASV benchmark suite exists under the benchmarks/ directory for the agents module.
  • The suite covers at least the following performance-critical paths:
    • Actor loading and initialisation latency
    • Tool dispatch latency (single and batch)
    • Skill resolution throughput
    • Agent lifecycle teardown cost
  • All benchmarks run without error via nox.
  • No existing nox sessions are broken by the addition of the benchmarks.

Subtasks

  • Identify all performance-critical code paths in the agents module suitable for ASV benchmarking (actor loading, tool dispatch, skill resolution, lifecycle teardown)
  • Create the ASV benchmark file(s) under benchmarks/ following the existing directory and naming conventions
  • Implement setup / teardown fixtures as needed to isolate benchmark measurements
  • Implement at least one benchmark per identified performance-critical path
  • Run nox benchmark session locally and confirm all new benchmarks execute without error
  • Verify no regressions are introduced in other nox sessions (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e integration_tests)
  • Verify coverage remains ≥ 97% via nox -e coverage_report
  • Run full nox (all default sessions) and confirm clean pass

Definition of Done

  • All subtasks above are completed and checked off
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(agents): add ASV performance benchmark suite for the agents module), followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/missing-asv-benchmarks-agents)
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/missing-asv-benchmarks-agents` - **Commit Message**: `test(agents): add ASV performance benchmark suite for the agents module` - **Milestone**: v3.7.0 - **Parent Epic**: #1678 ## Background and Context The `agents` module is missing ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in `CONTRIBUTING.md`, every module must have tests at all required levels: Behave BDD unit tests, Robot Framework integration tests, and ASV performance benchmarks. The absence of ASV benchmarks for the `agents` module means there is no automated baseline for detecting performance regressions in one of the system's most critical components. The `agents` module is a core component of the CleverAgents architecture, responsible for actor/agent lifecycle management, tool-calling runtime, and skill resolution (see `docs/specification.md` — Actor Abstraction and Tool & Skill Abstraction). Performance regressions in this module would directly impact the responsiveness and throughput of the entire agent orchestration pipeline. ## Current Behaviour The `agents` module has no ASV benchmark suite. Running `nox` (or the benchmark session directly) produces no benchmark results for this module, leaving a gap in the project's performance regression detection coverage. ## Expected Behaviour - An ASV benchmark suite exists under the `benchmarks/` directory for the `agents` module. - The suite covers at least the following performance-critical paths: - Actor loading and initialisation latency - Tool dispatch latency (single and batch) - Skill resolution throughput - Agent lifecycle teardown cost - All benchmarks run without error via `nox`. - No existing nox sessions are broken by the addition of the benchmarks. ## Subtasks - [ ] Identify all performance-critical code paths in the `agents` module suitable for ASV benchmarking (actor loading, tool dispatch, skill resolution, lifecycle teardown) - [ ] Create the ASV benchmark file(s) under `benchmarks/` following the existing directory and naming conventions - [ ] Implement `setup` / `teardown` fixtures as needed to isolate benchmark measurements - [ ] Implement at least one benchmark per identified performance-critical path - [ ] Run `nox` benchmark session locally and confirm all new benchmarks execute without error - [ ] Verify no regressions are introduced in other nox sessions (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`) - [ ] Verify coverage remains ≥ 97% via `nox -e coverage_report` - [ ] Run full `nox` (all default sessions) and confirm clean pass ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(agents): add ASV performance benchmark suite for the agents module`), followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/missing-asv-benchmarks-agents`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-04 19:32:53 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified | MoSCoW: Could Have — ASV benchmark suites are nice to have for performance tracking but not blocking feature work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified | **MoSCoW**: Could Have — ASV benchmark suites are nice to have for performance tracking but not blocking feature work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2787
No description provided.