Tests (ASV): Add missing ASV benchmark suite for the acp module #2776

Open
opened 2026-04-04 19:23:37 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: test/acp-asv-benchmarks
  • Commit Message: test(acp): add ASV benchmark suite for acp module performance
  • Milestone: v3.7.0
  • Parent Epic: #933

Background and Context

The src/cleveragents/acp/ module is the A2A protocol layer — the sole Presentation-Application boundary per the project specification (ADR-047, §Architecture). It contains the following sub-modules (confirmed via compiled artefacts):

  • clients — A2A client transport wrappers
  • errors — ACP/A2A error types
  • events — SSE / push-notification event models
  • facade — high-level facade used by CLI and TUI layers
  • models — domain models (requests, responses, task lifecycle)
  • server_config — server-side configuration helpers
  • transport — low-level JSON-RPC 2.0 transport
  • versioning — protocol version negotiation

A scan of benchmarks/ confirms that no ASV benchmark file currently targets the acp module — there is a gap in the performance test suite that leaves the entire protocol layer without any throughput or latency regression tracking. Companion issues exist for the Behave unit-test layer (#2762) and the Robot Framework integration layer (#2768); this issue covers the ASV benchmark layer.

Because the acp module sits on the critical hot path for every agent interaction (all client-server communication flows through it), performance regressions here can silently degrade the entire platform. ASV benchmarks provide the continuous performance signal needed to catch such regressions before they reach production.

Subtasks

  • Audit each acp sub-module (clients, errors, events, facade, models, server_config, transport, versioning) and identify all performance-sensitive code paths to be benchmarked
  • Create benchmarks/benchmarks_acp.py with ASV Benchmark classes covering each identified code path (instantiation, serialisation/deserialisation, dispatch latency, version negotiation, etc.)
  • Ensure benchmarks are self-contained and do not rely on live network services (use in-process stubs where necessary, keeping mocks out of production code)
  • Run asv run locally and confirm all new benchmarks execute without errors
  • Run nox (all default sessions) and fix any errors
  • Verify coverage >= 97% via nox -e coverage_report

Definition of Done

  • All subtasks above are completed and checked off
  • benchmarks/benchmarks_acp.py exists and covers all performance-sensitive public behaviours of the acp module
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(acp): add ASV benchmark suite for acp module performance), followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/acp-asv-benchmarks)
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/acp-asv-benchmarks` - **Commit Message**: `test(acp): add ASV benchmark suite for acp module performance` - **Milestone**: v3.7.0 - **Parent Epic**: #933 ## Background and Context The `src/cleveragents/acp/` module is the A2A protocol layer — the sole Presentation-Application boundary per the project specification (ADR-047, §Architecture). It contains the following sub-modules (confirmed via compiled artefacts): - `clients` — A2A client transport wrappers - `errors` — ACP/A2A error types - `events` — SSE / push-notification event models - `facade` — high-level facade used by CLI and TUI layers - `models` — domain models (requests, responses, task lifecycle) - `server_config` — server-side configuration helpers - `transport` — low-level JSON-RPC 2.0 transport - `versioning` — protocol version negotiation A scan of `benchmarks/` confirms that **no ASV benchmark file currently targets the `acp` module** — there is a gap in the performance test suite that leaves the entire protocol layer without any throughput or latency regression tracking. Companion issues exist for the Behave unit-test layer (#2762) and the Robot Framework integration layer (#2768); this issue covers the ASV benchmark layer. Because the `acp` module sits on the critical hot path for every agent interaction (all client-server communication flows through it), performance regressions here can silently degrade the entire platform. ASV benchmarks provide the continuous performance signal needed to catch such regressions before they reach production. ## Subtasks - [ ] Audit each `acp` sub-module (`clients`, `errors`, `events`, `facade`, `models`, `server_config`, `transport`, `versioning`) and identify all performance-sensitive code paths to be benchmarked - [ ] Create `benchmarks/benchmarks_acp.py` with ASV `Benchmark` classes covering each identified code path (instantiation, serialisation/deserialisation, dispatch latency, version negotiation, etc.) - [ ] Ensure benchmarks are self-contained and do not rely on live network services (use in-process stubs where necessary, keeping mocks out of production code) - [ ] Run `asv run` locally and confirm all new benchmarks execute without errors - [ ] Run `nox` (all default sessions) and fix any errors - [ ] Verify coverage >= 97% via `nox -e coverage_report` ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] `benchmarks/benchmarks_acp.py` exists and covers all performance-sensitive public behaviours of the `acp` module - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(acp): add ASV benchmark suite for acp module performance`), followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/acp-asv-benchmarks`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-04 19:23:42 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2776
No description provided.