TEST-INFRA: [missing-test-levels] Add ASV tests for the 'core' module #1936

Open
opened 2026-04-03 00:18:10 +00:00 by freemo · 2 comments
Owner

Metadata

  • Branch: test/core-asv-benchmarks
  • Commit Message: test(core): add ASV performance benchmarks for the core module
  • Milestone: v3.5.0
  • Parent Epic: #1678

Background and Context

The core module in src/cleveragents/core is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done.

The core module houses the foundational resilience and error-handling primitives used across all agent operations, including circuit breakers, retry patterns, async cleanup, and exception handling. Given the central role of these components in every agent execution path, ASV benchmarks are essential for detecting performance regressions as the codebase evolves.

The core module contains the following source files requiring benchmark coverage:

  • circuit_breaker.pyCircuitBreaker, CircuitBreakerState, CircuitBreakerOpen
  • retry_patterns.py — retry decorator factory functions and invocation paths
  • retry_service_patterns.pyretry_service_operation, retry_auto_debug, RetryContext
  • async_cleanup.py — async cleanup lifecycle operations
  • error_handling.py — error handling utilities
  • exceptions.py — exception hierarchy

Expected Behavior

The benchmarks/ directory should contain ASV benchmark suites covering the performance-critical paths of the core module, enabling automated detection of performance regressions in CI.

Acceptance Criteria

  • ASV benchmark file(s) exist in benchmarks/ covering the core module
  • Benchmarks cover CircuitBreaker (closed-state call overhead, state transitions, open-state fast-fail latency)
  • Benchmarks cover retry decorator construction and happy-path invocation overhead (retry_patterns.py)
  • Benchmarks cover retry_service_operation and retry_auto_debug paths (retry_service_patterns.py)
  • Benchmarks cover async cleanup lifecycle operations (async_cleanup.py)
  • Benchmarks follow the project's ASV benchmark conventions (refer to existing benchmarks in benchmarks/ for style)
  • All nox stages pass with the new benchmarks in place
  • Coverage remains >= 97%

Subtasks

  • Review existing ASV benchmarks in benchmarks/ to understand conventions and patterns
  • Audit src/cleveragents/core/ to identify all performance-critical paths requiring benchmarks
  • Create benchmarks/core_circuit_breaker_bench.py with suites for CircuitBreaker state machine operations
  • Create benchmarks/core_retry_patterns_bench.py with suites for decorator construction and invocation overhead
  • Create benchmarks/core_retry_service_patterns_bench.py with suites for service-level retry operations
  • Create benchmarks/core_async_cleanup_bench.py with suites for async cleanup lifecycle operations
  • Verify all benchmarks are importable and runnable via ASV (asv run or nox -s benchmarks)
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • ASV benchmark file(s) for the core module exist in benchmarks/ and cover all performance-critical paths identified in the audit.
  • All new benchmarks are importable and runnable without errors.
  • All nox stages pass.
  • Coverage >= 97%
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/core-asv-benchmarks` - **Commit Message**: `test(core): add ASV performance benchmarks for the core module` - **Milestone**: v3.5.0 - **Parent Epic**: #1678 ## Background and Context The `core` module in `src/cleveragents/core` is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done. The `core` module houses the foundational resilience and error-handling primitives used across all agent operations, including circuit breakers, retry patterns, async cleanup, and exception handling. Given the central role of these components in every agent execution path, ASV benchmarks are essential for detecting performance regressions as the codebase evolves. The `core` module contains the following source files requiring benchmark coverage: - `circuit_breaker.py` — `CircuitBreaker`, `CircuitBreakerState`, `CircuitBreakerOpen` - `retry_patterns.py` — retry decorator factory functions and invocation paths - `retry_service_patterns.py` — `retry_service_operation`, `retry_auto_debug`, `RetryContext` - `async_cleanup.py` — async cleanup lifecycle operations - `error_handling.py` — error handling utilities - `exceptions.py` — exception hierarchy ## Expected Behavior The `benchmarks/` directory should contain ASV benchmark suites covering the performance-critical paths of the `core` module, enabling automated detection of performance regressions in CI. ## Acceptance Criteria - [ ] ASV benchmark file(s) exist in `benchmarks/` covering the `core` module - [ ] Benchmarks cover `CircuitBreaker` (closed-state call overhead, state transitions, open-state fast-fail latency) - [ ] Benchmarks cover retry decorator construction and happy-path invocation overhead (`retry_patterns.py`) - [ ] Benchmarks cover `retry_service_operation` and `retry_auto_debug` paths (`retry_service_patterns.py`) - [ ] Benchmarks cover async cleanup lifecycle operations (`async_cleanup.py`) - [ ] Benchmarks follow the project's ASV benchmark conventions (refer to existing benchmarks in `benchmarks/` for style) - [ ] All nox stages pass with the new benchmarks in place - [ ] Coverage remains >= 97% ## Subtasks - [ ] Review existing ASV benchmarks in `benchmarks/` to understand conventions and patterns - [ ] Audit `src/cleveragents/core/` to identify all performance-critical paths requiring benchmarks - [ ] Create `benchmarks/core_circuit_breaker_bench.py` with suites for `CircuitBreaker` state machine operations - [ ] Create `benchmarks/core_retry_patterns_bench.py` with suites for decorator construction and invocation overhead - [ ] Create `benchmarks/core_retry_service_patterns_bench.py` with suites for service-level retry operations - [ ] Create `benchmarks/core_async_cleanup_bench.py` with suites for async cleanup lifecycle operations - [ ] Verify all benchmarks are importable and runnable via ASV (`asv run` or `nox -s benchmarks`) - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - [ ] All subtasks above are completed and checked off. - [ ] ASV benchmark file(s) for the `core` module exist in `benchmarks/` and cover all performance-critical paths identified in the audit. - [ ] All new benchmarks are importable and runnable without errors. - [ ] All nox stages pass. - [ ] Coverage >= 97% - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.5.0 milestone 2026-04-03 00:18:36 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1936
No description provided.