Tests (ASV): Add missing ASV benchmark suite for the application module #2790

Open
opened 2026-04-04 19:38:15 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/missing-asv-benchmarks-application
  • Commit Message: test(application): add ASV performance benchmark suite for the application module
  • Milestone: v3.7.0
  • Parent Epic: #1678

Background and Context

The application module is missing ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in CONTRIBUTING.md, every module must have tests at all required levels: Behave BDD unit tests, Robot Framework integration tests, and ASV performance benchmarks. The absence of ASV benchmarks for the application module means there is no automated baseline for detecting performance regressions in this top-level orchestration layer.

The application module is a core component of the CleverAgents architecture, responsible for bootstrapping the agent runtime, wiring together Projects, Resources, Actors, Tools, and Skills, and managing the top-level Plan Lifecycle (Action → Strategize → Execute → Apply). Performance regressions in this module would directly impact startup latency, plan throughput, and overall system responsiveness (see docs/specification.md — Plan Lifecycle and High-Level Architecture).

Current Behaviour

The application module has no ASV benchmark suite. Running nox (or the benchmark session directly) produces no benchmark results for this module, leaving a gap in the project's performance regression detection coverage.

Expected Behaviour

  • An ASV benchmark suite exists under the benchmarks/ directory for the application module.
  • The suite covers at least the following performance-critical paths:
    • Application bootstrap and initialisation latency
    • Plan lifecycle phase transition throughput (Action → Strategize → Execute → Apply)
    • Project and Resource wiring cost
    • Top-level teardown and cleanup latency
  • All benchmarks run without error via nox.
  • No existing nox sessions are broken by the addition of the benchmarks.

Subtasks

  • Identify all performance-critical code paths in the application module suitable for ASV benchmarking (bootstrap/init, plan lifecycle transitions, project/resource wiring, teardown)
  • Create the ASV benchmark file(s) under benchmarks/ following the existing directory and naming conventions
  • Implement setup / teardown fixtures as needed to isolate benchmark measurements
  • Implement at least one benchmark per identified performance-critical path
  • Run nox benchmark session locally and confirm all new benchmarks execute without error
  • Verify no regressions are introduced in other nox sessions (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e integration_tests)
  • Verify coverage remains ≥ 97% via nox -e coverage_report
  • Run full nox (all default sessions) and confirm clean pass

Definition of Done

  • All subtasks above are completed and checked off
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(application): add ASV performance benchmark suite for the application module), followed by a blank line, then additional lines providing relevant details about the implementation
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/missing-asv-benchmarks-application)
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/missing-asv-benchmarks-application` - **Commit Message**: `test(application): add ASV performance benchmark suite for the application module` - **Milestone**: v3.7.0 - **Parent Epic**: #1678 ## Background and Context The `application` module is missing ASV (airspeed velocity) performance benchmarks. Per the project's Multi-Level Testing Mandate in `CONTRIBUTING.md`, every module must have tests at all required levels: Behave BDD unit tests, Robot Framework integration tests, and ASV performance benchmarks. The absence of ASV benchmarks for the `application` module means there is no automated baseline for detecting performance regressions in this top-level orchestration layer. The `application` module is a core component of the CleverAgents architecture, responsible for bootstrapping the agent runtime, wiring together Projects, Resources, Actors, Tools, and Skills, and managing the top-level Plan Lifecycle (`Action → Strategize → Execute → Apply`). Performance regressions in this module would directly impact startup latency, plan throughput, and overall system responsiveness (see `docs/specification.md` — Plan Lifecycle and High-Level Architecture). ## Current Behaviour The `application` module has no ASV benchmark suite. Running `nox` (or the benchmark session directly) produces no benchmark results for this module, leaving a gap in the project's performance regression detection coverage. ## Expected Behaviour - An ASV benchmark suite exists under the `benchmarks/` directory for the `application` module. - The suite covers at least the following performance-critical paths: - Application bootstrap and initialisation latency - Plan lifecycle phase transition throughput (`Action → Strategize → Execute → Apply`) - Project and Resource wiring cost - Top-level teardown and cleanup latency - All benchmarks run without error via `nox`. - No existing nox sessions are broken by the addition of the benchmarks. ## Subtasks - [ ] Identify all performance-critical code paths in the `application` module suitable for ASV benchmarking (bootstrap/init, plan lifecycle transitions, project/resource wiring, teardown) - [ ] Create the ASV benchmark file(s) under `benchmarks/` following the existing directory and naming conventions - [ ] Implement `setup` / `teardown` fixtures as needed to isolate benchmark measurements - [ ] Implement at least one benchmark per identified performance-critical path - [ ] Run `nox` benchmark session locally and confirm all new benchmarks execute without error - [ ] Verify no regressions are introduced in other nox sessions (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`) - [ ] Verify coverage remains ≥ 97% via `nox -e coverage_report` - [ ] Run full `nox` (all default sessions) and confirm clean pass ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(application): add ASV performance benchmark suite for the application module`), followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/missing-asv-benchmarks-application`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.7.0 milestone 2026-04-04 19:38:23 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified | MoSCoW: Could Have — ASV benchmark suite for the application module. Performance benchmarks are valuable for regression detection but not blocking feature work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified | **MoSCoW**: Could Have — ASV benchmark suite for the application module. Performance benchmarks are valuable for regression detection but not blocking feature work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2790
No description provided.