TEST-INFRA: [missing-test-levels] Add ASV tests for domain module #1897

Closed
opened 2026-04-03 00:09:51 +00:00 by freemo · 4 comments
Owner

Metadata

  • Branch: test/domain-asv-benchmarks
  • Commit Message: test(domain): add ASV performance benchmarks for the domain module
  • Milestone: v3.5.0
  • Parent Epic: #934

Background and Context

The domain module in src/cleveragents/domain is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done.

The domain module is the core architectural layer of CleverAgents, housing the data models, repositories, context fragments, plan lifecycle models, and provider abstractions that underpin all agent operations. It contains critical subsystems including ACMS (Adaptive Context Management System) models, plan/action domain models, resource and project models, actor/skill/tool domain models, and repository abstractions. Given the central role of these models in every plan execution and agent interaction, ASV benchmarks are essential for detecting performance regressions as the codebase evolves.

Expected Behavior

The benchmarks/ directory should contain ASV benchmark suites covering the performance-critical paths of the domain module, enabling automated detection of performance regressions in CI.

Acceptance Criteria

  • ASV benchmark file(s) exist in benchmarks/ covering the domain module
  • Benchmarks cover key domain model construction and serialization (e.g., plan, action, resource, actor, skill, tool models)
  • Benchmarks cover ACMS model operations (e.g., scope resolution, tier lookups, strategy selection)
  • Benchmarks cover context fragment operations in domain/contexts/
  • Benchmarks cover repository abstraction operations in domain/repositories/
  • Benchmarks follow the project's ASV benchmark conventions (refer to existing benchmarks in benchmarks/ for style)
  • All nox stages pass with the new benchmarks in place
  • Coverage remains >= 97%

Subtasks

  • Review existing ASV benchmarks in benchmarks/ to understand conventions and patterns
  • Audit src/cleveragents/domain/ to identify all performance-critical paths requiring benchmarks
  • Create benchmarks/domain_models_bench.py (or equivalent) with benchmark classes/functions for domain models
  • Create benchmarks/domain_acms_bench.py (or equivalent) for ACMS model operations
  • Create benchmarks/domain_contexts_bench.py (or equivalent) for context fragment operations
  • Create benchmarks/domain_repositories_bench.py (or equivalent) for repository abstraction operations
  • Verify all benchmarks are importable and runnable via ASV
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • ASV benchmark file(s) for the domain module exist in benchmarks/ and cover all performance-critical paths identified in the audit.
  • All new benchmarks are importable and runnable without errors.
  • All nox stages pass.
  • Coverage >= 97%
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/domain-asv-benchmarks` - **Commit Message**: `test(domain): add ASV performance benchmarks for the domain module` - **Milestone**: v3.5.0 - **Parent Epic**: #934 ## Background and Context The `domain` module in `src/cleveragents/domain` is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done. The `domain` module is the core architectural layer of CleverAgents, housing the data models, repositories, context fragments, plan lifecycle models, and provider abstractions that underpin all agent operations. It contains critical subsystems including ACMS (Adaptive Context Management System) models, plan/action domain models, resource and project models, actor/skill/tool domain models, and repository abstractions. Given the central role of these models in every plan execution and agent interaction, ASV benchmarks are essential for detecting performance regressions as the codebase evolves. ## Expected Behavior The `benchmarks/` directory should contain ASV benchmark suites covering the performance-critical paths of the `domain` module, enabling automated detection of performance regressions in CI. ## Acceptance Criteria - [ ] ASV benchmark file(s) exist in `benchmarks/` covering the `domain` module - [ ] Benchmarks cover key domain model construction and serialization (e.g., plan, action, resource, actor, skill, tool models) - [ ] Benchmarks cover ACMS model operations (e.g., scope resolution, tier lookups, strategy selection) - [ ] Benchmarks cover context fragment operations in `domain/contexts/` - [ ] Benchmarks cover repository abstraction operations in `domain/repositories/` - [ ] Benchmarks follow the project's ASV benchmark conventions (refer to existing benchmarks in `benchmarks/` for style) - [ ] All nox stages pass with the new benchmarks in place - [ ] Coverage remains >= 97% ## Subtasks - [ ] Review existing ASV benchmarks in `benchmarks/` to understand conventions and patterns - [ ] Audit `src/cleveragents/domain/` to identify all performance-critical paths requiring benchmarks - [ ] Create `benchmarks/domain_models_bench.py` (or equivalent) with benchmark classes/functions for domain models - [ ] Create `benchmarks/domain_acms_bench.py` (or equivalent) for ACMS model operations - [ ] Create `benchmarks/domain_contexts_bench.py` (or equivalent) for context fragment operations - [ ] Create `benchmarks/domain_repositories_bench.py` (or equivalent) for repository abstraction operations - [ ] Verify all benchmarks are importable and runnable via ASV - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - [ ] All subtasks above are completed and checked off. - [ ] ASV benchmark file(s) for the `domain` module exist in `benchmarks/` and cover all performance-critical paths identified in the audit. - [ ] All new benchmarks are importable and runnable without errors. - [ ] All nox stages pass. - [ ] Coverage >= 97% - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.5.0 milestone 2026-04-03 00:10:49 +00:00
Author
Owner

Duplicate Detection Notice

A request was made to create a new issue for "TEST-INFRA: [missing-test-levels] Add ASV performance benchmarks for the domain module", but this issue already exists and fully covers that work.

The incoming request body matched this issue on all key dimensions:

  • Module: src/cleveragents/domain
  • Missing test level: ASV performance benchmarks (benchmarks/)
  • Present test levels: Behave unit tests (features/) , Robot integration tests (robot/)
  • Labels: State/Unverified, Type/Testing, Priority/Medium

No duplicate issue was created. This issue (#1897) remains the canonical tracking ticket for this work.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Duplicate Detection Notice A request was made to create a new issue for **"TEST-INFRA: [missing-test-levels] Add ASV performance benchmarks for the `domain` module"**, but this issue already exists and fully covers that work. The incoming request body matched this issue on all key dimensions: - **Module**: `src/cleveragents/domain` - **Missing test level**: ASV performance benchmarks (`benchmarks/`) - **Present test levels**: Behave unit tests (`features/`) ✅, Robot integration tests (`robot/`) ✅ - **Labels**: `State/Unverified`, `Type/Testing`, `Priority/Medium` No duplicate issue was created. This issue (#1897) remains the canonical tracking ticket for this work. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Author
Owner

This issue is a duplicate of #1925 ("TEST-INFRA: [missing-test-levels] Add ASV tests for the domain module"), which was created prior to this one and covers the same scope. Closing in favour of #1925.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

This issue is a duplicate of #1925 ("TEST-INFRA: [missing-test-levels] Add ASV tests for the `domain` module"), which was created prior to this one and covers the same scope. Closing in favour of #1925. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1897
No description provided.