TEST-INFRA: [missing-test-levels] Add ASV performance benchmarks for the 'lsp' module #1881

Open
opened 2026-04-03 00:04:59 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: test/lsp-asv-benchmarks
  • Commit Message: test(lsp): add ASV performance benchmarks for the lsp module
  • Milestone: v3.5.0
  • Parent Epic: #934

Background and Context

The lsp module is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done. The lsp module — which manages Language Server Protocol server lifecycle, tool adapter execution, workspace mapping, and context enrichment — has no coverage in benchmarks/, leaving performance regressions undetectable.

The LSP subsystem is a core architectural component (spec §LSP Integration) that exposes language intelligence capabilities (diagnostics, hover, completions, references, rename, code actions) as callable tools for actors. Given the latency-sensitive nature of LSP operations and the potential for performance regressions as the runtime matures, ASV benchmarks are essential.

Expected Behavior

The benchmarks/ directory should contain ASV benchmark suites covering the performance-critical paths of the lsp module, enabling automated detection of performance regressions in CI.

Acceptance Criteria

  • ASV benchmark file(s) exist in benchmarks/ covering the lsp module
  • Benchmarks cover LspRegistry operations (register, lookup, list)
  • Benchmarks cover LspToolAdapter tool spec generation
  • Benchmarks cover LspRuntime lifecycle operations (start/stop server stubs or mocked)
  • Benchmarks follow the project's ASV benchmark conventions (refer to CONTRIBUTING.md)
  • All nox stages pass with the new benchmarks in place
  • Coverage remains >= 97%

Subtasks

  • Review existing ASV benchmarks in benchmarks/ to understand conventions
  • Create benchmarks/bench_lsp.py (or equivalent) with benchmark classes/functions
  • Add benchmarks for LspRegistry: register, lookup by name, lookup by language, list
  • Add benchmarks for LspToolAdapter: get_tools() spec generation
  • Add benchmarks for LspRuntime: server lifecycle stubs (using mocks where real servers are unavailable)
  • Verify benchmarks run cleanly via the project task runner
  • Run nox (all default sessions), fix any errors
  • Verify coverage >= 97% via nox -s coverage_report

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (test(lsp): add ASV performance benchmarks for the lsp module), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (test/lsp-asv-benchmarks).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `test/lsp-asv-benchmarks` - **Commit Message**: `test(lsp): add ASV performance benchmarks for the lsp module` - **Milestone**: v3.5.0 - **Parent Epic**: #934 ## Background and Context The `lsp` module is missing ASV (airspeed velocity) performance benchmarks. Per the project's multi-level testing mandate (CONTRIBUTING.md), every coding task must include performance benchmarks as part of the definition of done. The `lsp` module — which manages Language Server Protocol server lifecycle, tool adapter execution, workspace mapping, and context enrichment — has no coverage in `benchmarks/`, leaving performance regressions undetectable. The LSP subsystem is a core architectural component (spec §LSP Integration) that exposes language intelligence capabilities (diagnostics, hover, completions, references, rename, code actions) as callable tools for actors. Given the latency-sensitive nature of LSP operations and the potential for performance regressions as the runtime matures, ASV benchmarks are essential. ## Expected Behavior The `benchmarks/` directory should contain ASV benchmark suites covering the performance-critical paths of the `lsp` module, enabling automated detection of performance regressions in CI. ## Acceptance Criteria - [ ] ASV benchmark file(s) exist in `benchmarks/` covering the `lsp` module - [ ] Benchmarks cover `LspRegistry` operations (register, lookup, list) - [ ] Benchmarks cover `LspToolAdapter` tool spec generation - [ ] Benchmarks cover `LspRuntime` lifecycle operations (start/stop server stubs or mocked) - [ ] Benchmarks follow the project's ASV benchmark conventions (refer to CONTRIBUTING.md) - [ ] All nox stages pass with the new benchmarks in place - [ ] Coverage remains >= 97% ## Subtasks - [ ] Review existing ASV benchmarks in `benchmarks/` to understand conventions - [ ] Create `benchmarks/bench_lsp.py` (or equivalent) with benchmark classes/functions - [ ] Add benchmarks for `LspRegistry`: register, lookup by name, lookup by language, list - [ ] Add benchmarks for `LspToolAdapter`: `get_tools()` spec generation - [ ] Add benchmarks for `LspRuntime`: server lifecycle stubs (using mocks where real servers are unavailable) - [ ] Verify benchmarks run cleanly via the project task runner - [ ] Run `nox` (all default sessions), fix any errors - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`test(lsp): add ASV performance benchmarks for the lsp module`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`test/lsp-asv-benchmarks`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.5.0 milestone 2026-04-03 00:05:19 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1881
No description provided.