cleveragents/cleveragents-core

Fork 3

[AUTO-INF-3] Benchmark files duplicate sys.path manipulation — extract to shared benchmark bootstrap module #10238

New issue

Open

opened 2026-04-17 09:38:01 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-17 09:38:01 +00:00

Owner

Metadata

Commit message: refactor(benchmarks): extract sys.path bootstrap to shared _bench_bootstrap module
Branch name: refactor/bench-bootstrap-shared-module

Background and Context

The benchmarks/ directory contains 231 benchmark files. Each file independently handles Python module path setup with a duplicated try/except ModuleNotFoundError pattern to ensure the local src/ tree is importable when ASV runs benchmarks. This pattern is copy-pasted across the entire benchmark suite.

Evidence of Duplication

benchmarks/session_model_bench.py (representative example):

try:
    from cleveragents.domain.models.core.session import (
        MessageRole,
        Session,
        SessionMessage,
        SessionTokenUsage,
    )
except ModuleNotFoundError:
    sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
    from cleveragents.domain.models.core.session import (
        MessageRole,
        Session,
        SessionMessage,
        SessionTokenUsage,
    )

benchmarks/cli_benchmark.py (same pattern):

try:
    from cleveragents.cli import main
except ModuleNotFoundError:
    sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
    from cleveragents.cli import main

benchmarks/_session_bench_common.py (slightly different variant):

_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
    sys.path.insert(0, _SRC)

Three different implementations of the same concern exist across the benchmark suite. The _session_bench_common.py file was already created to eliminate duplication (its docstring references "review finding F9"), but the sys.path pattern was not extracted there.

Problem

231 benchmark files each independently handle sys.path setup
At least two different patterns are in use (try/except vs. direct insert)
New benchmark files must know which pattern to use
If the src/ directory path changes, 231 files need updating
The _session_bench_common.py file already exists as a shared utility but does not centralize sys.path setup

Proposed Solution

Extract the sys.path bootstrap logic into a single location:

Option A: benchmarks/_bench_bootstrap.py (new file):

"""Shared bootstrap for ASV benchmark files.

Import this module at the top of every benchmark file to ensure the local
src/ tree is importable regardless of ASV's build state.
"""
from __future__ import annotations
import sys
from pathlib import Path

_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
    sys.path.insert(0, _SRC)

Then each benchmark file simply does:

import benchmarks._bench_bootstrap  # noqa: F401
from cleveragents.xxx import Yyy

Option B: Extend benchmarks/_session_bench_common.py to include the sys.path setup and rename it to benchmarks/_bench_common.py to reflect its broader scope.

Option C: benchmarks/conftest.py — if ASV supports conftest-style setup files.

Expected Behavior

A single shared module (benchmarks/_bench_bootstrap.py or equivalent) handles all sys.path manipulation for the benchmark suite. Every benchmark file imports this shared module instead of duplicating the path setup inline. The src/ directory path is defined in exactly one place, and all benchmarks continue to run correctly via asv run.

Acceptance Criteria

A single shared module handles sys.path bootstrap for all benchmark files
All 231 benchmark files use the shared bootstrap instead of inline sys.path manipulation
The shared module is documented with a docstring explaining its purpose
All existing benchmarks continue to pass (asv run succeeds)
New benchmark files added after this change follow the shared pattern (documented in CONTRIBUTING.md or a benchmarks/README.md)
Test coverage remains >= 97%

Subtasks

Audit all 231 benchmark files to identify all sys.path manipulation patterns
Create benchmarks/_bench_bootstrap.py (or extend _session_bench_common.py) with the canonical sys.path setup
Update all benchmark files to use the shared bootstrap
Verify asv run passes after the change
Document the pattern in CONTRIBUTING.md or benchmarks/README.md

Definition of Done

This issue is closed when:

A single shared module handles sys.path bootstrap for all benchmark files
All 231 benchmark files use the shared bootstrap
asv run passes without errors
The pattern is documented for future contributors
A PR has been reviewed and merged to main

Duplicate Check

Searched open issues (pages 1–7) and closed issues (pages 1–7) for keywords:

"benchmark" — Found: #9886 (add benchmark coverage), #9951 (fix typo in benchmark jobs), #10023 (performance regression test suite), #9871 (fix benchmark workflow), #9870 (fix benchmark trigger) — none address sys.path duplication or shared bootstrap
"sys.path" — No results
"conftest" — No results
"bench_common" — No results
"base class" — No results
"benchmark shared" — No results
"benchmark base" — No results

No duplicate found. This is a new, specific finding backed by direct code inspection of benchmarks/session_model_bench.py, benchmarks/cli_benchmark.py, and benchmarks/_session_bench_common.py.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

Automated by CleverAgents Bot
Agent: new-issue-creator

## Metadata - **Commit message:** `refactor(benchmarks): extract sys.path bootstrap to shared _bench_bootstrap module` - **Branch name:** `refactor/bench-bootstrap-shared-module` ## Background and Context The `benchmarks/` directory contains 231 benchmark files. Each file independently handles Python module path setup with a duplicated `try/except ModuleNotFoundError` pattern to ensure the local `src/` tree is importable when ASV runs benchmarks. This pattern is copy-pasted across the entire benchmark suite. ### Evidence of Duplication **`benchmarks/session_model_bench.py`** (representative example): ```python try: from cleveragents.domain.models.core.session import ( MessageRole, Session, SessionMessage, SessionTokenUsage, ) except ModuleNotFoundError: sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) from cleveragents.domain.models.core.session import ( MessageRole, Session, SessionMessage, SessionTokenUsage, ) ``` **`benchmarks/cli_benchmark.py`** (same pattern): ```python try: from cleveragents.cli import main except ModuleNotFoundError: sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) from cleveragents.cli import main ``` **`benchmarks/_session_bench_common.py`** (slightly different variant): ```python _SRC = str(Path(__file__).resolve().parents[1] / "src") if _SRC not in sys.path: sys.path.insert(0, _SRC) ``` Three different implementations of the same concern exist across the benchmark suite. The `_session_bench_common.py` file was already created to eliminate duplication (its docstring references "review finding F9"), but the sys.path pattern was not extracted there. ## Problem - **231 benchmark files** each independently handle sys.path setup - At least **two different patterns** are in use (try/except vs. direct insert) - New benchmark files must know which pattern to use - If the `src/` directory path changes, 231 files need updating - The `_session_bench_common.py` file already exists as a shared utility but does not centralize sys.path setup ## Proposed Solution Extract the sys.path bootstrap logic into a single location: **Option A: `benchmarks/_bench_bootstrap.py`** (new file): ```python """Shared bootstrap for ASV benchmark files. Import this module at the top of every benchmark file to ensure the local src/ tree is importable regardless of ASV's build state. """ from __future__ import annotations import sys from pathlib import Path _SRC = str(Path(__file__).resolve().parents[1] / "src") if _SRC not in sys.path: sys.path.insert(0, _SRC) ``` Then each benchmark file simply does: ```python import benchmarks._bench_bootstrap # noqa: F401 from cleveragents.xxx import Yyy ``` **Option B: Extend `benchmarks/_session_bench_common.py`** to include the sys.path setup and rename it to `benchmarks/_bench_common.py` to reflect its broader scope. **Option C: `benchmarks/conftest.py`** — if ASV supports conftest-style setup files. ## Expected Behavior A single shared module (`benchmarks/_bench_bootstrap.py` or equivalent) handles all sys.path manipulation for the benchmark suite. Every benchmark file imports this shared module instead of duplicating the path setup inline. The `src/` directory path is defined in exactly one place, and all benchmarks continue to run correctly via `asv run`. ## Acceptance Criteria - [ ] A single shared module handles sys.path bootstrap for all benchmark files - [ ] All 231 benchmark files use the shared bootstrap instead of inline sys.path manipulation - [ ] The shared module is documented with a docstring explaining its purpose - [ ] All existing benchmarks continue to pass (`asv run` succeeds) - [ ] New benchmark files added after this change follow the shared pattern (documented in CONTRIBUTING.md or a benchmarks/README.md) - [ ] Test coverage remains >= 97% ## Subtasks - [ ] Audit all 231 benchmark files to identify all sys.path manipulation patterns - [ ] Create `benchmarks/_bench_bootstrap.py` (or extend `_session_bench_common.py`) with the canonical sys.path setup - [ ] Update all benchmark files to use the shared bootstrap - [ ] Verify `asv run` passes after the change - [ ] Document the pattern in CONTRIBUTING.md or `benchmarks/README.md` ## Definition of Done This issue is closed when: 1. A single shared module handles sys.path bootstrap for all benchmark files 2. All 231 benchmark files use the shared bootstrap 3. `asv run` passes without errors 4. The pattern is documented for future contributors 5. A PR has been reviewed and merged to `main` ### Duplicate Check Searched open issues (pages 1–7) and closed issues (pages 1–7) for keywords: - `"benchmark"` — Found: #9886 (add benchmark coverage), #9951 (fix typo in benchmark jobs), #10023 (performance regression test suite), #9871 (fix benchmark workflow), #9870 (fix benchmark trigger) — **none address sys.path duplication or shared bootstrap** - `"sys.path"` — No results - `"conftest"` — No results - `"bench_common"` — No results - `"base class"` — No results - `"benchmark shared"` — No results - `"benchmark base"` — No results No duplicate found. This is a new, specific finding backed by direct code inspection of `benchmarks/session_model_bench.py`, `benchmarks/cli_benchmark.py`, and `benchmarks/_session_bench_common.py`. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor --- **Automated by CleverAgents Bot** Agent: new-issue-creator

HAL9000 added the

labels

2026-04-17 09:55:37 +00:00

HAL9000 referenced this issue