TEST-INFRA: [test-architecture] Remove sys.path manipulation from benchmarks #3632

New issue

Open

opened 2026-04-05 20:59:44 +00:00 by freemo · 0 comments

freemo commented

2026-04-05 20:59:44 +00:00

Owner

Metadata

Branch: refactor/test-architecture/remove-benchmark-syspath-manipulation
Commit Message: refactor(test-architecture): remove sys.path manipulation from benchmarks and install package in editable mode
Milestone: N/A — Backlog
Parent Epic: #2810

Background and context

Some of the ASV benchmark files in the benchmarks/ directory manipulate sys.path to import the project's source code. For example, benchmarks/skill_registry_bench.py contains the following code:

try:
    # ...
except ModuleNotFoundError:
    sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
    # ...

This is a code smell that indicates that the project is not properly installed as a package in the benchmark environment. This can lead to issues with dependencies and makes the benchmarks harder to run and maintain.

Current behavior

Inconsistent sys.path manipulation: Some benchmark files manipulate sys.path, while others do not. This suggests an inconsistent and unreliable setup for the benchmark environment.
Reduced portability: The benchmarks may not run correctly in different environments where the directory structure is not the same.
Increased maintenance overhead: The sys.path manipulation adds unnecessary complexity and makes the code harder to understand and maintain.

Expected behavior

All sys.path manipulation should be removed from the benchmark files.
The project should be installed in editable mode (pip install -e .) in the benchmark environment to ensure that the source code is available for import without any sys.path hacks.

Subtasks

Identify all benchmark files that manipulate sys.path.
Update the benchmark environment setup to install the project in editable mode.
Remove all sys.path manipulation from the benchmark files.
Ensure that all benchmarks still run correctly after the changes.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone Test Infrastructure. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `refactor/test-architecture/remove-benchmark-syspath-manipulation` - **Commit Message**: `refactor(test-architecture): remove sys.path manipulation from benchmarks and install package in editable mode` - **Milestone**: N/A — Backlog - **Parent Epic**: #2810 ## Background and context Some of the ASV benchmark files in the `benchmarks/` directory manipulate `sys.path` to import the project's source code. For example, `benchmarks/skill_registry_bench.py` contains the following code: ```python try: # ... except ModuleNotFoundError: sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) # ... ``` This is a code smell that indicates that the project is not properly installed as a package in the benchmark environment. This can lead to issues with dependencies and makes the benchmarks harder to run and maintain. ## Current behavior - **Inconsistent `sys.path` manipulation:** Some benchmark files manipulate `sys.path`, while others do not. This suggests an inconsistent and unreliable setup for the benchmark environment. - **Reduced portability:** The benchmarks may not run correctly in different environments where the directory structure is not the same. - **Increased maintenance overhead:** The `sys.path` manipulation adds unnecessary complexity and makes the code harder to understand and maintain. ## Expected behavior - All `sys.path` manipulation should be removed from the benchmark files. - The project should be installed in editable mode (`pip install -e .`) in the benchmark environment to ensure that the source code is available for import without any `sys.path` hacks. ## Subtasks - [ ] Identify all benchmark files that manipulate `sys.path`. - [ ] Update the benchmark environment setup to install the project in editable mode. - [ ] Remove all `sys.path` manipulation from the benchmark files. - [ ] Ensure that all benchmarks still run correctly after the changes. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. > **Backlog note:** This issue was discovered during autonomous operation > on milestone Test Infrastructure. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-05 20:59:51 +00:00

#2810 Epic: CI Quality Gates Restoration — Restore All 11 CI Jobs to Passing on Master

freemo added this to the v3.7.0 milestone

2026-04-05 21:01:02 +00:00

freemo referenced this issue

2026-04-05 21:01:33 +00:00

[Automated] CleverAgents Build Session - 2026-04-05 #3572