UAT: asv.conf.json uses `virtualenv` environment type instead of `uv`, inconsistent with project tooling #3949

New issue

Open

opened 2026-04-06 07:42:53 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 07:42:53 +00:00

Owner

Bug Report

Feature Area: Benchmarks and Performance
Severity: Low (backlog)
Found by: UAT automated testing (instance: uat-benchmarks-perf-001)

What Was Tested

The asv.conf.json configuration was inspected for consistency with the project's standard tooling (uv).

Expected Behavior

The project uses uv as its standard package manager and virtual environment tool throughout (all nox sessions use venv_backend="uv", pyproject.toml uses uv for dependency management). The ASV configuration should be consistent with this choice.

Actual Behavior

asv.conf.json specifies "environment_type": "virtualenv" and "install_command": ["python -m pip install {build_dir}"]:

{
  "version": 1,
  "project": "CleverAgents",
  "project_url": "https://git.cleverthis.com/cleveragents/cleveragents-core",
  "repo": ".",
  "branches": ["HEAD"],
  "pythons": ["3.13"],
  "environment_type": "virtualenv",
  "install_command": ["python -m pip install {build_dir}"],
  "benchmark_dir": "benchmarks",
  "env_dir": "build/asv/env",
  "results_dir": "build/asv/results",
  "html_dir": "build/asv/html"
}

This uses virtualenv (which requires virtualenv to be installed separately) and pip for installation, while the rest of the project uses uv for both virtual environment creation and package installation.

Evidence

All nox sessions use venv_backend="uv":

@nox.session(python=DEFAULT_PYTHON, reuse_venv=True, venv_backend="uv")
def benchmark(session: nox.Session):
    ...

The benchmark nox session itself installs with session.install("-e", ".[tests]") which uses uv (via the venv_backend="uv" setting), but ASV's own environment management uses virtualenv + pip.

Impact

Minor inconsistency in tooling — virtualenv must be installed separately for ASV to work
Slower environment creation compared to uv (uv is significantly faster)
ASV 0.6.4+ supports uv as an environment type via "environment_type": "uv" (see ASV docs)
The install_command using pip instead of uv pip is slower and inconsistent

Recommended Fix

Update asv.conf.json to use uv:

{
  "version": 1,
  "project": "CleverAgents",
  "project_url": "https://git.cleverthis.com/cleveragents/cleveragents-core",
  "repo": ".",
  "branches": ["HEAD"],
  "pythons": ["3.13"],
  "environment_type": "uv",
  "install_command": ["uv pip install {build_dir}"],
  "benchmark_dir": "benchmarks",
  "env_dir": "build/asv/env",
  "results_dir": "build/asv/results",
  "html_dir": "build/asv/html"
}

Note: Verify that the version of ASV in use (>=0.6.5 per pyproject.toml) supports "environment_type": "uv" before making this change. If not supported, this issue can remain in the backlog until ASV adds support.

Code Location

asv.conf.json, lines 7-8: "environment_type" and "install_command" fields

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Bug Report **Feature Area:** Benchmarks and Performance **Severity:** Low (backlog) **Found by:** UAT automated testing (instance: uat-benchmarks-perf-001) ### What Was Tested The `asv.conf.json` configuration was inspected for consistency with the project's standard tooling (`uv`). ### Expected Behavior The project uses `uv` as its standard package manager and virtual environment tool throughout (all nox sessions use `venv_backend="uv"`, `pyproject.toml` uses `uv` for dependency management). The ASV configuration should be consistent with this choice. ### Actual Behavior `asv.conf.json` specifies `"environment_type": "virtualenv"` and `"install_command": ["python -m pip install {build_dir}"]`: ```json { "version": 1, "project": "CleverAgents", "project_url": "https://git.cleverthis.com/cleveragents/cleveragents-core", "repo": ".", "branches": ["HEAD"], "pythons": ["3.13"], "environment_type": "virtualenv", "install_command": ["python -m pip install {build_dir}"], "benchmark_dir": "benchmarks", "env_dir": "build/asv/env", "results_dir": "build/asv/results", "html_dir": "build/asv/html" } ``` This uses `virtualenv` (which requires `virtualenv` to be installed separately) and `pip` for installation, while the rest of the project uses `uv` for both virtual environment creation and package installation. ### Evidence All nox sessions use `venv_backend="uv"`: ```python @nox.session(python=DEFAULT_PYTHON, reuse_venv=True, venv_backend="uv") def benchmark(session: nox.Session): ... ``` The `benchmark` nox session itself installs with `session.install("-e", ".[tests]")` which uses `uv` (via the `venv_backend="uv"` setting), but ASV's own environment management uses `virtualenv` + `pip`. ### Impact - Minor inconsistency in tooling — `virtualenv` must be installed separately for ASV to work - Slower environment creation compared to `uv` (uv is significantly faster) - ASV 0.6.4+ supports `uv` as an environment type via `"environment_type": "uv"` (see [ASV docs](https://asv.readthedocs.io/en/stable/asv.conf.json.html#environment-type)) - The `install_command` using `pip` instead of `uv pip` is slower and inconsistent ### Recommended Fix Update `asv.conf.json` to use `uv`: ```json { "version": 1, "project": "CleverAgents", "project_url": "https://git.cleverthis.com/cleveragents/cleveragents-core", "repo": ".", "branches": ["HEAD"], "pythons": ["3.13"], "environment_type": "uv", "install_command": ["uv pip install {build_dir}"], "benchmark_dir": "benchmarks", "env_dir": "build/asv/env", "results_dir": "build/asv/results", "html_dir": "build/asv/html" } ``` **Note:** Verify that the version of ASV in use (`>=0.6.5` per `pyproject.toml`) supports `"environment_type": "uv"` before making this change. If not supported, this issue can remain in the backlog until ASV adds support. ### Code Location - `asv.conf.json`, lines 7-8: `"environment_type"` and `"install_command"` fields --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo referenced this issue

2026-04-06 07:43:14 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:49:16 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775