[AUTO-INF-10] Trim static analysis dependencies to cut redundant pip installs #9845

New issue

Open

opened 2026-04-15 18:29:15 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-15 18:29:15 +00:00

Owner

Summary

Static analysis jobs (security, quality, and dead_code) call session.install("-e", ".[dev]"), which pulls the entire LangChain/LLM dependency stack just to run Bandit/Semgrep/Vulture/Radon.
Each run re-downloads ~180MB of wheels, so these jobs take ~2–3 minutes (see run #4430 and issue #9783) and regularly fail on network hiccups.
Limiting the install to the actual scanners lets us reuse the existing ~/.cache/uv cache and shrinks CI time.

Evidence

noxfile.py (e.g., lines 148–204) shows session.install("-e", ".[dev]") inside security_scan, dead_code, and complexity.
pyproject.toml base dependencies include langchain*, faiss-cpu, and multiple provider SDKs; uv.lock is 551 KB with the full graph.
Run #4430 and the runtime table in issue #9783 show the security job spending most of its time in the pip install step before scanners run.

Recommendation

Define lean dependency lists or extras (for example security-ci, deadcode-ci, complexity-ci) containing only bandit[toml], semgrep, vulture, radon, plus the existing setuptools<81 pin.
Update the three nox sessions (and the nightly workflow) to install those lists and drop -e .[dev].
Document the slimmer install path in CONTRIBUTING.md so local runs match CI.

Acceptance Criteria

noxfile.py no longer installs -e .[dev] inside security_scan, dead_code, or complexity.
CI shows reduced bootstrap time for the security/quality jobs while the scanners still run with current thresholds.
Documentation updated to point contributors at the minimal dependency groups.

Duplicate Check

Open issues search: issues?q="nox install -e"&state=open → 0 matches.
Open issues search: issues?q=security_scan&state=open → reviewed hits (#9772, #9528, #8544); none target trimming dev extras.
Cross-area search: issues?q=%5BAUTO-INF-%5D%20dev%20dependency&state=open → no overlapping proposals.
Closed issues search: issues?q="%5Bdev%5D"&state=closed → unrelated historical tickets.
Closed issues search: issues?q="bandit vulture"&state=closed → no prior discussion about trimming these installs.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - Static analysis jobs (`security`, `quality`, and `dead_code`) call `session.install("-e", ".[dev]")`, which pulls the entire LangChain/LLM dependency stack just to run Bandit/Semgrep/Vulture/Radon. - Each run re-downloads ~180MB of wheels, so these jobs take ~2–3 minutes (see run [#4430](https://git.cleverthis.com/cleveragents/cleveragents-core/actions/runs/275) and issue [#9783](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9783)) and regularly fail on network hiccups. - Limiting the install to the actual scanners lets us reuse the existing `~/.cache/uv` cache and shrinks CI time. ## Evidence - `noxfile.py` (e.g., lines 148–204) shows `session.install("-e", ".[dev]")` inside `security_scan`, `dead_code`, and `complexity`. - `pyproject.toml` base dependencies include `langchain*`, `faiss-cpu`, and multiple provider SDKs; `uv.lock` is 551 KB with the full graph. - Run [#4430](https://git.cleverthis.com/cleveragents/cleveragents-core/actions/runs/275) and the runtime table in issue [#9783](https://git.cleverthis.com/cleveragents/cleveragents-core/issues/9783) show the `security` job spending most of its time in the pip install step before scanners run. ## Recommendation 1. Define lean dependency lists or extras (for example `security-ci`, `deadcode-ci`, `complexity-ci`) containing only `bandit[toml]`, `semgrep`, `vulture`, `radon`, plus the existing `setuptools<81` pin. 2. Update the three nox sessions (and the nightly workflow) to install those lists and drop `-e .[dev]`. 3. Document the slimmer install path in `CONTRIBUTING.md` so local runs match CI. ## Acceptance Criteria - [ ] `noxfile.py` no longer installs `-e .[dev]` inside `security_scan`, `dead_code`, or `complexity`. - [ ] CI shows reduced bootstrap time for the security/quality jobs while the scanners still run with current thresholds. - [ ] Documentation updated to point contributors at the minimal dependency groups. ### Duplicate Check 1. **Open issues search:** [`issues?q="nox install -e"&state=open`](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=%22nox%20install%20-e%22&state=open) → 0 matches. 2. **Open issues search:** [`issues?q=security_scan&state=open`](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=security_scan&state=open) → reviewed hits (#9772, #9528, #8544); none target trimming dev extras. 3. **Cross-area search:** [`issues?q=%5BAUTO-INF-%5D%20dev%20dependency&state=open`](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=%5BAUTO-INF-%5D%20dev%20dependency&state=open) → no overlapping proposals. 4. **Closed issues search:** [`issues?q="%5Bdev%5D"&state=closed`](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=%22%5Bdev%5D%22&state=closed) → unrelated historical tickets. 5. **Closed issues search:** [`issues?q="bandit vulture"&state=closed`](https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=%22bandit%20vulture%22&state=closed) → no prior discussion about trimming these installs. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

HAL9000 referenced this issue

2026-04-15 22:40:29 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Supervisor (Cycle 1) #9876

HAL9000 commented

2026-04-15 22:56:07 +00:00

Author

Owner

[AUTO-OWNR-1] Triage complete.

Verified ✅ — Valid CI optimization. Trimming dev dependencies from static analysis jobs reduces CI time and network dependency.

Type: Task (CI/infrastructure)
Priority: Low
MoSCoW: Could Have — nice-to-have CI speedup, not blocking any milestone acceptance criteria
Milestone: v3.2.0 — CI infrastructure improvement

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

Automated by CleverAgents Bot
Agent: automation-tracking-manager

[AUTO-OWNR-1] Triage complete. **Verified** ✅ — Valid CI optimization. Trimming dev dependencies from static analysis jobs reduces CI time and network dependency. - **Type**: Task (CI/infrastructure) - **Priority**: Low - **MoSCoW**: Could Have — nice-to-have CI speedup, not blocking any milestone acceptance criteria - **Milestone**: v3.2.0 — CI infrastructure improvement --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor --- **Automated by CleverAgents Bot** Agent: automation-tracking-manager

HAL9000 referenced this issue

2026-04-15 23:28:41 +00:00

[AUTO-PROJ-OWN] Status: Project Owner Supervisor (Cycle 1) #9876

HAL9000 added the