TEST-INFRA: [dependency-security] Unable to perform dependency security analysis #3779

Open
opened 2026-04-06 06:16:51 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: task/backlog-dependency-security-scanning
  • Commit Message: feat(test-infra): add dependency security scanning tool capability
  • Milestone: (backlog — see note below)
  • Parent Epic: #400

Background and Context

The ca-test-infra-improver agent is currently unable to perform security analysis on test dependencies. The agent can identify test dependencies from pyproject.toml, but lacks the necessary tooling to check for known vulnerabilities (CVEs) in those dependencies. This gap means the project has no automated mechanism to detect insecure dependency versions in its test toolchain.

Current Behavior

When the ca-test-infra-improver agent attempts dependency security analysis, it falls back to using the webfetch tool to search for vulnerability information — an approach that is unreliable, incomplete, and not scalable. No structured vulnerability report is produced.

The following test dependencies were identified but could not be scanned:

  • ruff>=0.15.0,<0.16.0
  • pyright>=1.1.350
  • types-pyyaml>=6.0.0
  • types-aiofiles>=23.0.0
  • behave==1.3.3
  • pytest>=8.0.0
  • pytest-asyncio>=0.23.0
  • pytest-cov>=4.1.0
  • pre-commit>=3.6.0
  • bandit[toml]>=1.7.5
  • semgrep>=1.60.0
  • vulture>=2.10
  • radon>=6.0.1
  • slipcover>=1.0.17
  • asv>=0.6.5
  • robotframework>=7.3.2
  • robotframework-pabot>=4.0.0

Expected Behavior

The agent should be able to run a dedicated dependency security scan (e.g., via pip-audit, safety, or an equivalent tool) against the project's pyproject.toml and lockfiles, producing a structured report of any known CVEs or advisories for the listed dependencies.

Acceptance Criteria

  • A dependency scanning tool (e.g., pip-audit or safety) is integrated into the agent's available toolset or the project's nox sessions.
  • The agent can invoke the tool against pyproject.toml and produce a structured vulnerability report.
  • The scan covers both production and test/dev dependency groups.
  • Any identified vulnerabilities are surfaced as actionable findings (issue or report).
  • The scan is repeatable and can be run in CI.

Supporting Information

  • Duplicate Check: Search queries "dependency security", "dependency analysis", "vulnerability scanning" returned 0 relevant issues.
  • Discovered by: ca-test-infra-improver agent during autonomous test infrastructure review.
  • Related Epic: #400 (Post-MVP Security)

Subtasks

  • Evaluate and select a dependency scanning tool (pip-audit, safety, or equivalent).
  • Add the selected tool to the project's dev/test dependencies in pyproject.toml.
  • Create a nox session (e.g., nox -s dep_audit) that runs the dependency scan.
  • Integrate the scan into CI pipeline.
  • Document the tool and session in the project's contributing/dev guide.
  • Run nox (all default sessions), fix any errors.
  • Verify coverage >= 97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Backlog note: This issue was discovered during autonomous operation
on milestone v3.7.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.


Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/backlog-dependency-security-scanning` - **Commit Message**: `feat(test-infra): add dependency security scanning tool capability` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #400 ## Background and Context The `ca-test-infra-improver` agent is currently unable to perform security analysis on test dependencies. The agent can identify test dependencies from `pyproject.toml`, but lacks the necessary tooling to check for known vulnerabilities (CVEs) in those dependencies. This gap means the project has no automated mechanism to detect insecure dependency versions in its test toolchain. ## Current Behavior When the `ca-test-infra-improver` agent attempts dependency security analysis, it falls back to using the `webfetch` tool to search for vulnerability information — an approach that is unreliable, incomplete, and not scalable. No structured vulnerability report is produced. The following test dependencies were identified but could not be scanned: - `ruff>=0.15.0,<0.16.0` - `pyright>=1.1.350` - `types-pyyaml>=6.0.0` - `types-aiofiles>=23.0.0` - `behave==1.3.3` - `pytest>=8.0.0` - `pytest-asyncio>=0.23.0` - `pytest-cov>=4.1.0` - `pre-commit>=3.6.0` - `bandit[toml]>=1.7.5` - `semgrep>=1.60.0` - `vulture>=2.10` - `radon>=6.0.1` - `slipcover>=1.0.17` - `asv>=0.6.5` - `robotframework>=7.3.2` - `robotframework-pabot>=4.0.0` ## Expected Behavior The agent should be able to run a dedicated dependency security scan (e.g., via `pip-audit`, `safety`, or an equivalent tool) against the project's `pyproject.toml` and lockfiles, producing a structured report of any known CVEs or advisories for the listed dependencies. ## Acceptance Criteria - [ ] A dependency scanning tool (e.g., `pip-audit` or `safety`) is integrated into the agent's available toolset or the project's nox sessions. - [ ] The agent can invoke the tool against `pyproject.toml` and produce a structured vulnerability report. - [ ] The scan covers both production and test/dev dependency groups. - [ ] Any identified vulnerabilities are surfaced as actionable findings (issue or report). - [ ] The scan is repeatable and can be run in CI. ## Supporting Information - **Duplicate Check**: Search queries "dependency security", "dependency analysis", "vulnerability scanning" returned 0 relevant issues. - **Discovered by**: `ca-test-infra-improver` agent during autonomous test infrastructure review. - Related Epic: #400 (Post-MVP Security) ## Subtasks - [ ] Evaluate and select a dependency scanning tool (`pip-audit`, `safety`, or equivalent). - [ ] Add the selected tool to the project's dev/test dependencies in `pyproject.toml`. - [ ] Create a `nox` session (e.g., `nox -s dep_audit`) that runs the dependency scan. - [ ] Integrate the scan into CI pipeline. - [ ] Document the tool and session in the project's contributing/dev guide. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.7.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#400 Epic: Post-MVP Security
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#3779
No description provided.