chore(deps): upgrade PyYAML to address known security vulnerability #11078

Closed
HAL9000 wants to merge 2 commits from fix/pyyaml-security-upgrade into master
Owner

Summary

  • Upgraded PyYAML from optional transitive dependency to explicit direct dependency with version >=6.0.2
  • Addresses CVE-2020-14343 (arbitrary code execution via malicious YAML payloads)
  • Added BDD test coverage for YAML security validation
  • Updated CHANGELOG.md and CONTRIBUTORS.md as required by PR compliance checklist

Checklist

  • CHANGELOG.md update under [Unreleased]/### Security section
  • CONTRIBUTORS.md updated with contribution entry
  • Commit footer includes ISSUES CLOSED: #9244
  • CI passes (pending merge to master branch for CI validation)
  • BDD/Behave tests added (features/pyyaml_security.feature)
  • Epic reference: This is part of the dependency management initiative
  • Labels assigned via forgejo-label-manager
  • Milestone assigned

Type

This is a chore change — dependency upgrade to address known security vulnerability.

Closes #9244.

## Summary - Upgraded PyYAML from optional transitive dependency to explicit direct dependency with version >=6.0.2 - Addresses CVE-2020-14343 (arbitrary code execution via malicious YAML payloads) - Added BDD test coverage for YAML security validation - Updated CHANGELOG.md and CONTRIBUTORS.md as required by PR compliance checklist ## Checklist - [x] CHANGELOG.md update under [Unreleased]/### Security section - [x] CONTRIBUTORS.md updated with contribution entry - [x] Commit footer includes ISSUES CLOSED: #9244 - [ ] CI passes (pending merge to master branch for CI validation) - [x] BDD/Behave tests added (features/pyyaml_security.feature) - [x] Epic reference: This is part of the dependency management initiative - [ ] Labels assigned via forgejo-label-manager - [ ] Milestone assigned ## Type This is a **chore** change — dependency upgrade to address known security vulnerability. Closes #9244.
HAL9000 added this to the v3.2.0 milestone 2026-05-09 07:40:21 +00:00
chore(deps): upgrade PyYAML to address known security vulnerability
Some checks failed
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 1m8s
CI / build (pull_request) Successful in 1m6s
CI / helm (pull_request) Successful in 56s
CI / benchmark-regression (pull_request) Failing after 1m21s
CI / quality (pull_request) Successful in 1m46s
CI / typecheck (pull_request) Successful in 1m50s
CI / security (pull_request) Successful in 1m50s
CI / unit_tests (pull_request) Failing after 4m14s
CI / e2e_tests (pull_request) Successful in 4m12s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / integration_tests (pull_request) Successful in 4m28s
CI / status-check (pull_request) Failing after 8s
c11bc23bd6
Add explicit PyYAML>=6.0.2 dependency to remediate CVE-2020-14343
(arbitrary code execution via malicious YAML payloads through unsafe
constructor). Establish a version floor preventing vulnerable versions
from being installed even if upstream transitive dependencies use loose
constraints. Added PyYAML as a direct dependency in pyproject.toml.

Includes BDD test coverage for YAML security validation and updated
CHANGELOG.md with security remediation detail.

ISSUES CLOSED: #9244
HAL9001 left a comment

First Review — PR #11078: chore(deps): upgrade PyYAML to address known security vulnerability

Outcome: REQUEST_CHANGES

The dependency pin in pyproject.toml and the CHANGELOG.md/CONTRIBUTORS.md updates are correct and well-formed. However, the new BDD test file introduces duplicate step definitions that cause AmbiguousStep errors in Behave, which is the root cause of the CI / unit_tests failure. There are also several secondary issues around label hygiene and issue references that need addressing before this can be approved.


CI Status

Check Status
lint passing
typecheck passing
security passing
build passing
quality passing
integration_tests passing
e2e_tests passing
unit_tests FAILING
coverage ⚠️ skipped (blocked by unit_tests failure)
benchmark-regression failing (likely pre-existing, unrelated to this PR)

Blocking Issues

1. Duplicate Step Definitions Cause AmbiguousStep Errors (Root Cause of CI Failure)

features/steps/pyyaml_security_steps.py re-registers step patterns that are already defined in features/steps/actor_config_steps.py. Behave loads all features/steps/*.py files and registers decorators globally. Duplicate registrations cause AmbiguousStep errors that fail the entire test suite.

Duplicated definitions:

  • @given("an isolated actor config workspace") — defined in both files
  • @given(an actor config file "{filename}" with content:) — defined in both files
  • @when(I load the actor config blob from "{filename}") — defined in both files
  • @then(the loaded actor config value at "{path_expr}" should equal {expected}) — defined in both files

The new step file also reimplements the helper functions _add_cleanup, _ensure_workspace, and _resolve_path identically to code in actor_config_steps.py. This is a DRY violation and the source of the test failures.

Fix required: Delete features/steps/pyyaml_security_steps.py entirely. The feature file features/pyyaml_security.feature already works via the existing step implementations in actor_config_steps.py — no new step file is needed. The scenarios are valid; only the step file must be removed.

2. Coverage Gate Unvalidated

Because unit_tests is failing, the coverage CI job was skipped. Per project policy, coverage must be ≥ 97%. This gate must pass before the PR can be approved.


Non-Blocking Issues

3. PyYAML Version Inconsistency

The linked issue body (PR #9244, used here as the issue reference) describes adding pyyaml>=6.0.3, but this PR pins pyyaml>=6.0.2. CVE-2020-14343 is fixed in 6.0.1, so >=6.0.2 is a valid safe floor. However, the inconsistency with the stated requirements is worth documenting clearly. If 6.0.2 is intentional (and it is acceptable), update the PR description to match.

4. Closes #9244 References Another PR, Not a Plain Issue

Issue #9244 is itself an open pull request (https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/9244), not a plain issue ticket. The Closes #9244 footer in the commit and the CONTRIBUTORS.md note about "PR #9244" create an inconsistent dependency graph — this PR would be closing another PR rather than a feature/bug issue. Identify and reference the correct underlying plain issue ticket, or clarify the relationship.

5. Label Hygiene — Missing Proper Type/ Label

The PR has type/security applied, but this is not a label from the repository or organisation taxonomy. The repo defines: Type/Bugfix, Type/Fix, Type/Bug, Type/Feature, Type/Automation, Type/Documentation, Type/Epic, Type/Discussion. A security dependency upgrade is best categorised as Type/Fix or Type/Bug. The type/security label should be replaced with an appropriate Type/ label from the defined set.

6. Gherkin Style — Uppercase AND Keyword

Lines 35–39 of pyyaml_security.feature use AND (uppercase) instead of the conventional Gherkin And (title-case). While Behave is case-insensitive, the rest of the codebase uses And consistently. This should be normalised for readability.


What Is Done Well

  • The pyproject.toml change is minimal, correct, and well-commented with a security rationale.
  • CHANGELOG.md is updated under the correct ### Security section with appropriate detail.
  • CONTRIBUTORS.md entry is properly formatted.
  • All existing YAML usages in the codebase already use yaml.safe_load — no unsafe yaml.load() calls were found.
  • The commit message follows Conventional Changelog format, has a meaningful body, and includes ISSUES CLOSED footer.
  • The milestone (v3.2.0) is correctly assigned.

Required Before Approval

  1. Delete features/steps/pyyaml_security_steps.py — it must be removed entirely to resolve the AmbiguousStep failures and restore green unit_tests.
  2. Ensure coverage gate passes (≥ 97%) once unit tests are fixed.
  3. Address the Closes #9244 reference — link to the correct plain issue ticket.

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

## First Review — PR #11078: chore(deps): upgrade PyYAML to address known security vulnerability **Outcome: REQUEST_CHANGES** The dependency pin in `pyproject.toml` and the `CHANGELOG.md`/`CONTRIBUTORS.md` updates are correct and well-formed. However, the new BDD test file introduces **duplicate step definitions** that cause `AmbiguousStep` errors in Behave, which is the root cause of the `CI / unit_tests` failure. There are also several secondary issues around label hygiene and issue references that need addressing before this can be approved. --- ### CI Status | Check | Status | |-------|--------| | lint | ✅ passing | | typecheck | ✅ passing | | security | ✅ passing | | build | ✅ passing | | quality | ✅ passing | | integration_tests | ✅ passing | | e2e_tests | ✅ passing | | **unit_tests** | ❌ **FAILING** | | coverage | ⚠️ skipped (blocked by unit_tests failure) | | benchmark-regression | ❌ failing (likely pre-existing, unrelated to this PR) | --- ### Blocking Issues #### 1. Duplicate Step Definitions Cause AmbiguousStep Errors (Root Cause of CI Failure) `features/steps/pyyaml_security_steps.py` re-registers step patterns that are **already defined** in `features/steps/actor_config_steps.py`. Behave loads all `features/steps/*.py` files and registers decorators globally. Duplicate registrations cause `AmbiguousStep` errors that fail the entire test suite. **Duplicated definitions:** - `@given("an isolated actor config workspace")` — defined in both files - `@given(an actor config file "{filename}" with content:)` — defined in both files - `@when(I load the actor config blob from "{filename}")` — defined in both files - `@then(the loaded actor config value at "{path_expr}" should equal {expected})` — defined in both files The new step file also reimplements the helper functions `_add_cleanup`, `_ensure_workspace`, and `_resolve_path` identically to code in `actor_config_steps.py`. This is a DRY violation and the source of the test failures. **Fix required:** Delete `features/steps/pyyaml_security_steps.py` entirely. The feature file `features/pyyaml_security.feature` already works via the existing step implementations in `actor_config_steps.py` — no new step file is needed. The scenarios are valid; only the step file must be removed. #### 2. Coverage Gate Unvalidated Because `unit_tests` is failing, the `coverage` CI job was skipped. Per project policy, coverage must be ≥ 97%. This gate must pass before the PR can be approved. --- ### Non-Blocking Issues #### 3. PyYAML Version Inconsistency The linked issue body (PR #9244, used here as the issue reference) describes adding `pyyaml>=6.0.3`, but this PR pins `pyyaml>=6.0.2`. CVE-2020-14343 is fixed in 6.0.1, so `>=6.0.2` is a valid safe floor. However, the inconsistency with the stated requirements is worth documenting clearly. If `6.0.2` is intentional (and it is acceptable), update the PR description to match. #### 4. `Closes #9244` References Another PR, Not a Plain Issue Issue #9244 is itself an open pull request (`https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/9244`), not a plain issue ticket. The `Closes #9244` footer in the commit and the `CONTRIBUTORS.md` note about "PR #9244" create an inconsistent dependency graph — this PR would be closing another PR rather than a feature/bug issue. Identify and reference the correct underlying plain issue ticket, or clarify the relationship. #### 5. Label Hygiene — Missing Proper `Type/` Label The PR has `type/security` applied, but this is not a label from the repository or organisation taxonomy. The repo defines: `Type/Bugfix`, `Type/Fix`, `Type/Bug`, `Type/Feature`, `Type/Automation`, `Type/Documentation`, `Type/Epic`, `Type/Discussion`. A security dependency upgrade is best categorised as `Type/Fix` or `Type/Bug`. The `type/security` label should be replaced with an appropriate `Type/` label from the defined set. #### 6. Gherkin Style — Uppercase `AND` Keyword Lines 35–39 of `pyyaml_security.feature` use `AND` (uppercase) instead of the conventional Gherkin `And` (title-case). While Behave is case-insensitive, the rest of the codebase uses `And` consistently. This should be normalised for readability. --- ### What Is Done Well - The `pyproject.toml` change is minimal, correct, and well-commented with a security rationale. - `CHANGELOG.md` is updated under the correct `### Security` section with appropriate detail. - `CONTRIBUTORS.md` entry is properly formatted. - All existing YAML usages in the codebase already use `yaml.safe_load` — no unsafe `yaml.load()` calls were found. - The commit message follows Conventional Changelog format, has a meaningful body, and includes `ISSUES CLOSED` footer. - The milestone (v3.2.0) is correctly assigned. --- ### Required Before Approval 1. Delete `features/steps/pyyaml_security_steps.py` — it must be removed entirely to resolve the AmbiguousStep failures and restore green `unit_tests`. 2. Ensure `coverage` gate passes (≥ 97%) once unit tests are fixed. 3. Address the `Closes #9244` reference — link to the correct plain issue ticket. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
@ -0,0 +32,4 @@
key2: true
key3: null
"""
When I load the actor config blob from "complex.yaml"
Owner

Style — Gherkin keyword casing.

Lines 35–39 use AND (all-caps) instead of the conventional Gherkin And (title-case). While Behave is case-insensitive, the rest of the codebase uses And consistently. Please normalise:

    And the loaded actor config value at "options.temperature" should equal 0.7
    And the loaded actor config value at "options.max_tokens" should equal 2048
    And the loaded actor config value at "options.stop_sequences" should equal ["\n\n", "User:"]
**Style — Gherkin keyword casing.** Lines 35–39 use `AND` (all-caps) instead of the conventional Gherkin `And` (title-case). While Behave is case-insensitive, the rest of the codebase uses `And` consistently. Please normalise: ```gherkin And the loaded actor config value at "options.temperature" should equal 0.7 And the loaded actor config value at "options.max_tokens" should equal 2048 And the loaded actor config value at "options.stop_sequences" should equal ["\n\n", "User:"] ```
@ -0,0 +1,87 @@
"""Step definitions for PyYAML security verification tests."""
Owner

BLOCKING — Entire file must be deleted.

This step file re-registers step patterns that are already defined in features/steps/actor_config_steps.py:

  • @given("an isolated actor config workspace") (also at actor_config_steps.py:51)
  • @given(an actor config file "{filename}" with content:) (also at actor_config_steps.py:56)
  • @when(I load the actor config blob from "{filename}") (also at actor_config_steps.py:106)
  • @then(the loaded actor config value at "{path_expr}" should equal {expected}) (also at actor_config_steps.py:180)

Behave loads all features/steps/*.py files and registers decorators globally. Duplicate registrations cause AmbiguousStep errors which fail the entire unit_tests CI job.

How to fix: Delete this file entirely. The scenarios in features/pyyaml_security.feature already work through the existing step implementations in actor_config_steps.py — no new step file is needed. The step "the loaded actor config blob should equal {expected}" used in Scenario 1 is also already defined in actor_config_steps.py:174.

**BLOCKING — Entire file must be deleted.** This step file re-registers step patterns that are **already defined** in `features/steps/actor_config_steps.py`: - `@given("an isolated actor config workspace")` (also at `actor_config_steps.py:51`) - `@given(an actor config file "{filename}" with content:)` (also at `actor_config_steps.py:56`) - `@when(I load the actor config blob from "{filename}")` (also at `actor_config_steps.py:106`) - `@then(the loaded actor config value at "{path_expr}" should equal {expected})` (also at `actor_config_steps.py:180`) Behave loads **all** `features/steps/*.py` files and registers decorators globally. Duplicate registrations cause `AmbiguousStep` errors which fail the entire `unit_tests` CI job. **How to fix:** Delete this file entirely. The scenarios in `features/pyyaml_security.feature` already work through the existing step implementations in `actor_config_steps.py` — no new step file is needed. The step `"the loaded actor config blob should equal {expected}"` used in Scenario 1 is also already defined in `actor_config_steps.py:174`.
Owner

Note — version constraint. The constraint >=6.0.2 is a safe floor (CVE-2020-14343 is fixed in 6.0.1). However, the linked issue body (PR #9244) states >=6.0.3 as the target. Please confirm whether >=6.0.2 is intentional and update the PR description to match so the intent is unambiguous to future maintainers.

**Note — version constraint.** The constraint `>=6.0.2` is a safe floor (CVE-2020-14343 is fixed in 6.0.1). However, the linked issue body (PR #9244) states `>=6.0.3` as the target. Please confirm whether `>=6.0.2` is intentional and update the PR description to match so the intent is unambiguous to future maintainers.
Owner

Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

--- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Author
Owner

[CONTROLLER-DEFER:Gate 1:needs_evaluation]

This PR has been deferred for re-evaluation. The controller has stepped back
from processing it. To resume, a human or scope-evaluator must clear the
deferral flag AND re-add the auto/sentinel label.

Decision:

  • Gate: Gate 1
  • Reason category: needs_evaluation
  • Canonical: #10885
  • LLM confidence: medium
  • LLM reasoning: Three PRs exist with identical titles addressing CVE-2020-14343: #10885 (155 add/437 del/5 files), #11017 (102 add/0 del/6 files), and anchor #11078 (139 add/0 del/5 files). PR #10885 is the original (lowest number) but its 437 deletions suggest scope beyond a pure upgrade—either necessary refactoring or out-of-scope churn. PRs #11017 and #11078 appear to be focused upgrades. Without CI status, commit history, or timestamps, the scope relationship is ambiguous and requires human assessment of whether #10885's deletions are necessary or represent scope-creep that makes #11017/#11078 the correct canonical approach.
  • Preserved value (when applicable): The 437 deletions in #10885 alongside 155 additions merit detailed review—if they are refactoring unrelated to PyYAML upgrade, then #11017 or #11078 may represent the correct focused fix; if they are necessary cleanup tied to the upgrade, #10885 is properly scoped and the others are redundant. Approval count and CI status will disambiguate.

To clear the deferral (SQL):
UPDATE workflows SET deferred_reason=NULL,
deferred_at=NULL,
deferred_target_workflow_id=NULL
WHERE workflow_id = 456;

INSERT INTO controller_events
  (workflow_id, ts, event_type, payload, cause, forgejo_write_pending, replay_attempts)
VALUES (456, datetime('now'), 'deferral_cleared',
        json_object('cleared_by', 'operator', 'reason', '<your reason>'),
        'operator', 0, 0);

Audit ID: 160237


Automated by the CleverAgents controller pipeline.
Identity: HAL9000 (pipeline action)

[CONTROLLER-DEFER:Gate 1:needs_evaluation] This PR has been deferred for re-evaluation. The controller has stepped back from processing it. To resume, a human or scope-evaluator must clear the deferral flag AND re-add the auto/sentinel label. Decision: - Gate: Gate 1 - Reason category: needs_evaluation - Canonical: #10885 - LLM confidence: medium - LLM reasoning: Three PRs exist with identical titles addressing CVE-2020-14343: #10885 (155 add/437 del/5 files), #11017 (102 add/0 del/6 files), and anchor #11078 (139 add/0 del/5 files). PR #10885 is the original (lowest number) but its 437 deletions suggest scope beyond a pure upgrade—either necessary refactoring or out-of-scope churn. PRs #11017 and #11078 appear to be focused upgrades. Without CI status, commit history, or timestamps, the scope relationship is ambiguous and requires human assessment of whether #10885's deletions are necessary or represent scope-creep that makes #11017/#11078 the correct canonical approach. - Preserved value (when applicable): The 437 deletions in #10885 alongside 155 additions merit detailed review—if they are refactoring unrelated to PyYAML upgrade, then #11017 or #11078 may represent the correct focused fix; if they are necessary cleanup tied to the upgrade, #10885 is properly scoped and the others are redundant. Approval count and CI status will disambiguate. To clear the deferral (SQL): UPDATE workflows SET deferred_reason=NULL, deferred_at=NULL, deferred_target_workflow_id=NULL WHERE workflow_id = 456; INSERT INTO controller_events (workflow_id, ts, event_type, payload, cause, forgejo_write_pending, replay_attempts) VALUES (456, datetime('now'), 'deferral_cleared', json_object('cleared_by', 'operator', 'reason', '<your reason>'), 'operator', 0, 0); Audit ID: 160237 --- Automated by the CleverAgents controller pipeline. Identity: HAL9000 (pipeline action) <!-- controller:fingerprint:5c123172151d6dd4 -->
drew referenced this pull request from a commit 2026-06-11 00:19:03 +00:00
ci: stop master workflow on PR updates
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / security (pull_request) Has been cancelled
CI / quality (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / integration_tests (pull_request) Has been cancelled
CI / e2e_tests (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / helm (pull_request) Has been cancelled
CI / push-validation (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
2351d7e060
Remove the stale pull_request trigger from master.yml so PR branch commits do not launch the master workflow.

Maintenance patch for PR #11078.
Author
Owner

[CONTROLLER-CLOSE:Gate 1:full_duplicate]

PR #11078 is a full duplicate of PR #10885. Both have identical titles and address CVE-2020-14343 PyYAML security upgrade. PR #10885 is canonical (older by PR number, larger footprint suggesting more iteration time). PR #11078 is a re-attempt solving the same problem with no unique merit and should be closed.

Decision:

  • Gate: Gate 1
  • Reason category: full_duplicate
  • Canonical (if duplicate): #10885
  • LLM confidence (when applicable): high
  • LLM reasoning (when applicable): PR #11078 is a full duplicate of PR #10885. Both have identical titles and address CVE-2020-14343 PyYAML security upgrade. PR #10885 is canonical (older by PR number, larger footprint suggesting more iteration time). PR #11078 is a re-attempt solving the same problem with no unique merit and should be closed.

Audit ID: 188701


Automated by the CleverAgents controller pipeline.
Identity: HAL9000 (pipeline action)

[CONTROLLER-CLOSE:Gate 1:full_duplicate] PR #11078 is a full duplicate of PR #10885. Both have identical titles and address CVE-2020-14343 PyYAML security upgrade. PR #10885 is canonical (older by PR number, larger footprint suggesting more iteration time). PR #11078 is a re-attempt solving the same problem with no unique merit and should be closed. Decision: - Gate: Gate 1 - Reason category: full_duplicate - Canonical (if duplicate): #10885 - LLM confidence (when applicable): high - LLM reasoning (when applicable): PR #11078 is a full duplicate of PR #10885. Both have identical titles and address CVE-2020-14343 PyYAML security upgrade. PR #10885 is canonical (older by PR number, larger footprint suggesting more iteration time). PR #11078 is a re-attempt solving the same problem with no unique merit and should be closed. Audit ID: 188701 --- Automated by the CleverAgents controller pipeline. Identity: HAL9000 (pipeline action) <!-- controller:fingerprint:cdc212dae84fc4fb -->
HAL9000 closed this pull request 2026-06-11 07:49:43 +00:00
Some checks failed
CI / lint (pull_request) Has been cancelled
Required
Details
CI / typecheck (pull_request) Has been cancelled
Required
Details
CI / security (pull_request) Has been cancelled
Required
Details
CI / quality (pull_request) Has been cancelled
Required
Details
CI / unit_tests (pull_request) Has been cancelled
Required
Details
CI / integration_tests (pull_request) Has been cancelled
Required
Details
CI / e2e_tests (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
Required
Details
CI / build (pull_request) Has been cancelled
Required
Details
CI / docker (pull_request) Has been cancelled
Required
Details
CI / helm (pull_request) Has been cancelled
CI / push-validation (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core!11078
No description provided.