UAT: `_redact_dict_inner` in `shared/redaction.py` does not redact secrets inside nested lists — potential secret leakage in structured log output #4013

New issue

Open

opened 2026-04-06 08:36:06 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 08:36:06 +00:00

Owner

Metadata

Branch: fix/redact-nested-list-secrets
Commit Message: fix(shared/redaction): handle nested lists in _redact_dict_inner and secrets_masking_processor
Milestone: (none — backlog)
Parent Epic: #400

Background

The _redact_dict_inner function in src/cleveragents/shared/redaction.py handles list values by iterating one level deep, but does not recursively handle nested lists (lists within lists). This means secrets inside nested list structures will not be redacted, creating a potential secret leakage path in structured log output.

Current Implementation (lines 180–186)

elif isinstance(value, list):
    result[key] = [
        _redact_dict_inner(item)
        if isinstance(item, dict)
        else (redact_value(item) if isinstance(item, str) else item)
        for item in value
    ]

What is handled:

list[dict] → recursively redacts each dict ✅
list[str] → calls redact_value() on each string ✅

What is NOT handled (security gaps):

list[list] → passes through unchanged ❌
list[list[str]] → nested strings NOT redacted ❌
list[list[dict]] → nested dicts NOT redacted ❌

Example of the Security Gap

data = {
    "messages": [
        ["Bearer sk-proj-abc123xyz456", "other message"],  # nested list
    ]
}
result = redact_dict(data)
# result["messages"] = [["Bearer sk-proj-abc123xyz456", "other message"]]
# The API key is NOT redacted!

Second Gap: `secrets_masking_processor` (lines 260–269)

The secrets_masking_processor function does not handle list values at all — it only handles str and dict types, passing all other types (including lists) through unchanged:

for key, value in event_dict.items():
    if is_sensitive_key(key):
        result[key] = REDACTED
    elif isinstance(value, str):
        result[key] = redact_value(value)
    elif isinstance(value, dict):
        result[key] = _redact_dict_inner(value)
    else:
        result[key] = value  # Lists pass through unredacted!

Impact

Secrets embedded in nested list structures (e.g., message arrays, batch API responses, log event lists) will not be redacted.
Affects both redact_dict() (used in error details) and secrets_masking_processor (used in structlog processor chain).
Structured log output containing nested lists with API keys or tokens will leak those secrets.

Code Locations

src/cleveragents/shared/redaction.py lines 180–186 (_redact_dict_inner list handling)
src/cleveragents/shared/redaction.py lines 260–269 (secrets_masking_processor missing list handling)

Expected Behavior

Both _redact_dict_inner and secrets_masking_processor should recursively handle nested lists, redacting secrets at any depth.

Discovered by: UAT tester (ca-uat-tester), feature area: Shared Utilities and Core Infrastructure

Subtasks

Refactor _redact_dict_inner list branch to recursively handle list[list] at arbitrary depth
Add list handling to secrets_masking_processor so list values are passed through _redact_dict_inner-equivalent logic
Write BDD unit test scenarios (Behave/Gherkin in features/) covering: list[list[str]], list[list[dict]], and deeply nested list structures
Ensure all new and modified code is 100% statically typed (Pyright must pass with zero errors)
Run nox to verify all quality gates pass (lint, typecheck, unit_tests, coverage_report)

Definition of Done

All subtasks above are completed and checked off
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(shared/redaction): handle nested lists in _redact_dict_inner and secrets_masking_processor), followed by a blank line, then additional lines providing relevant details about the implementation
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/redact-nested-list-secrets)
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests, nox -e coverage_report)
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone v3.3.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/redact-nested-list-secrets` - **Commit Message**: `fix(shared/redaction): handle nested lists in _redact_dict_inner and secrets_masking_processor` - **Milestone**: *(none — backlog)* - **Parent Epic**: #400 ## Background The `_redact_dict_inner` function in `src/cleveragents/shared/redaction.py` handles list values by iterating one level deep, but does **not** recursively handle nested lists (lists within lists). This means secrets inside nested list structures will **not** be redacted, creating a potential secret leakage path in structured log output. ### Current Implementation (lines 180–186) ```python elif isinstance(value, list): result[key] = [ _redact_dict_inner(item) if isinstance(item, dict) else (redact_value(item) if isinstance(item, str) else item) for item in value ] ``` **What is handled:** - `list[dict]` → recursively redacts each dict ✅ - `list[str]` → calls `redact_value()` on each string ✅ **What is NOT handled (security gaps):** - `list[list]` → passes through unchanged ❌ - `list[list[str]]` → nested strings NOT redacted ❌ - `list[list[dict]]` → nested dicts NOT redacted ❌ ### Example of the Security Gap ```python data = { "messages": [ ["Bearer sk-proj-abc123xyz456", "other message"], # nested list ] } result = redact_dict(data) # result["messages"] = [["Bearer sk-proj-abc123xyz456", "other message"]] # The API key is NOT redacted! ``` ### Second Gap: `secrets_masking_processor` (lines 260–269) The `secrets_masking_processor` function does not handle list values at all — it only handles `str` and `dict` types, passing all other types (including lists) through unchanged: ```python for key, value in event_dict.items(): if is_sensitive_key(key): result[key] = REDACTED elif isinstance(value, str): result[key] = redact_value(value) elif isinstance(value, dict): result[key] = _redact_dict_inner(value) else: result[key] = value # Lists pass through unredacted! ``` ### Impact - Secrets embedded in nested list structures (e.g., message arrays, batch API responses, log event lists) will not be redacted. - Affects both `redact_dict()` (used in error details) and `secrets_masking_processor` (used in structlog processor chain). - Structured log output containing nested lists with API keys or tokens will leak those secrets. ### Code Locations - `src/cleveragents/shared/redaction.py` lines 180–186 (`_redact_dict_inner` list handling) - `src/cleveragents/shared/redaction.py` lines 260–269 (`secrets_masking_processor` missing list handling) ### Expected Behavior Both `_redact_dict_inner` and `secrets_masking_processor` should recursively handle nested lists, redacting secrets at any depth. **Discovered by:** UAT tester (ca-uat-tester), feature area: Shared Utilities and Core Infrastructure ## Subtasks - [ ] Refactor `_redact_dict_inner` list branch to recursively handle `list[list]` at arbitrary depth - [ ] Add list handling to `secrets_masking_processor` so list values are passed through `_redact_dict_inner`-equivalent logic - [ ] Write BDD unit test scenarios (Behave/Gherkin in `features/`) covering: `list[list[str]]`, `list[list[dict]]`, and deeply nested list structures - [ ] Ensure all new and modified code is 100% statically typed (Pyright must pass with zero errors) - [ ] Run `nox` to verify all quality gates pass (lint, typecheck, unit_tests, coverage_report) ## Definition of Done - [ ] All subtasks above are completed and checked off - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(shared/redaction): handle nested lists in _redact_dict_inner and secrets_masking_processor`), followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/redact-nested-list-secrets`) - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e coverage_report`) - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.3.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added a new dependency

2026-04-06 08:36:18 +00:00

#400 Epic: Post-MVP Security

freemo referenced this issue

2026-04-06 08:36:43 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo added the

Points

label

2026-04-06 08:37:10 +00:00

freemo referenced this issue

2026-04-06 08:37:16 +00:00