UAT: JWT, GitHub PAT, and GitLab PAT patterns missing from base redaction.py — tokens leak in logs when error_handling not imported #1749

New issue

freemo · 2026-04-02T23:41:19Z

freemo commented

2026-04-02 23:41:19 +00:00

Metadata

Branch: fix/redaction-missing-jwt-pat-patterns
Commit Message: fix(redaction): move JWT and PAT patterns into base _SECRET_PATTERNS list
Milestone: v3.3.0
Parent Epic: #362

Bug Report

Summary

The cleveragents.shared.redaction module is missing JWT token, GitHub PAT, and GitLab PAT detection patterns. These patterns are only registered when cleveragents.core.error_handling is imported (which calls register_pattern() at module load time). If any code path uses redaction.py directly without first importing error_handling.py, JWT tokens and PATs embedded in log message strings will NOT be redacted.

Expected Behavior (from spec)

Per specification section "Secret Management" (line 46077):

String values are scanned for known secret patterns (OpenAI keys, Anthropic keys, JWT tokens, GitHub PATs, GitLab PATs, Bearer tokens).

The redaction.py module is described as the single redaction implementation shared with structlog processors and CLI output. All patterns should be defined in redaction.py itself, not depend on a side-effect import of error_handling.py.

Actual Behavior

cleveragents/shared/redaction.py only contains these patterns:

OpenAI keys (sk-proj-..., sk-...)
Anthropic keys (sk-ant-...)
Google/Gemini API keys (AIzaSy...)
Token IDs (tok_...)
Bearer tokens (Bearer ...)
Generic long hex/base64 keys (key-...)

Missing patterns (only added by error_handling.py at import time):

JWT tokens (eyJ...)
GitHub PATs (ghp_...)
GitHub App tokens (ghs_...)
GitLab PATs (glpat-...)

Steps to Reproduce

# Import ONLY redaction (not error_handling)
from cleveragents.shared.redaction import redact_value, secrets_masking_processor, REDACTED

# JWT token in a log message string (non-sensitive key name)
jwt = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.test'
event_dict = {'event': f'auth failed with jwt {jwt}', 'user': 'alice'}
result = secrets_masking_processor(None, 'info', event_dict)
print(result['event'])  # JWT NOT redacted!

# GitHub PAT in a log message
github_pat = 'ghp_abcdefghijklmnopqrstuvwxyz0123456'
event_dict2 = {'event': f'auth with {github_pat}', 'user': 'alice'}
result2 = secrets_masking_processor(None, 'info', event_dict2)
print(result2['event'])  # GitHub PAT NOT redacted!

Code Location

src/cleveragents/shared/redaction.py — _SECRET_PATTERNS list (missing JWT/PAT patterns)

src/cleveragents/core/error_handling.py — lines registering patterns at module load:

_register_pattern(r"eyJ[a-zA-Z0-9_-]{10,}")  # JWT tokens
_register_pattern(r"ghp_[a-zA-Z0-9]{30,}")  # GitHub PATs
_register_pattern(r"ghs_[a-zA-Z0-9]{30,}")  # GitHub App tokens
_register_pattern(r"glpat-[a-zA-Z0-9_-]{20,}")  # GitLab PATs

Fix

Move the JWT, GitHub PAT, GitHub App token, and GitLab PAT patterns from error_handling.py into the _SECRET_PATTERNS list in redaction.py directly.

Severity

High — Authentication tokens (JWTs used as bearer tokens, GitHub/GitLab PATs used as API keys) can leak in structured logs if the structlog processor is initialized before error_handling is imported.

Subtasks

Move JWT pattern (eyJ[a-zA-Z0-9_-]{10,}) from error_handling.py into _SECRET_PATTERNS in redaction.py
Move GitHub PAT pattern (ghp_[a-zA-Z0-9]{30,}) from error_handling.py into _SECRET_PATTERNS in redaction.py
Move GitHub App token pattern (ghs_[a-zA-Z0-9]{30,}) from error_handling.py into _SECRET_PATTERNS in redaction.py
Move GitLab PAT pattern (glpat-[a-zA-Z0-9_-]{20,}) from error_handling.py into _SECRET_PATTERNS in redaction.py
Remove the now-redundant _register_pattern() calls from error_handling.py
Tests (Behave): Add BDD scenarios verifying JWT, GitHub PAT, GitHub App token, and GitLab PAT are redacted when error_handling is NOT imported
Tests (Behave): Add BDD scenarios verifying no double-redaction occurs when error_handling IS imported
Tests (Robot): Add integration test confirming structlog processor redacts all token types end-to-end
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(redaction): move JWT and PAT patterns into base _SECRET_PATTERNS list), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/redaction-missing-jwt-pat-patterns).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/redaction-missing-jwt-pat-patterns` - **Commit Message**: `fix(redaction): move JWT and PAT patterns into base _SECRET_PATTERNS list` - **Milestone**: v3.3.0 - **Parent Epic**: #362 ## Bug Report ### Summary The `cleveragents.shared.redaction` module is missing JWT token, GitHub PAT, and GitLab PAT detection patterns. These patterns are only registered when `cleveragents.core.error_handling` is imported (which calls `register_pattern()` at module load time). If any code path uses `redaction.py` directly without first importing `error_handling.py`, JWT tokens and PATs embedded in log message strings will NOT be redacted. ### Expected Behavior (from spec) Per specification section "Secret Management" (line 46077): > String values are scanned for known secret patterns (OpenAI keys, Anthropic keys, JWT tokens, GitHub PATs, GitLab PATs, Bearer tokens). The `redaction.py` module is described as the single redaction implementation shared with structlog processors and CLI output. All patterns should be defined in `redaction.py` itself, not depend on a side-effect import of `error_handling.py`. ### Actual Behavior `cleveragents/shared/redaction.py` only contains these patterns: - OpenAI keys (`sk-proj-...`, `sk-...`) - Anthropic keys (`sk-ant-...`) - Google/Gemini API keys (`AIzaSy...`) - Token IDs (`tok_...`) - Bearer tokens (`Bearer ...`) - Generic long hex/base64 keys (`key-...`) Missing patterns (only added by `error_handling.py` at import time): - JWT tokens (`eyJ...`) - GitHub PATs (`ghp_...`) - GitHub App tokens (`ghs_...`) - GitLab PATs (`glpat-...`) ### Steps to Reproduce ```python # Import ONLY redaction (not error_handling) from cleveragents.shared.redaction import redact_value, secrets_masking_processor, REDACTED # JWT token in a log message string (non-sensitive key name) jwt = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.test' event_dict = {'event': f'auth failed with jwt {jwt}', 'user': 'alice'} result = secrets_masking_processor(None, 'info', event_dict) print(result['event']) # JWT NOT redacted! # GitHub PAT in a log message github_pat = 'ghp_abcdefghijklmnopqrstuvwxyz0123456' event_dict2 = {'event': f'auth with {github_pat}', 'user': 'alice'} result2 = secrets_masking_processor(None, 'info', event_dict2) print(result2['event']) # GitHub PAT NOT redacted! ``` ### Code Location - `src/cleveragents/shared/redaction.py` — `_SECRET_PATTERNS` list (missing JWT/PAT patterns) - `src/cleveragents/core/error_handling.py` — lines registering patterns at module load: ```python _register_pattern(r"eyJ[a-zA-Z0-9_-]{10,}") # JWT tokens _register_pattern(r"ghp_[a-zA-Z0-9]{30,}") # GitHub PATs _register_pattern(r"ghs_[a-zA-Z0-9]{30,}") # GitHub App tokens _register_pattern(r"glpat-[a-zA-Z0-9_-]{20,}") # GitLab PATs ``` ### Fix Move the JWT, GitHub PAT, GitHub App token, and GitLab PAT patterns from `error_handling.py` into the `_SECRET_PATTERNS` list in `redaction.py` directly. ### Severity **High** — Authentication tokens (JWTs used as bearer tokens, GitHub/GitLab PATs used as API keys) can leak in structured logs if the structlog processor is initialized before `error_handling` is imported. ## Subtasks - [ ] Move JWT pattern (`eyJ[a-zA-Z0-9_-]{10,}`) from `error_handling.py` into `_SECRET_PATTERNS` in `redaction.py` - [ ] Move GitHub PAT pattern (`ghp_[a-zA-Z0-9]{30,}`) from `error_handling.py` into `_SECRET_PATTERNS` in `redaction.py` - [ ] Move GitHub App token pattern (`ghs_[a-zA-Z0-9]{30,}`) from `error_handling.py` into `_SECRET_PATTERNS` in `redaction.py` - [ ] Move GitLab PAT pattern (`glpat-[a-zA-Z0-9_-]{20,}`) from `error_handling.py` into `_SECRET_PATTERNS` in `redaction.py` - [ ] Remove the now-redundant `_register_pattern()` calls from `error_handling.py` - [ ] Tests (Behave): Add BDD scenarios verifying JWT, GitHub PAT, GitHub App token, and GitLab PAT are redacted when `error_handling` is NOT imported - [ ] Tests (Behave): Add BDD scenarios verifying no double-redaction occurs when `error_handling` IS imported - [ ] Tests (Robot): Add integration test confirming structlog processor redacts all token types end-to-end - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(redaction): move JWT and PAT patterns into base _SECRET_PATTERNS list`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/redaction-missing-jwt-pat-patterns`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.3.0 milestone

2026-04-02 23:41:42 +00:00

freemo added the

labels

2026-04-02 23:42:32 +00:00

freemo referenced this issue

2026-04-02 23:48:06 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 451 — v3.7.0 #1778

freemo added a new dependency

2026-04-02 23:49:24 +00:00

#362 Epic: Security & Safety Hardening

freemo removed the

labels

2026-04-03 00:28:14 +00:00

freemo commented

2026-04-05 09:17:05 +00:00

Issue triaged by project owner:

State: Verified
Priority: Critical — Missing redaction patterns for JWT, GitHub PAT, and GitLab PAT tokens is a security risk. Sensitive credentials could be exposed in logs, session exports, or error messages.
Milestone: v3.3.0 — Keeping in v3.3.0 where security hardening (Epic #362) is scoped.
MoSCoW: Must Have — Security-related credential exposure is always Must Have. The specification requires robust redaction of sensitive data.
Parent Epic: #362 (Security & Safety Hardening) — Credential redaction is a core security feature.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Critical — Missing redaction patterns for JWT, GitHub PAT, and GitLab PAT tokens is a security risk. Sensitive credentials could be exposed in logs, session exports, or error messages. - **Milestone**: v3.3.0 — Keeping in v3.3.0 where security hardening (Epic #362) is scoped. - **MoSCoW**: Must Have — Security-related credential exposure is always Must Have. The specification requires robust redaction of sensitive data. - **Parent Epic**: #362 (Security & Safety Hardening) — Credential redaction is a core security feature. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

HAL9000 added the

labels

2026-04-13 02:21:35 +00:00

Sign in to join this conversation.