UAT: `configure_structlog()` is never called at application startup — secrets masking processor not installed in structlog pipeline #1883

New issue

Open

opened 2026-04-03 00:05:46 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 00:05:46 +00:00

Owner

Metadata

Branch: fix/startup-configure-structlog-secrets-masking
Commit Message: fix(logging): call configure_structlog() at application startup to install secrets masking processor
Milestone: v3.6.0
Parent Epic: #945

Background and Context

The configure_structlog() function in src/cleveragents/config/logging.py is defined but never called anywhere in the production codebase. This is a security issue: the secrets_masking_processor from cleveragents.shared.redaction is never installed in the structlog processor chain, meaning API keys, passwords, and other secrets passed as log fields are emitted in plaintext.

The specification (§Observability, §Structured Logging) requires that all logging uses structlog with JSON output format and that secrets are automatically redacted. The configure_structlog() function is designed to set up this pipeline, but it is never invoked.

Current Behavior

configure_structlog() is defined in src/cleveragents/config/logging.py but has zero callers in production code (verified by grep).
The structlog pipeline uses default processors — no secrets masking, no JSON renderer.
API keys, passwords, and other secrets passed as log fields are NOT redacted.

Steps to Reproduce:

import sys
sys.path.insert(0, 'src')
import structlog
from cleveragents.shared.redaction import secrets_masking_processor

config = structlog.get_config()
in_chain = any(
    p is secrets_masking_processor or
    getattr(p, '__name__', '') == 'secrets_masking_processor'
    for p in config['processors']
)
print(f'secrets_masking_processor in chain: {in_chain}')  # Prints: False

# Demonstrate secret leaking
logger = structlog.get_logger('test')
logger.info('test_event', api_key='sk-proj-secret123abc')
# Output: api_key=sk-proj-secret123abc  (NOT redacted!)

Expected Behavior

Per the specification (§Observability, §Structured Logging):

configure_structlog() is called during application startup with the correct env and log_level from settings.
The secrets_masking_processor appears in the structlog processor chain after startup.
The JSON renderer is active in production mode.
API keys and other secrets are automatically redacted in all log output.

Affected Code Locations

src/cleveragents/config/logging.py — defines configure_structlog() but it is never called
src/cleveragents/__main__.py — application entry point, does not call configure_structlog()
src/cleveragents/application/container.py — DI container, does not call configure_structlog()

Fix Required: Call configure_structlog(env=settings.env, log_level=settings.log_level) during application startup (e.g., in the DI container or CLI main callback).

Subtasks

Call configure_structlog() at application startup with correct env and log_level
Verify secrets_masking_processor is installed in the structlog pipeline after startup
Verify JSON renderer is used in production mode
Add BDD test scenarios verifying startup logging configuration
Add integration test confirming secrets are redacted in log output
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

configure_structlog() is called during application startup
The secrets_masking_processor appears in the structlog processor chain after startup
API keys and other secrets are redacted in log output (not emitted in plaintext)
Tests verify the startup configuration (BDD scenarios + integration test)
All nox stages pass
Coverage >= 97%
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/startup-configure-structlog-secrets-masking` - **Commit Message**: `fix(logging): call configure_structlog() at application startup to install secrets masking processor` - **Milestone**: v3.6.0 - **Parent Epic**: #945 ## Background and Context The `configure_structlog()` function in `src/cleveragents/config/logging.py` is defined but never called anywhere in the production codebase. This is a **security issue**: the `secrets_masking_processor` from `cleveragents.shared.redaction` is never installed in the structlog processor chain, meaning API keys, passwords, and other secrets passed as log fields are emitted in plaintext. The specification (§Observability, §Structured Logging) requires that all logging uses structlog with JSON output format and that secrets are automatically redacted. The `configure_structlog()` function is designed to set up this pipeline, but it is never invoked. ## Current Behavior - `configure_structlog()` is defined in `src/cleveragents/config/logging.py` but has **zero callers** in production code (verified by grep). - The structlog pipeline uses default processors — no secrets masking, no JSON renderer. - API keys, passwords, and other secrets passed as log fields are **NOT redacted**. **Steps to Reproduce:** ```python import sys sys.path.insert(0, 'src') import structlog from cleveragents.shared.redaction import secrets_masking_processor config = structlog.get_config() in_chain = any( p is secrets_masking_processor or getattr(p, '__name__', '') == 'secrets_masking_processor' for p in config['processors'] ) print(f'secrets_masking_processor in chain: {in_chain}') # Prints: False # Demonstrate secret leaking logger = structlog.get_logger('test') logger.info('test_event', api_key='sk-proj-secret123abc') # Output: api_key=sk-proj-secret123abc (NOT redacted!) ``` ## Expected Behavior Per the specification (§Observability, §Structured Logging): - `configure_structlog()` is called during application startup with the correct `env` and `log_level` from settings. - The `secrets_masking_processor` appears in the structlog processor chain after startup. - The JSON renderer is active in production mode. - API keys and other secrets are automatically redacted in all log output. ## Affected Code Locations - `src/cleveragents/config/logging.py` — defines `configure_structlog()` but it is never called - `src/cleveragents/__main__.py` — application entry point, does not call `configure_structlog()` - `src/cleveragents/application/container.py` — DI container, does not call `configure_structlog()` **Fix Required:** Call `configure_structlog(env=settings.env, log_level=settings.log_level)` during application startup (e.g., in the DI container or CLI main callback). ## Subtasks - [ ] Call `configure_structlog()` at application startup with correct `env` and `log_level` - [ ] Verify `secrets_masking_processor` is installed in the structlog pipeline after startup - [ ] Verify JSON renderer is used in production mode - [ ] Add BDD test scenarios verifying startup logging configuration - [ ] Add integration test confirming secrets are redacted in log output - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - [ ] `configure_structlog()` is called during application startup - [ ] The `secrets_masking_processor` appears in the structlog processor chain after startup - [ ] API keys and other secrets are redacted in log output (not emitted in plaintext) - [ ] Tests verify the startup configuration (BDD scenarios + integration test) - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.6.0 milestone

2026-04-03 00:05:58 +00:00

freemo referenced this issue

2026-04-03 00:08:26 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 471 — v3.7.0 #1891

freemo added a new dependency

2026-04-03 00:10:36 +00:00

#945 Legendary: Specification Alignment — Close All Gaps Between Implementation and Specification

freemo commented

2026-04-03 00:12:58 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: MoSCoW/Should Have — bug or error handling improvement.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Should Have — bug or error handling improvement. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-03 00:23:51 +00:00

UAT: structlog debug logs written to stdout by default — pollutes JSON/YAML CLI output and breaks programmatic parsing #1958