cleveragents-core/docs/api/shared.md

cleveragents.shared — Secret Redaction Utilities

The cleveragents.shared package exposes cross-cutting helpers that can be used from any layer without violating architecture boundaries. At the moment its public surface focuses on a single responsibility: deterministic masking of secrets before they reach logs, error payloads, or tool output.

All exports live in cleveragents.shared.redaction and are re-exported from the package root for convenience.

---

Quick start

from cleveragents.shared import redact_dict, register_pattern

# redact nested structures before logging
payload = {
    "provider_api_key": "sk-ant-api03-secret-1234567890",
    "token_usage": 1280,
    "upstream": {
        "authorization": "Bearer 0123456789ABCDEFGHIJKLMNOPQRSTUV",
    },
}
print(redact_dict(payload))
# {'provider_api_key': '***REDACTED***', 'token_usage': 1280,
#  'upstream': {'authorization': '***REDACTED***'}}

# extend detection with an internal pattern
register_pattern(r"my-secret-[0-9a-f]{16}")

---

Global redaction controls

Symbol	Description
REDACTED	Constant string (***REDACTED***) used as the replacement token.
get_show_secrets() -> bool	Returns the global flag that determines whether redaction is bypassed.
set_show_secrets(value: bool) -> None	Toggles the flag. Accepts only boolean values and is thread-safe.

When get_show_secrets() returns True, the redaction helpers shortcut and return the original data unmodified—useful for debugging in trusted environments. The flag is protected by a lock so it can be flipped safely from CLI options or unit tests.

---

Key helpers

is_sensitive_key(name: str) -> bool

Heuristically classifies dictionary keys as sensitive (e.g. containing "token", "secret", "password"). False positives such as token_count are explicitly whitelisted in the implementation.

redact_value(value: str) -> str

Scans a string for known secret patterns (OpenAI, Anthropic, Google keys, tok_ IDs, bearer tokens, long hex strings) and replaces each match with REDACTED. Additional patterns can be registered at runtime via register_pattern().

redact_dict(data: Mapping[str, Any], *, show_secrets: bool | None = None) -> dict

Recursively traverses nested dictionaries and lists, redacting sensitive keys with REDACTED and scrubbing string values via redact_value. By default it honours the global get_show_secrets() flag; override it per call with show_secrets=True when you explicitly need the original values.

mask_database_url(url: str) -> str

Masks the password segment of SQL connection URLs. SQLite URLs are returned unchanged; any scheme://user:password@host pattern is rewritten as scheme://user:***@host.

register_pattern(pattern: str) -> None

Allows projects or extensions to add custom regular expressions to the redaction list. Patterns are compiled under a lock so multiple workers can register safely. Passing an empty pattern raises ValueError.

---

Structlog integration

secrets_masking_processor(logger, method_name, event_dict) implements the structlog processor contract. It walks every value in the event dictionary, redacting keys detected by is_sensitive_key and values using redact_value and redact_dict. The processor is designed to be inserted near the top of the structlog pipeline so that downstream formatters never observe raw secrets.

import structlog
from cleveragents.shared import secrets_masking_processor

structlog.configure(
    processors=[
        secrets_masking_processor,
        structlog.processors.JSONRenderer(),
    ],
)

---

Thread-safety notes

The module guards both the global show_secrets flag and the list of compiled patterns with locks. You can safely call set_show_secrets() or register_pattern() from concurrent tasks without corrupting shared state.

Because all helpers return new dictionaries (never mutating inputs), you may confidently hand the redacted payloads to logging subsystems or telemetry pipelines without affecting the original objects used by business logic.