# Secrets Handling (SEC5)

CleverAgents automatically masks API keys, tokens, and credentials in
CLI output, structured logs, and error messages.  This document covers
the design and usage of the redaction subsystem.

## Overview

The `cleveragents.shared.redaction` module provides centralised,
pattern-based secret detection and masking.  It is integrated into:

- **CLI output** via `format_output()` in `cleveragents.cli.formatting`
- **Error handlers** in `main.py`, `project.py`, and `auto_debug.py`
- **structlog processors** via `secrets_masking_processor`
- **Settings repr** via `Settings.__repr__`

## Detected patterns

| Pattern family | Regex summary | Example |
|---|---|---|
| OpenAI keys | `sk-(?:proj-)?[A-Za-z0-9_-]{10,}` | `sk-proj-abc123…` |
| Anthropic keys | `sk-ant-[A-Za-z0-9_-]{10,}` | `sk-ant-api03-xyz…` |
| Token IDs | `tok_[A-Za-z0-9]{10,}` | `tok_01HXYZ…` |
| Bearer tokens | `Bearer\s+[A-Za-z0-9._~+/=-]{20,}` | `Bearer eyJhb…` |
| Generic keys | `(?:key\|KEY)-[A-Za-z0-9]{20,}` | `KEY-abcdef…` |

Sensitive **key names** (dictionary keys, config fields) are also
detected by substring matching:

`api_key`, `apikey`, `password`, `passwd`, `secret`, `token`,
`credential`, `private_key`, `access_key`, `auth`

False-positive key names like `token_count`, `max_tokens`, etc. are
excluded.

## Replacement token

All detected secrets are replaced with `***REDACTED***`.

## Global `--show-secrets` flag

Pass `--show-secrets` to any CLI command to reveal secrets in that
invocation:

```bash
agents --show-secrets info
agents --show-secrets diagnostics
```

The flag sets a global thread-safe boolean that the redaction layer
checks.  When `True`, all masking is bypassed.

Programmatic access:

```python
from cleveragents.shared.redaction import get_show_secrets, set_show_secrets

set_show_secrets(True)   # reveal secrets
set_show_secrets(False)  # re-enable masking (default)
```

## Custom patterns

Register additional secret patterns at runtime:

```python
from cleveragents.shared.redaction import register_pattern

register_pattern(r"myorg_[a-z0-9]{32}")
```

Patterns are compiled once and appended to the global list.

## Database URL masking

`mask_database_url()` replaces embedded passwords in connection
strings while preserving the host and database name:

```python
from cleveragents.shared.redaction import mask_database_url

mask_database_url("postgresql://user:pass@host/db")
# → "postgresql://user:***@host/db"

mask_database_url("sqlite:///data.db")
# → "sqlite:///data.db"  (unchanged)
```

## structlog integration

The `secrets_masking_processor` is inserted into the structlog
processor chain by `configure_structlog()`:

```python
from cleveragents.config.logging import configure_structlog

configure_structlog(env="production", log_level="INFO")
```

Every log event is scanned for secret patterns before rendering.

## Settings safety

`Settings.__repr__()` masks any field whose name matches a sensitive
key pattern, so accidental `print(settings)` or debug output never
leaks credentials.

## Testing

- **Behave**: `features/security_secrets.feature` (43 scenarios)
- **Robot Framework**: `robot/security_secrets.robot` (10 smoke tests)
- **ASV benchmarks**: `benchmarks/security_secrets_bench.py`