cleveragents/cleveragents-core

Fork 3

feat(security): implement Secret Masking in LLM Context Construction #573

New issue

Open

opened 2026-03-04 23:40:08 +00:00 by freemo · 3 comments

freemo commented

2026-03-04 23:40:08 +00:00

Owner

Metadata

Field	Value
Commit Message	`feat(security): implement Secret Masking in LLM Context Construction`
Branch	`feature/m4-secret-masking-llm-context`

Summary

The specification requires that before constructing LLM prompts, the context builder scans for secret patterns and replaces them with [REDACTED] (spec line ~43889). While the shared redaction.py module exists and handles CLI/log redaction, it is not wired into the LLM context assembly path.

Spec Reference

Section: Architecture > Security Model > Secret Management > item 4
Lines: 43888-43891

Description

The spec states:

"Secret masking in LLM context: Before constructing LLM prompts, the context builder scans for patterns matching known secret formats and replaces them with [REDACTED]. This prevents accidental exposure of secrets in LLM training data."

Current State

cleveragents/shared/redaction.py has redact_value(), redact_dict(), is_sensitive_key(), and secrets_masking_processor() (for structlog)
These are used in CLI output and structured logging
They are NOT applied to LLM prompt/context construction

Implementation Needed

Add a redact_context_for_llm(content: str) -> str function in shared/redaction.py that scans text content for secret patterns using the existing _SECRET_PATTERNS list
Wire this function into the context assembly pipeline -- specifically in the PreambleGenerator or as a final pass before the assembled context is sent to the LLM
Also apply to: action arguments, invariant text, session messages before they enter the LLM prompt
Ensure resource content (file contents from read_file etc.) is also scanned before inclusion in context
Tests: verify secrets in file content, environment variable dumps, and config snippets are redacted before reaching the LLM

Epic: Security & Safety Hardening #362

Suggested Milestone

v3.3.0 (Security hardening)

Priority

High

Suggested Assignee

@CoreRasurae (validation/context pipeline)

Subtasks

Code: Implement redact_context_for_llm(content: str) -> str in shared/redaction.py using existing _SECRET_PATTERNS
Code: Wire redaction into the context assembly pipeline (PreambleGenerator or final pass before LLM invocation)
Code: Apply redaction to action arguments, invariant text, session messages, and resource content before LLM prompt inclusion
Docs: Update security documentation to describe secret masking in LLM context path
Behave tests: Add BDD feature file features/security/secret_masking_llm_context.feature verifying secrets are redacted before LLM
Robot tests: Add Robot Framework integration test: include a mock secret in resource content, verify [REDACTED] appears in assembled context
ASV benchmarks: Add ASV benchmark for redaction throughput on large context payloads (benchmarks/bench_secret_redaction.py)
Quality: coverage ≥97%: Verify via nox -s coverage_report
Quality: nox full suite: Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

## Metadata | Field | Value | |-------|-------| | **Commit Message** | `feat(security): implement Secret Masking in LLM Context Construction` | | **Branch** | `feature/m4-secret-masking-llm-context` | ## Summary The specification requires that **before constructing LLM prompts**, the context builder scans for secret patterns and replaces them with `[REDACTED]` (spec line ~43889). While the shared `redaction.py` module exists and handles CLI/log redaction, it is **not wired into the LLM context assembly path**. ## Spec Reference - **Section**: Architecture > Security Model > Secret Management > item 4 - **Lines**: 43888-43891 ## Description The spec states: > "Secret masking in LLM context: Before constructing LLM prompts, the context builder scans for patterns matching known secret formats and replaces them with `[REDACTED]`. This prevents accidental exposure of secrets in LLM training data." ### Current State - `cleveragents/shared/redaction.py` has `redact_value()`, `redact_dict()`, `is_sensitive_key()`, and `secrets_masking_processor()` (for structlog) - These are used in CLI output and structured logging - They are **NOT** applied to LLM prompt/context construction ### Implementation Needed - [ ] Add a `redact_context_for_llm(content: str) -> str` function in `shared/redaction.py` that scans text content for secret patterns using the existing `_SECRET_PATTERNS` list - [ ] Wire this function into the context assembly pipeline -- specifically in the `PreambleGenerator` or as a final pass before the assembled context is sent to the LLM - [ ] Also apply to: action arguments, invariant text, session messages before they enter the LLM prompt - [ ] Ensure resource content (file contents from `read_file` etc.) is also scanned before inclusion in context - [ ] Tests: verify secrets in file content, environment variable dumps, and config snippets are redacted before reaching the LLM ## Related Issues - Epic: Security & Safety Hardening #362 ## Suggested Milestone v3.3.0 (Security hardening) ## Priority High ## Suggested Assignee @CoreRasurae (validation/context pipeline) ## Subtasks - [ ] **Code**: Implement `redact_context_for_llm(content: str) -> str` in `shared/redaction.py` using existing `_SECRET_PATTERNS` - [ ] **Code**: Wire redaction into the context assembly pipeline (PreambleGenerator or final pass before LLM invocation) - [ ] **Code**: Apply redaction to action arguments, invariant text, session messages, and resource content before LLM prompt inclusion - [ ] **Docs**: Update security documentation to describe secret masking in LLM context path - [ ] **Behave tests**: Add BDD feature file `features/security/secret_masking_llm_context.feature` verifying secrets are redacted before LLM - [ ] **Robot tests**: Add Robot Framework integration test: include a mock secret in resource content, verify `[REDACTED]` appears in assembled context - [ ] **ASV benchmarks**: Add ASV benchmark for redaction throughput on large context payloads (`benchmarks/bench_secret_redaction.py`) - [ ] **Quality: coverage ≥97%**: Verify via `nox -s coverage_report` - [ ] **Quality: nox full suite**: Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.

CoreRasurae was assigned by freemo

2026-03-05 00:29:59 +00:00

freemo added this to the v3.3.0 milestone

2026-03-05 00:29:59 +00:00

freemo added the

labels

2026-03-05 00:30:00 +00:00

freemo added a new dependency

2026-03-05 00:34:08 +00:00

#362 Epic: Security & Safety Hardening

freemo added

and removed

labels

2026-03-09 20:09:44 +00:00

freemo commented

2026-03-09 20:17:37 +00:00

Author

Owner

PM Note (Day 29) — State Change

Changes:

State: Unverified → Verified

Rationale: Secret masking in LLM context construction (5 SP, M3) is a Must-have security feature. The specification clearly defines the masking requirements. Verified and ready for @CoreRasurae to begin work. This should be prioritized after #581 (AuditService wiring).

**PM Note (Day 29) — State Change** **Changes:** - **State**: Unverified → **Verified** **Rationale:** Secret masking in LLM context construction (5 SP, M3) is a Must-have security feature. The specification clearly defines the masking requirements. Verified and ready for @CoreRasurae to begin work. This should be prioritized after #581 (AuditService wiring).

CoreRasurae added

and removed

labels

2026-03-09 23:45:37 +00:00

CoreRasurae referenced this issue from a commit

2026-03-10 00:21:31 +00:00

feat(security): implement Secret Masking in LLM Context Construction

~~CoreRasurae referenced this issue 2026-03-10 00:21:48 +00:00~~

feat(security): implement Secret Masking in LLM Context Construction #656

CoreRasurae added

and removed

labels

2026-03-10 00:57:06 +00:00

CoreRasurae commented

2026-03-10 00:57:48 +00:00

Member

Implementation Summary

Branch: feature/m4-secret-masking-llm-context
Commit: 286a96a263a11d48ca00b4ab22b8fb167008b4b0
PR: #656

Changes

src/cleveragents/shared/redaction.py: Added LLM_REDACTED = "[REDACTED]" constant and redact_context_for_llm(content: str) -> str function that scans text for secret patterns using existing _SECRET_PATTERNS and replaces matches with [REDACTED] as specified by the architecture spec (Secret Management, item 4).
src/cleveragents/shared/__init__.py: Exported redact_context_for_llm and LLM_REDACTED.
src/cleveragents/application/services/acms_service.py and acms_pipeline.py: Wired redaction as a final pass in assemble() — fragment content and preamble text are redacted before being returned in the ContextPayload.
docs/reference/secrets_handling.md: Updated with LLM context masking documentation.
features/security/secret_masking_llm_context.feature: 9 BDD scenarios covering OpenAI keys, Anthropic keys, Bearer tokens, multiple secrets, normal text, empty content, constant value, pipeline integration, and preamble redaction.
robot/secret_masking_llm_context.robot: 5 integration tests.
benchmarks/bench_secret_redaction.py: ASV benchmark measuring redaction throughput at 100/1K/10K/100K content sizes.

Quality Gates

Gate	Result
`nox -s lint`	PASS
`nox -s typecheck`	PASS (0 errors)
`nox -s unit_tests`	PASS (9708 scenarios, 0 failed)
`nox -s integration_tests`	17 pre-existing failures (Cli Plan Context Commands / Core Cli Commands) — all new tests pass
`nox -s coverage_report`	PASS (99% >= 97% threshold)

Note on Pre-existing Failures

17 integration test failures in Cli Plan Context Commands and Core Cli Commands robot suites pre-date this change and are unrelated to secret masking. These need separate investigation.

## Implementation Summary **Branch**: `feature/m4-secret-masking-llm-context` **Commit**: `286a96a263a11d48ca00b4ab22b8fb167008b4b0` **PR**: #656 ### Changes - **`src/cleveragents/shared/redaction.py`**: Added `LLM_REDACTED = "[REDACTED]"` constant and `redact_context_for_llm(content: str) -> str` function that scans text for secret patterns using existing `_SECRET_PATTERNS` and replaces matches with `[REDACTED]` as specified by the architecture spec (Secret Management, item 4). - **`src/cleveragents/shared/__init__.py`**: Exported `redact_context_for_llm` and `LLM_REDACTED`. - **`src/cleveragents/application/services/acms_service.py`** and **`acms_pipeline.py`**: Wired redaction as a final pass in `assemble()` — fragment content and preamble text are redacted before being returned in the `ContextPayload`. - **`docs/reference/secrets_handling.md`**: Updated with LLM context masking documentation. - **`features/security/secret_masking_llm_context.feature`**: 9 BDD scenarios covering OpenAI keys, Anthropic keys, Bearer tokens, multiple secrets, normal text, empty content, constant value, pipeline integration, and preamble redaction. - **`robot/secret_masking_llm_context.robot`**: 5 integration tests. - **`benchmarks/bench_secret_redaction.py`**: ASV benchmark measuring redaction throughput at 100/1K/10K/100K content sizes. ### Quality Gates | Gate | Result | |------|--------| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (9708 scenarios, 0 failed) | | `nox -s integration_tests` | 17 pre-existing failures (Cli Plan Context Commands / Core Cli Commands) — all new tests pass | | `nox -s coverage_report` | PASS (99% >= 97% threshold) | ### Note on Pre-existing Failures 17 integration test failures in `Cli Plan Context Commands` and `Core Cli Commands` robot suites pre-date this change and are unrelated to secret masking. These need separate investigation.

CoreRasurae referenced this issue from a commit

2026-03-10 01:01:38 +00:00

feat(security): implement Secret Masking in LLM Context Construction

CoreRasurae referenced this issue from a pull request that will close it,

2026-03-10 01:02:17 +00:00

feat(security): implement Secret Masking in LLM Context Construction #656

CoreRasurae added a new dependency

2026-03-10 01:04:21 +00:00

#656 feat(security): implement Secret Masking in LLM Context Construction

freemo commented

2026-03-11 05:42:12 +00:00

Author

Owner

PM Acknowledgment (Day 31):

Thank you @CoreRasurae. PR #656 submitted.

Status: PR #656 has merge conflict and no reviews. Milestone v3.3.0.

Action needed: Rebase and request review once TDD infra work (#627, #629) is complete.

Priority: Medium — after TDD infra.

**PM Acknowledgment (Day 31)**: Thank you @CoreRasurae. PR #656 submitted. **Status**: PR #656 has merge conflict and no reviews. Milestone v3.3.0. **Action needed**: Rebase and request review once TDD infra work (#627, #629) is complete. **Priority**: Medium — after TDD infra.

freemo referenced this issue

2026-03-11 05:43:25 +00:00

Epic: Security & Safety Hardening #362

CoreRasurae referenced this issue

2026-03-12 20:54:16 +00:00

feat(security): implement Secret Masking in LLM Context Construction #656

CoreRasurae referenced this issue

2026-03-13 15:04:19 +00:00

feat(security): implement Secret Masking in LLM Context Construction #656

CoreRasurae referenced this issue from a commit

2026-03-13 15:35:53 +00:00

feat(security): implement Secret Masking in LLM Context Construction