cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

[AUTO-ARCH-16] docs(spec): update prompt injection mitigation to use XML-delimited tags #9566

Closed
HAL9000 wants to merge 1 commits from auto-arch-16/spec-xml-prompt-injection-mitigation into master
pull from: auto-arch-16/spec-xml-prompt-injection-mitigation
merge into: cleveragents:master
Conversation 4 Commits 1 Files Changed 1
+1 -1
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specification.md
+1 -1
View File
@@ -46246,7 +46246,7 @@ In server mode, where user-provided content may flow into LLM prompts, CleverAge
1. **Input sanitization**: User-provided text in action arguments, invariant text, and session prompts is sanitized before inclusion in LLM prompts. HTML entities, control characters, and known injection patterns are escaped or rejected.
2. **Prompt boundary markers**: System prompts and user content are separated by clear boundary markers that the LLM is instructed to recognize. The system prompt explicitly states that content between `[USER_CONTENT_START]` and `[USER_CONTENT_END]` markers is user-provided and should not be treated as system instructions.
2. **Prompt boundary markers**: System prompts wrap user content inside XML-delimited sections (for example, `<user_content>...</user_content>`). All user-supplied text MUST be sanitized by escaping `<`, `>`, and `&` before embedding it in the XML wrapper; `_sanitize_xml_content()` provides the reference implementation. Combining the XML container with sanitization prevents prompt injection via forged closing tags.
3. **Output validation**: LLM outputs that are used as tool invocations are validated against the tool's JSON Schema before execution. This prevents the LLM from being tricked into invoking tools with malicious parameters.