forked from HAL9000/cleveragents-core
57 lines
3.0 KiB
Gherkin
57 lines
3.0 KiB
Gherkin
Feature: LLM System Prompt Context Rendering
|
|
As a developer
|
|
I want LLM agent system prompts to support JINJA2 template rendering with runtime context
|
|
So that agents can receive dynamic context information in their system prompts
|
|
|
|
Background:
|
|
Given a JINJA2 template renderer is available
|
|
|
|
Scenario: System prompt with simple context variable is rendered at runtime
|
|
Given an LLM agent with system prompt "You are analyzing: {{ context.topic }}"
|
|
When the agent processes a message with context containing topic "AI Ethics"
|
|
Then the rendered system prompt should contain "You are analyzing: AI Ethics"
|
|
|
|
Scenario: System prompt with multiple context variables
|
|
Given an LLM agent with system prompt "Topic: {{ context.topic }}, Audience: {{ context.audience }}"
|
|
When the agent processes a message with multiple context variables topic "Machine Learning" and audience "students"
|
|
Then the rendered system prompt should contain "Topic: Machine Learning, Audience: students"
|
|
|
|
Scenario: System prompt with nested context variables
|
|
Given an LLM agent with system prompt "Paper topic: {{ context.paper_details.topic }}"
|
|
When the agent processes a message with nested context paper_details.topic "COVID-19 Impact"
|
|
Then the rendered system prompt should contain "Paper topic: COVID-19 Impact"
|
|
|
|
Scenario: System prompt with JINJA2 filters
|
|
Given an LLM agent with system prompt "Topic: {{ context.topic | tojson }}"
|
|
When the agent processes a message with context containing topic "AI Safety"
|
|
Then the rendered system prompt should contain '"AI Safety"'
|
|
|
|
Scenario: System prompt rendering fallback on error
|
|
Given an LLM agent with system prompt "Invalid template: {{ context.nonexistent.deeply.nested }}"
|
|
When the agent processes a message with empty context
|
|
Then the agent should fall back to the original system prompt
|
|
And no exception should be raised
|
|
|
|
Scenario: System prompt without templates is unchanged
|
|
Given an LLM agent with system prompt "You are a helpful assistant."
|
|
When the agent processes a message with any context
|
|
Then the system prompt should remain "You are a helpful assistant."
|
|
|
|
Scenario: Config parser preserves template syntax during YAML loading
|
|
Given a YAML config file with system_prompt containing "{{ context.topic }}"
|
|
When the config is parsed
|
|
Then the system_prompt should still contain "{{ context.topic }}"
|
|
And the system_prompt should not contain empty strings
|
|
|
|
Scenario: Template markers are protected from premature rendering
|
|
Given a YAML config with multiple agents having templated system prompts
|
|
When the config is loaded at startup
|
|
Then all template markers {{ and }} should be preserved
|
|
And no templates should be rendered with empty context
|
|
|
|
Scenario: Runtime rendering uses actual context values
|
|
Given a fully configured LLM agent from YAML
|
|
When the agent receives a message with runtime context
|
|
Then the system prompt templates should be rendered with actual values
|
|
And the LLM should receive the fully rendered prompt
|