forked from cleveragents/cleveragents-core
452 lines
14 KiB
Gherkin
452 lines
14 KiB
Gherkin
Feature: Core Application Coverage
|
|
As a developer
|
|
I want to test all uncovered functionality in ReactiveCleverAgentsApp
|
|
So that we achieve 90%+ code coverage for application.py
|
|
|
|
Background:
|
|
Given the application test environment is initialized
|
|
|
|
Scenario: Application initialization with prompt template processing
|
|
Given I have a configuration with prompt templates:
|
|
"""
|
|
cleveragents:
|
|
template_engine: JINJA2
|
|
|
|
agents:
|
|
template_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
prompts:
|
|
greeting:
|
|
content: "Hello {{name}}"
|
|
simple_prompt: "Just a string"
|
|
dict_prompt:
|
|
content: "Dict content"
|
|
metadata: "extra"
|
|
"""
|
|
When I load the configuration
|
|
Then the application should initialize successfully
|
|
And prompt templates should be registered correctly
|
|
And both string and dict prompts should be processed
|
|
|
|
Scenario: Enhanced template registry initialization
|
|
Given I have a configuration with advanced complex templates requiring preprocessing:
|
|
"""
|
|
agents:
|
|
test_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
templates:
|
|
agents:
|
|
complex_agent:
|
|
_needs_preprocessing: true
|
|
_raw_template: "{{agent_type}}"
|
|
type: "{{agent_type}}"
|
|
graphs:
|
|
simple_graph:
|
|
__jinja_template__: true
|
|
type: graph
|
|
streams:
|
|
template_stream:
|
|
__is_template__: true
|
|
type: stream
|
|
"""
|
|
When I load the configuration
|
|
Then the enhanced template registry should be used
|
|
And complex templates should be processed correctly
|
|
|
|
Scenario: Regular template registry for simple templates
|
|
Given I have a configuration with simple templates:
|
|
"""
|
|
agents:
|
|
simple_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
templates:
|
|
agents:
|
|
basic_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
"""
|
|
When I load the configuration
|
|
Then the regular template registry should be used
|
|
And simple templates should be registered
|
|
|
|
Scenario: Agent creation with template instances
|
|
Given I have a configuration with agent template instances:
|
|
"""
|
|
agents:
|
|
simple_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
template_instance_agent:
|
|
type: template_instance
|
|
config:
|
|
template: basic_agent
|
|
params:
|
|
model: gpt-4
|
|
|
|
templates:
|
|
agents:
|
|
basic_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
When I load the configuration
|
|
Then agents should be created from templates
|
|
And template instances should be processed correctly
|
|
|
|
Scenario: Agent creation with enhanced registry template instances
|
|
Given I have a configuration requiring enhanced registry with template instances:
|
|
"""
|
|
agents:
|
|
enhanced_template_agent:
|
|
type: template_instance
|
|
config:
|
|
agent_template: complex_agent
|
|
params:
|
|
agent_type: tool
|
|
|
|
templates:
|
|
agents:
|
|
complex_agent:
|
|
_needs_preprocessing: true
|
|
_raw_template: "tool"
|
|
type: tool
|
|
config:
|
|
tools: ["echo"]
|
|
"""
|
|
When I load the configuration
|
|
Then the enhanced template registry should instantiate agents
|
|
And complex template parameters should be applied
|
|
|
|
Scenario: Bridge route configuration
|
|
Given I have a configuration with bridge routes:
|
|
"""
|
|
agents:
|
|
bridge_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
routes:
|
|
bridge_route:
|
|
type: bridge
|
|
source: input_stream
|
|
target: output_stream
|
|
"""
|
|
When I load the configuration
|
|
Then bridge routes should be registered
|
|
And the route bridge should be initialized
|
|
|
|
Scenario: Graph route with state class resolution
|
|
Given I have a configuration with graph routes and state class:
|
|
"""
|
|
agents:
|
|
graph_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
routes:
|
|
state_graph:
|
|
type: graph
|
|
state_class: "builtins.dict"
|
|
nodes:
|
|
start:
|
|
agent: graph_agent
|
|
edges:
|
|
- source: start
|
|
target: END
|
|
entry_point: start
|
|
"""
|
|
When I load the configuration
|
|
Then the state class should be resolved correctly
|
|
And the graph should be created with the state class
|
|
|
|
Scenario: Graph route with invalid state class
|
|
Given I have a configuration with invalid state class:
|
|
"""
|
|
agents:
|
|
graph_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
routes:
|
|
invalid_state_graph:
|
|
type: graph
|
|
state_class: "nonexistent.module.Class"
|
|
nodes:
|
|
start:
|
|
agent: graph_agent
|
|
edges:
|
|
- source: start
|
|
target: END
|
|
"""
|
|
When I load the configuration
|
|
Then a warning should be logged about the invalid state class
|
|
And the graph should still be created
|
|
|
|
Scenario: Route template instantiation
|
|
Given I have a configuration with route templates:
|
|
"""
|
|
agents:
|
|
template_route_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
routes:
|
|
templated_route:
|
|
template_config:
|
|
template: stream_template
|
|
params:
|
|
agent_name: template_route_agent
|
|
|
|
templates:
|
|
streams:
|
|
stream_template:
|
|
type: stream
|
|
stream_type: cold
|
|
agents:
|
|
- "{{agent_name}}"
|
|
"""
|
|
When I load the configuration
|
|
Then route templates should be instantiated
|
|
And template parameters should be applied to routes
|
|
|
|
Scenario: Stream operations setup with merges and splits
|
|
Given I have a configuration with merge and split operations:
|
|
"""
|
|
agents:
|
|
merge_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
routes:
|
|
source1:
|
|
type: stream
|
|
stream_type: cold
|
|
source2:
|
|
type: stream
|
|
stream_type: cold
|
|
target:
|
|
type: stream
|
|
stream_type: cold
|
|
operators:
|
|
- type: map
|
|
params:
|
|
agent: merge_agent
|
|
split_stream:
|
|
type: stream
|
|
stream_type: cold
|
|
|
|
merges:
|
|
- sources: [source1, source2]
|
|
target: target
|
|
|
|
splits:
|
|
- source: split_stream
|
|
targets:
|
|
positive: "content.startswith('good')"
|
|
negative: "content.startswith('bad')"
|
|
"""
|
|
When I load the configuration
|
|
Then merges should be set up correctly
|
|
And splits should be configured properly
|
|
And subscriptions should be re-setup after operations
|
|
|
|
Scenario: Hybrid pipeline setup
|
|
Given I have a configuration with hybrid pipelines:
|
|
"""
|
|
agents:
|
|
pipeline_agent:
|
|
type: llm
|
|
config:
|
|
provider: openai
|
|
model: gpt-3.5-turbo
|
|
|
|
pipelines:
|
|
hybrid_pipeline:
|
|
name: test_pipeline
|
|
stages:
|
|
- type: stream
|
|
config:
|
|
operators:
|
|
- type: map
|
|
params:
|
|
agent: pipeline_agent
|
|
- type: graph
|
|
config:
|
|
nodes:
|
|
process:
|
|
agent: pipeline_agent
|
|
metadata:
|
|
description: "Test hybrid pipeline"
|
|
"""
|
|
When I load the configuration
|
|
Then hybrid pipelines should be created
|
|
And the pipeline should be registered with the bridge
|
|
|
|
@skip
|
|
Scenario: Single-shot processing with timeout
|
|
Given I have a loaded application for single-shot testing
|
|
When I run single-shot processing with a slow operation
|
|
Then the operation should timeout after 3 seconds
|
|
And a timeout error should be raised
|
|
|
|
Scenario: Single-shot processing with None message handling
|
|
Given I have a loaded application that returns None messages
|
|
When I run single-shot processing with prompt "test"
|
|
Then the result should handle None messages gracefully
|
|
And return an empty string result
|
|
|
|
Scenario: Single-shot processing error handling
|
|
Given I have a loaded application for error testing
|
|
When I run single-shot processing that causes an error
|
|
Then the error should be wrapped in CleverAgentsException
|
|
And the original error should be preserved
|
|
|
|
Scenario: Interactive session with help command
|
|
Given I have a loaded application for interactive testing
|
|
When I start an interactive session and request help
|
|
Then help information should be displayed
|
|
And available commands should be shown
|
|
|
|
Scenario: Interactive session with stream commands
|
|
Given I have a loaded application with named streams
|
|
When I use stream commands in interactive session
|
|
Then messages should be sent to specific streams
|
|
And stream command errors should be handled
|
|
|
|
Scenario: Interactive session with graph commands
|
|
Given I have a loaded application with graph routes
|
|
When I use graph commands in interactive session
|
|
Then graphs should be executed with messages
|
|
And graph results should be displayed
|
|
|
|
Scenario: Interactive session with unknown streams and graphs
|
|
Given I have a loaded application for command testing
|
|
When I use commands with unknown streams or graphs
|
|
Then appropriate error messages should be shown
|
|
And available options should be listed
|
|
|
|
Scenario: Interactive session error handling
|
|
Given I have a loaded application for interactive error testing
|
|
When errors occur during interactive session
|
|
Then errors should be caught and displayed
|
|
And the session should continue running
|
|
|
|
Scenario: Network visualization with different formats
|
|
Given I have a complex application configuration
|
|
When I request network visualization in mermaid format
|
|
Then a mermaid network diagram should be generated
|
|
And agents, streams, and graphs should be shown
|
|
|
|
Scenario: Network visualization with unsupported format
|
|
Given I have a loaded application for visualization
|
|
When I request network visualization in unsupported format
|
|
Then an unsupported format message should be returned
|
|
|
|
Scenario: Application disposal and cleanup
|
|
Given I have a running application with active streams
|
|
When I perform application cleanup
|
|
Then all streams should be disposed
|
|
And cleanup should be logged
|
|
And resources should be freed
|
|
|
|
Scenario: Configuration without agents or routes
|
|
Given I have an empty configuration
|
|
When I try to load the configuration
|
|
Then the application should handle empty configuration
|
|
And no errors should occur for missing sections
|
|
|
|
Scenario: Agent factory error handling
|
|
Given I have a configuration that causes agent factory errors
|
|
When I try to load the configuration
|
|
Then agent creation errors should be properly handled
|
|
And meaningful error messages should be provided
|
|
|
|
Scenario: Agent type registration with built-in types
|
|
Given I have a configuration with various agent types
|
|
When I load the configuration
|
|
Then built-in agent types should be registered
|
|
And LLM and tool agents should be available
|
|
|
|
Scenario: Template renderer with different template types
|
|
Given I have a configuration with mixed template content types
|
|
When the templates are processed
|
|
Then string templates should be handled correctly
|
|
And dict templates should extract content properly
|
|
And invalid templates should be skipped
|
|
|
|
Scenario: Configuration to dict conversion
|
|
Given I have a reactive configuration
|
|
When the configuration is converted to dictionary format
|
|
Then agents should be properly mapped
|
|
And global context should be included
|
|
And prompts should be preserved
|
|
|
|
Scenario: Error stream subscription in interactive mode
|
|
Given I have an application with error handling
|
|
When I start interactive session with error streams
|
|
Then error observers should be set up
|
|
And errors should be displayed to user
|
|
|
|
Scenario: Tool command processing with valid JSON
|
|
Given I have an application with tool agents configured
|
|
And I have content with tool execution commands:
|
|
"""
|
|
Here is the result: [TOOL_EXECUTE:echo]{"text": "hello world"}[/TOOL_EXECUTE]
|
|
"""
|
|
When I process tool commands in the content
|
|
Then tool commands should be executed successfully
|
|
And the result should contain "hello world"
|
|
|
|
# Note: Tool command processing tests removed due to environment compatibility issues
|
|
# These tests require async execution context that conflicts with the test runner
|
|
|
|
Scenario: Dispose application with agent cleanup
|
|
Given an application with agents having cleanup
|
|
When disposing the application
|
|
Then all agent cleanup methods are called
|
|
|
|
Scenario: Dispose application with cleanup failure
|
|
Given an application with failing agent cleanup
|
|
When disposing the application
|
|
Then cleanup errors are logged as warnings
|
|
And disposal completes successfully
|
|
|
|
@skip
|
|
Scenario: Deferred initialization in run_single_shot
|
|
Given an app with deferred initialization
|
|
When calling run_single_shot
|
|
Then routes are initialized before execution
|
|
|
|
@skip
|
|
Scenario: Deferred initialization in interactive session
|
|
Given an app with deferred initialization
|
|
When starting interactive session
|
|
Then routes are initialized before session |