docs: Add readme file and yaml file syntax documentation

This commit is contained in:
2025-10-02 17:51:49 +05:30
parent 4947b17521
commit 9e4896e92c
2 changed files with 902 additions and 0 deletions
+306
View File
@@ -0,0 +1,306 @@
# CleverAgents
A powerful, reactive Agent Framework using RxPy streams for complex AI agent orchestration and message routing.
[![PyPI Package](https://img.shields.io/pypi/v/cleveragents.svg)](https://pypi.org/project/cleveragents/)
[![Travis-CI Build Status](https://img.shields.io/travis/cleverthis/cleveragents.svg)](https://travis-ci.org/cleverthis/cleveragents)
## 🌊 What is CleverAgents?
CleverAgents is a **reactive agent orchestration framework** that combines the power of RxPy reactive streams with LangGraph stateful workflows to create sophisticated AI agent networks. It's designed for building complex, multi-agent systems that can handle real-time data processing, conversational AI, and workflow automation.
### Key Features
- **🌊 Reactive Architecture**: Built on RxPy streams for real-time data processing
- **🤖 Multiple Agent Types**: LLM agents, tool agents, composite agents, and more
- **🔄 Unified Routes**: Single configuration system for streams and graphs
- **🧠 Memory Management**: Conversation history and state persistence
- **🔗 LangGraph Integration**: Stateful workflows with conditional logic
- **⚡ Async Processing**: Full async/await support throughout
- **🎯 Template System**: Reusable agent and workflow templates
## 📦 Installation
### Prerequisites
- Python 3.9+ (recommended: Python 3.11+)
- pip package manager
### Install from PyPI
```bash
pip install cleveragents
```
### Install from Source
```bash
git clone https://git.cleverthis.com/cleveragents/cleveragents-core
cd cleveragents
pip install -e .
```
### Dependencies
CleverAgents automatically installs these dependencies:
- `click` - Command-line interface
- `rx>=3.2.0` - Reactive extensions for Python
- `jinja2` - Template engine
- `pystache` - Mustache template support
- `pyyaml` - YAML configuration parsing
- `langchain-core>=0.3.0` - LangChain integration
- `langchain-openai>=0.2.0` - OpenAI integration
- `langchain-anthropic>=0.2.0` - Anthropic integration
- `langchain-google-genai>=2.0.0` - Google Gemini integration
- `aiohttp` - Async HTTP client
## 🚀 Quick Start
### 1. Create a Configuration File
Create a `config.yaml` file:
```yaml
agents:
chat_agent:
type: llm
config:
provider: openai
model: gpt-4
temperature: 0.7
memory_enabled: true
routes:
chat_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: chat_agent
publications:
- __output__
merges:
- sources: [__input__]
target: chat_stream
```
### 2. Set Up API Keys
Configure your API keys via environment variables:
```bash
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export GOOGLE_API_KEY="your-google-key"
```
Or include them directly in your configuration:
```yaml
agents:
chat_agent:
type: llm
config:
provider: openai
model: gpt-4
api_key: "your-openai-key"
```
### 3. Run CleverAgents
**Interactive Mode:**
```bash
cleveragents interactive -c config.yaml
```
**Single-Shot Mode:**
```bash
cleveragents run -c config.yaml -p "Hello, how are you?"
```
## 🛠️ Command Line Interface
### Main Commands
#### `cleveragents run`
Process a single prompt through the agent network.
```bash
cleveragents run -c config.yaml -p "Your prompt here"
```
**Options:**
- `-c, --config`: Path to configuration file(s) (can be used multiple times)
- `-p, --prompt`: The prompt to send to the agent network
- `-o, --output`: Optional file to write output to
- `-v, --verbose`: Enable verbose output
- `-u, --unsafe`: Enable unsafe mode for code execution
#### `cleveragents interactive`
Start an interactive chat session.
```bash
cleveragents interactive -c config.yaml
```
**Options:**
- `-c, --config`: Path to configuration file(s)
- `-h, --history`: Optional file to load/save conversation history
- `-v, --verbose`: Enable verbose output
- `-u, --unsafe`: Enable unsafe mode
#### `cleveragents generate-examples`
Generate example configuration files.
```bash
cleveragents generate-examples -o ./my-examples
```
**Options:**
- `-o, --output`: Directory to write example files to (default: `./examples`)
#### `cleveragents visualize`
Visualize the agent network.
```bash
cleveragents visualize -c config.yaml -f mermaid
```
**Options:**
- `-c, --config`: Path to configuration file(s)
- `-o, --output`: Output file for the diagram
- `-f, --format`: Output format (`mermaid`, `dot`, `ascii`)
### Interactive Session Commands
When in interactive mode, you can use these commands:
- `exit` - Exit the session
- `help` - Show available commands
- `/stream <name> <message>` - Send message to specific stream
- `/graph <name> <message>` - Execute a LangGraph with message
## 🔧 API Key Configuration
CleverAgents supports multiple LLM providers through LangChain. Configure API keys in two ways:
### Environment Variables (Recommended)
```bash
# OpenAI
export OPENAI_API_KEY="sk-your-key-here"
# Anthropic
export ANTHROPIC_API_KEY="your-anthropic-key"
# Google Gemini
export GOOGLE_API_KEY="your-google-key"
```
### In Configuration Files
```yaml
agents:
my_agent:
type: llm
config:
provider: openai
model: gpt-4
api_key: "sk-your-key-here"
```
### Supported Models
**OpenAI:**
- GPT-4, GPT-4o, GPT-3.5-turbo
- o1-preview, o1-mini
**Anthropic:**
- Claude-3.5-Sonnet, Claude-3-Opus, Claude-3-Haiku
**Google:**
- Gemini-1.5-Pro, Gemini-1.5-Flash, Gemini-2.0-Flash
## 🏗️ Architecture
### Reactive Streams (RxPy)
CleverAgents uses RxPy for reactive programming:
- **Hot Streams**: Always active, replay last value
- **Cold Streams**: Start when subscribed
- **Replay Streams**: Replay all previous values
- **Operators**: map, filter, merge, split, buffer, throttle, debounce
### Agent Types
1. **LLM Agents**: Powered by large language models
2. **Tool Agents**: Execute functions and tools
3. **Composite Agents**: Combine multiple agents
4. **Chain Agents**: Sequential processing chains
### Unified Routes System
Single configuration system for different processing types:
- **Stream Routes**: Reactive, stateless processing
- **Graph Routes**: Stateful workflows with conditional logic
- **Bridge Routes**: Dynamic type conversion
## 📚 Examples
### Basic Chat
```bash
cleveragents interactive -c examples/basic_chat.yaml
```
### Scientific Paper Writer
```bash
cleveragents interactive -c examples/scientific_paper_writer.yaml
```
### Multi-Agent Collaboration
```bash
cleveragents interactive -c examples/collaboration_reactive.yaml
```
## 🧪 Development
### Running Tests
```bash
# Run all tests
python -m behave tests/features
# Run with coverage
python -m pytest --cov=cleveragents
# Run with tox for multiple environments
tox
```
### Contributing
1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request
## 📄 License
Apache License 2.0
## 🔗 Links
- **Documentation**: https://cleveragents.readthedocs.io/
- **PyPI**: https://pypi.org/project/cleveragents/
- **GitHub**: https://github.com/cleverthis/cleveragents
- **Issues**: https://github.com/cleverthis/cleveragents/issues
## 🤝 Support
For questions, issues, or contributions, please visit our [GitHub repository](https://github.com/cleverthis/cleveragents) or contact us at [dev@cleverthis.com](mailto:dev@cleverthis.com).
+596
View File
@@ -0,0 +1,596 @@
# CleverAgents YAML Configuration Syntax
This document provides comprehensive documentation for CleverAgents YAML configuration files, including all sections, syntax, and expected behaviors.
## 📋 Table of Contents
- [Configuration Structure](#configuration-structure)
- [Agents Section](#agents-section)
- [Routes Section](#routes-section)
- [Merges Section](#merges-section)
- [Splits Section](#splits-section)
- [Templates Section](#templates-section)
- [Context Section](#context-section)
- [Complete Examples](#complete-examples)
- [Best Practices](#best-practices)
## 🏗️ Configuration Structure
A CleverAgents configuration file follows this structure:
```yaml
# Optional: Global configuration
cleveragents:
template_engine: JINJA2 # or MUSTACHE
# Required: Agent definitions
agents:
agent_name:
type: agent_type
config:
# Agent-specific configuration
# Required: Route definitions (unified streams and graphs)
routes:
route_name:
type: stream|graph|bridge
# Route-specific configuration
# Optional: Stream operations
merges:
- sources: [source_streams]
target: target_stream
splits:
- source: source_stream
targets:
target1: condition1
target2: condition2
# Optional: Template definitions
templates:
agents:
template_name:
# Template definition
routes:
template_name:
# Template definition
# Optional: Global context
context:
global:
variable_name: value
```
## 🤖 Agents Section
The `agents` section defines AI agents that can process messages.
### Agent Types
#### LLM Agents
```yaml
agents:
chat_agent:
type: llm
config:
provider: openai|anthropic|google
model: gpt-4|claude-3-5-sonnet|gemini-1.5-pro
temperature: 0.7
max_tokens: 1000
system_prompt: "You are a helpful assistant"
memory_enabled: true
max_history: 10
api_key: "optional-api-key"
```
**LLM Agent Configuration:**
- `provider`: LLM provider (openai, anthropic, google)
- `model`: Specific model name
- `temperature`: Creativity level (0.0-1.0)
- `max_tokens`: Maximum response length
- `system_prompt`: System message for the agent
- `memory_enabled`: Enable conversation memory
- `max_history`: Number of messages to remember
- `api_key`: Optional API key (can use environment variables)
#### Tool Agents
```yaml
agents:
tool_agent:
type: tool
config:
tools: ["math", "http", "file", "json", "shell"]
safe_mode: true
timeout: 30
```
**Tool Agent Configuration:**
- `tools`: List of available tools
- `safe_mode`: Enable safety restrictions
- `timeout`: Execution timeout in seconds
#### Composite Agents
```yaml
agents:
composite_agent:
type: composite
config:
components:
agents:
- name: sub_agent1
type: llm
graphs:
- name: sub_graph1
type: graph
streams:
- name: sub_stream1
type: stream
routing:
input: sub_agent1
output: sub_stream1
```
## 🛤️ Routes Section
The `routes` section defines data flow through your agent network using a unified system.
### Stream Routes
```yaml
routes:
my_stream:
type: stream # REQUIRED
stream_type: cold|hot|replay
operators:
- type: map
params:
agent: agent_name
- type: filter
params:
condition:
type: content_contains
text: "keyword"
- type: debounce
params:
duration: 1.0
- type: buffer
params:
count: 5
- type: throttle
params:
duration: 2.0
subscriptions:
- source_stream
publications:
- target_stream
- __output__
agents:
- agent_name
initial_value: "default"
buffer_size: 10
```
**Stream Types:**
- `cold`: Start processing when subscribed
- `hot`: Always active, replay last value
- `replay`: Replay all previous values
**Operators:**
- `map`: Transform messages using agents or functions
- `filter`: Filter messages based on conditions
- `debounce`: Wait for quiet period before processing
- `buffer`: Collect messages before processing
- `throttle`: Limit processing rate
- `catch`: Handle errors
- `retry`: Retry failed operations
### Graph Routes
```yaml
routes:
my_graph:
type: graph # REQUIRED
entry_point: start
nodes:
process_node:
type: agent
agent: agent_name
conditional_node:
type: conditional
condition:
type: content_contains
text: "keyword"
function_node:
type: function
function: my_function
edges:
- source: start
target: process_node
- source: process_node
target: conditional_node
condition:
type: metadata_has
key: success
- source: conditional_node
target: end
checkpointing: true
checkpoint_dir: "./checkpoints"
enable_time_travel: true
parallel_execution: true
state_class: "my_module.MyState"
```
**Node Types:**
- `agent`: Process using an agent
- `conditional`: Conditional routing
- `function`: Custom function execution
**Graph Features:**
- `checkpointing`: Save state for recovery
- `enable_time_travel`: Allow state rollback
- `parallel_execution`: Run nodes in parallel
- `state_class`: Custom state management
### Bridge Routes
```yaml
routes:
adaptive_route:
type: stream
# ... stream configuration ...
bridge:
upgrade_conditions:
needs_state: true
message_count: 5
custom_predicate: "lambda msg, cfg: 'NEEDS_STATE' in msg.content"
downgrade_conditions:
idle_time: 300
state_size: 2
no_conditionals_used: true
state_extractor: "lambda msg: {'last_message': msg.content}"
state_flattener: "lambda state: state.data.get('summary', '')"
preserve_subscriptions: true
preserve_checkpointing: true
```
## 🔄 Merges Section
Merge multiple streams into one:
```yaml
merges:
- sources: [stream1, stream2, stream3]
target: combined_stream
- sources: [__input__]
target: main_processor
```
## 🔀 Splits Section
Split one stream into multiple streams:
```yaml
splits:
- source: input_stream
targets:
high_priority:
type: content_contains
text: "urgent"
normal_priority:
type: content_not_contains
text: "urgent"
error_stream:
type: metadata_has
key: error
```
**Split Conditions:**
- `content_contains`: Message contains text
- `content_not_contains`: Message doesn't contain text
- `metadata_has`: Message has metadata key
- `metadata_equals`: Metadata equals value
- `custom`: Custom condition function
## 🎨 Templates Section
Define reusable templates:
```yaml
templates:
agents:
basic_llm:
type: llm
config:
provider: openai
model: gpt-4
temperature: 0.7
system_prompt: "{{system_message}}"
routes:
chat_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: "{{agent_name}}"
publications:
- __output__
graphs:
simple_workflow:
type: graph
nodes:
process:
type: agent
agent: "{{agent_name}}"
edges:
- source: start
target: process
- source: process
target: end
# Use templates
agents:
my_agent:
template: basic_llm
params:
system_message: "You are a helpful assistant"
my_chat:
template: chat_stream
params:
agent_name: my_agent
```
## 🌍 Context Section
Global variables and configuration:
```yaml
context:
global:
app_name: "My CleverAgents App"
debug_mode: true
max_retries: 3
timeout: 30
api_keys:
openai: "sk-your-key"
anthropic: "your-key"
user_preferences:
language: "en"
theme: "dark"
```
## 📝 Complete Examples
### Basic Chat with Memory
```yaml
agents:
chat_agent:
type: llm
config:
provider: openai
model: gpt-4
temperature: 0.8
memory_enabled: true
max_history: 10
system_prompt: |
You are a helpful and friendly AI assistant with memory.
You can remember our conversation and refer back to previous topics.
routes:
chat_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: chat_agent
publications:
- __output__
merges:
- sources: [__input__]
target: chat_stream
context:
global:
conversation_mode: true
memory_enabled: true
```
### Multi-Agent Research Pipeline
```yaml
agents:
researcher:
type: llm
config:
provider: openai
model: gpt-4
system_prompt: "Research and gather information on topics"
analyzer:
type: llm
config:
provider: openai
model: gpt-4
system_prompt: "Analyze and synthesize research findings"
writer:
type: llm
config:
provider: openai
model: gpt-4
system_prompt: "Write comprehensive reports based on analysis"
routes:
research_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: researcher
- type: buffer
params:
time: 2.0
publications:
- analysis_stream
analysis_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: analyzer
publications:
- writing_stream
writing_stream:
type: stream
stream_type: cold
operators:
- type: map
params:
agent: writer
publications:
- __output__
merges:
- sources: [__input__]
target: research_stream
```
### Stateful Workflow with LangGraph
```yaml
agents:
classifier:
type: llm
config:
provider: openai
model: gpt-3.5-turbo
system_prompt: "Classify the input as: question, command, or statement"
question_handler:
type: llm
config:
provider: openai
model: gpt-4
system_prompt: "Answer questions concisely and accurately"
command_handler:
type: tool
config:
tools: ["execute", "search"]
routes:
workflow_graph:
type: graph
entry_point: start
nodes:
classify:
type: agent
agent: classifier
handle_question:
type: agent
agent: question_handler
handle_command:
type: agent
agent: command_handler
edges:
- source: start
target: classify
- source: classify
target: handle_question
condition:
type: content_contains
text: "question"
- source: classify
target: handle_command
condition:
type: content_contains
text: "command"
- source: handle_question
target: end
- source: handle_command
target: end
checkpointing: true
input_stream:
type: stream
stream_type: cold
operators:
- type: graph_execute
params:
graph: workflow_graph
publications:
- __output__
merges:
- sources: [__input__]
target: input_stream
```
## ✅ Best Practices
### 1. Configuration Organization
- Use descriptive names for agents and routes
- Group related configurations together
- Use templates for reusable patterns
- Keep configurations modular
### 2. Memory Management
- Enable memory for conversational agents
- Set appropriate `max_history` limits
- Use context variables for global state
### 3. Error Handling
- Use `catch` operators for error handling
- Implement retry logic for unreliable operations
- Use splits to route errors to error handlers
### 4. Performance
- Use `throttle` and `debounce` for rate limiting
- Implement `buffer` for batch processing
- Use `hot` streams for real-time updates
### 5. Testing
- Start with simple configurations
- Test each component individually
- Use verbose mode for debugging
- Validate configurations before deployment
## 🚨 Common Issues
### Configuration Errors
- Missing `type` field in routes (REQUIRED)
- Invalid agent types
- Circular dependencies in merges/splits
- Missing required configuration fields
### Runtime Errors
- API key not configured
- Invalid model names
- Network connectivity issues
- Memory limits exceeded
### Debugging Tips
- Use `--verbose` flag for detailed logging
- Check agent capabilities with `/help` in interactive mode
- Validate YAML syntax before running
- Test with simple configurations first
## 📚 Additional Resources
- [CleverAgents Documentation](https://cleveragents.readthedocs.io/)
- [RxPy Documentation](https://rxpy.readthedocs.io/)
- [LangGraph Documentation](https://langchain-ai.github.io/langgraph/)
- [YAML Syntax Reference](https://yaml.org/spec/1.2/spec.html)
For more examples and advanced configurations, see the `examples/` directory in the CleverAgents repository.