cleveragents-core/CONTRIBUTING.md

# Contributing

[![Commitizen friendly](https://img.shields.io/badge/commitizen-friendly-brightgreen.svg)](http://commitizen.github.io/cz-cli/)
[![Semantic Versioning](https://img.shields.io/SemVer/2.0.0.png)](http://semver.org/spec/v2.0.0.html)
[![Matrix](https://img.shields.io/matrix/cleverthis%3Aqoto.org?server_fqdn=matrix.qoto.org&label=Matrix%20chat)](https://matrix.to/#/#CleverThis:qoto.org)

When contributing to this repository, it is usually a good idea to first discuss the change you
wish to make via issue, email, or any other method with the owners of this repository before
making a change. This could potentially save a lot of wasted hours.

Please note we have a code of conduct, please follow it in all your interactions with the project.

## File Organization

**CRITICAL: Never create files in the project root directory.** All new files must be organized in appropriate subdirectories:

- `/src/cleveragents` - Source code files. MUST NOT contain test files, documentation, mocking code, or examples.
- `/features/` - Behave-style unit test files (behavioral tests). Mocking code belongs under `/features/mocks/`.
- `/robot/` - Robot Framework integration tests (separate from unit tests).
- `/docs/` - Documentation and markdown files (must be written for mkdocs).
- `/config/` - Configuration files.
- `/scripts/` - Utility scripts.
- `/examples/` - Example code.

**Behave Guidelines:**
- **Group new steps with related ones.** Before adding a file under `features/steps/`, check for an existing file that covers the same behavior and extend it instead of creating a duplicate.
- **Name feature-specific step files after their feature.** Steps used only by `foo.feature` must live in `foo_steps.py` (and so on for every feature).
- **Keep shared steps in purpose-driven modules.** Steps meant for multiple features belong in clearly named, reusable files; prefer updating an existing shared file when it already fits the purpose before creating another one.
- **Ship features with complete implementations.** Every feature file must arrive with all of its steps fully implemented—never add placeholder steps or commit a feature without its supporting step code already in place.

## Development

### Testing

Run tests using `nox`:

- **All tests (including static checks):** `nox`
- **Unit tests only:** `nox -e unit_tests`
- **Integration tests:** `nox -e integration_tests`
- **Benchmarks:** `nox -e benchmark`
- **Coverage report:** `nox -e coverage_report`

### Commit Message Format

All commits on this repository must follow the
[Conventional Changelog standard](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md).
It is a very simple format so you can still write commit messages by hand. However it is
highly recommended developers install [Commitizen](https://commitizen.github.io/cz-cli/),
it extends the git command and will make writing commit messages a breeze. All CleverThis
repositories are configured with local Commitizen configuration scripts.

Getting Commitizen installed is usually trivial, just install it via npm. You will also
need to install the cz-customizable adapter which CleverThis repositories are configured
to use.

```bash
npm install -g commitizen@2.8.6 cz-customizable@4.0.0
```

Below is an example of Commitizen in action. It replaces your usual `git commit` command
with `git cz` instead. The new command takes all the same arguments however it leads you
through an interactive process to generate the commit message.

![Commitizen friendly](https://docs.cleverthis.com/public_media/commitizen.gif)

Commit messages are used to automatically generate our changelogs, and to ensure
commits are searchable in a useful way. So please use the Commitizen tool and adhere to
the commit message standard or else we cannot accept Pull Requests without editing
them first.

Below is an example of a properly formated commit message.

```
chore(Commitizen): Made repository Commitizen friendly.

Added standard Commitizen configuration files to the repo along with all the custom rules.

ISSUES CLOSED: #31
```

### Pull Request Process

1. Ensure that install or build dependencies do not appear in any commits in your code branch. 
2. Ensure all commit messages follow the [Conventional Changelog](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md)
   standard explained earlier.
3. Update the CONTRIBUTORS.md file to add your name to it if it isn't already there (one entry
   per person).
4. Adjust the project version to the new version that this Pull Request would represent. The
   versioning scheme we use is [Semantic Versioning](http://semver.org/).
5. Your pull request will either be approved or feedback will be given on what needs to be
   fixed to get approval. We usually review and comment on Pull Requests within 48 hours.

## Code Style & Best Practices

### General Principles

- **Modular Design:** Keep files under 500 lines.
- **Environment Safety:** Never hardcode secrets or sensitive information.
- **Test-First Development:** Write tests before implementation.
- **Clean Architecture:** Separate concerns, maintain clear boundaries between layers.
- **Documentation:** Keep all documentation updated alongside code changes.

### Python Import Guidelines

- Ensure all imports (including `from` statements) are at the top of the python file.
- Never encapsulate imports inside an indented code block (like an `if` or `try` statement).
- The only exception is for imports used exclusively for type checking purposes and only when strictly needed (e.g., `if TYPE_CHECKING:`).

### Programming Patterns

Use these patterns and principles liberally throughout the codebase:

#### SOLID Principles

- **Single Responsibility Principle (SRP):** Each class should have one reason to change; one responsibility only.
- **Open/Closed Principle (OCP):** Classes should be open for extension but closed for modification.
- **Liskov Substitution Principle (LSP):** Subtypes must be substitutable for their base types without altering program correctness.
- **Interface Segregation Principle (ISP):** Clients should not be forced to depend on interfaces they don't use; prefer small, specific interfaces.
- **Dependency Inversion Principle (DIP):** Depend on abstractions, not concretions; high-level modules should not depend on low-level modules.

#### Creational Patterns

- **Factory Pattern:** For object creation with complex initialization logic.
- **Abstract Factory Pattern:** For creating families of related objects without specifying concrete classes.
- **Builder Pattern:** For constructing complex objects step by step with fluent interfaces.
- **Prototype Pattern:** For cloning objects instead of creating new instances.
- **Singleton Pattern:** For single-instance resources (use sparingly, prefer dependency injection).
- **Object Pool Pattern:** For reusing expensive-to-create objects.
- **Dependency Injection:** For testability and inversion of control.

#### Structural Patterns

- **Adapter Pattern:** For interface compatibility between incompatible classes.
- **Bridge Pattern:** For separating abstraction from implementation to vary them independently.
- **Composite Pattern:** For treating individual objects and compositions uniformly in tree structures.
- **Decorator Pattern:** For extending functionality without modifying original code.
- **Facade Pattern:** For providing simplified interfaces to complex subsystems.
- **Flyweight Pattern:** For sharing common state among many objects to reduce memory usage.
- **Proxy Pattern:** For controlling access to objects (lazy loading, access control, logging).
- **Module Pattern:** For encapsulating related functionality in cohesive units.

#### Behavioral Patterns

- **Chain of Responsibility Pattern:** For passing requests along a chain of handlers.
- **Command Pattern:** For encapsulating operations as objects.
- **Interpreter Pattern:** For defining grammar and interpreting language sentences.
- **Iterator Pattern:** For sequential access to collection elements without exposing representation.
- **Mediator Pattern:** For centralizing complex communications between objects.
- **Memento Pattern:** For capturing and restoring object state without violating encapsulation.
- **Observer Pattern:** For event-driven architectures and loose coupling between publishers and subscribers.
- **State Pattern:** For altering object behavior when internal state changes.
- **Strategy Pattern:** For algorithms that can be selected at runtime.
- **Template Method Pattern:** For defining algorithm skeletons with customizable steps.
- **Visitor Pattern:** For separating algorithms from the objects they operate on.
- **Null Object Pattern:** For providing default behavior instead of null references.

#### Architectural Patterns

- **Repository Pattern:** For data access abstraction and separation of domain from data access logic.
- **Unit of Work Pattern:** For maintaining a list of objects affected by transactions and coordinating changes.
- **Service Layer Pattern:** For defining application boundaries and encapsulating business logic.
- **MVC (Model-View-Controller):** For separating data, presentation, and control logic.
- **CQRS (Command Query Responsibility Segregation):** For separating read and write operations.
- **Event Sourcing Pattern:** For storing state changes as a sequence of events.
- **Specification Pattern:** For encapsulating business rules and making them reusable and composable.

## LangChain/LangGraph Best Practices

### Graph Design Patterns

When creating LangGraph workflows, follow these patterns:

- **State Management:** Use TypedDict classes for state definition with clear field documentation
- **Node Naming:** Use descriptive verb-based names (e.g., `analyze_requirements`, `generate_plan`)
- **Error Handling:** Include error fields in state for graceful failure tracking
- **Checkpointing:** Always integrate MemorySaver for workflow resumption capabilities
- **Conditional Edges:** Use clear decision functions with descriptive names

### LangChain Integration Guidelines

- **Provider Abstraction:** Always use LangChain's unified interfaces (BaseLanguageModel, BaseLLM)
- **Prompt Templates:** Use ChatPromptTemplate or PromptTemplate for all prompts
- **Streaming:** Implement both sync (invoke) and async (ainvoke) methods with streaming support
- **Memory Management:** Use appropriate memory classes (ConversationBufferMemory, EntityMemory)
- **Output Parsing:** Use structured output parsers (StrOutputParser, JsonOutputParser)

### Testing LangChain/LangGraph Code

- **Mock Providers:** Use FakeListLLM or custom mock providers for deterministic testing
- **State Testing:** Test each node's state transformation independently
- **Graph Testing:** Verify complete workflow execution with expected state transitions
- **Streaming Testing:** Test both event emission and final results
- **Memory Testing:** Verify conversation history and entity tracking

### Configuration Best Practices

- **Environment Variables:** Use standard LangChain env vars (LANGCHAIN_TRACING_V2, LANGCHAIN_API_KEY)
- **Observability:** Keep LangSmith/OpenTelemetry disabled by default (user-configurable)
- **Provider Configuration:** Support multiple providers through configuration, not hardcoding
- **Retry Logic:** Use LangChain's built-in retry decorators for LLM calls
- **Token Limits:** Respect model token limits with appropriate chunking strategies

### Common Patterns

```python
# Example: Proper LangGraph node implementation
async def analyze_requirements(state: PlanGenerationState) -> dict:
    """Analyze requirements node with proper error handling."""
    try:
        prompt = ChatPromptTemplate.from_template(
            "Analyze these requirements: {requirements}"
        )
        chain = prompt | llm | StrOutputParser()
        analysis = await chain.ainvoke({"requirements": state["description"]})
        return {"analysis": analysis}
    except Exception as e:
        return {"error": f"Analysis failed: {str(e)}"}
```

## Error and Exception Handling

### Argument Validation

**All public and protected class methods must validate arguments as the first guard.** Perform these checks before any other logic:

**Checks to perform:**
- **Value Range:** Ensure numeric values are within acceptable bounds.
- **Null/None Checks:** Reject `None` where not expected; explicitly accept it only when documented.
- **Type Checking:** Verify correct types, especially in dynamically-typed contexts.
- **Empty Strings:** Reject empty strings where non-empty strings are required.
- **Empty Collections:** Check for empty lists, dicts, sets if they must contain elements.
- **Invalid States:** Verify object state is valid for the operation.

**Example:**
```python
def process_data(self, data: list[str], threshold: int) -> None:
    """Process data with validation."""
    # Argument validation first
    if data is None:
        raise ValueError("data cannot be None")
    if not data:  # empty list
        raise ValueError("data cannot be empty")
    if not all(isinstance(item, str) for item in data):
        raise TypeError("data must contain only strings")
    if threshold < 0 or threshold > 100:
        raise ValueError(f"threshold must be between 0 and 100, got {threshold}")
    
    # Now perform actual logic
    ...
```

### Exception Propagation

**CRITICAL: Do not suppress errors. Let exceptions propagate to top-level execution.**

- Exceptions should bubble up to the top level where they can be properly logged and handled.
- Do not catch exceptions just to log and re-raise; let them propagate naturally.
- Do not use bare `except:` or `except Exception:` without re-raising unless you have specific recovery logic.

**Only catch exceptions when you can meaningfully handle them** (e.g., retry logic, resource cleanup, adding context). Otherwise, let them propagate.

### Fail-Fast Principles

**Design code to fail immediately when something is wrong:**

- **Static Typing:** Use type hints extensively; run static type checkers (mypy).
- **Early Validation:** Check preconditions at function entry, not deep in logic.
- **Assertions:** Use assertions for invariants that should never be violated during development.
- **No Silent Failures:** Avoid returning `None` or default values when an error condition exists—raise exceptions.
- **Explicit Over Implicit:** Make failure conditions explicit rather than hiding them.

**Benefits:**
- Bugs are caught closer to their source.
- Stack traces are more meaningful.
- Debugging is faster.
- System state remains consistent.

### Best Practices Summary

- Validate all arguments in public/protected methods immediately.
- Use type hints and static type checking.
- Let exceptions propagate; don't suppress them.
- Fail fast when preconditions aren't met.
- Only catch exceptions when you have meaningful recovery logic.
- Never catch exceptions just to log them—let them bubble up for centralized handling.