Files
cleveragents-core/docs/adr/ADR-005-technical-stack.md
freemo d5b122d4a3
CI / lint (push) Successful in 14s
CI / quality (push) Successful in 21s
CI / security (push) Successful in 36s
CI / build (push) Successful in 36s
CI / typecheck (push) Successful in 39s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m57s
CI / integration_tests (push) Successful in 3m23s
CI / docker (push) Successful in 53s
CI / coverage (push) Successful in 5m58s
CI / benchmark-publish (push) Successful in 19m27s
Docs: Updated to A2A and integrating rest standard
2026-03-11 13:19:55 -04:00

201 lines
11 KiB
Markdown

---
adr_number: 5
title: Technical Stack
status_history:
- - '2026-02-16'
- Proposed
- Jeffrey Phillips Freeman
- - '2026-02-16'
- Accepted
- Jeffrey Phillips Freeman
tier: 1
authors:
- Jeffrey Phillips Freeman
superseded_by: null
related_adrs:
- number: 1
title: Layered Architecture
relationship: Stack choices are organized according to the four-layer structure
- number: 3
title: Dependency Injection
relationship: '`dependency-injector` is a core stack choice for service wiring'
- number: 4
title: Data Validation
relationship: Pydantic V2 is a core stack choice for validation and modeling
- number: 19
title: Storage and Persistence
relationship: SQLAlchemy and SQLite/PostgreSQL are the chosen persistence stack
- number: 21
title: CLI and Output Rendering
relationship: Typer and Rich are the chosen CLI stack
- number: 22
title: LangChain/LangGraph Integration
relationship: LangChain and LangGraph are the chosen LLM orchestration stack
- number: 25
title: Observability and Logging
relationship: structlog is the chosen logging stack
acceptance:
votes_for:
- voter: Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com>
comment: This stack balances maturity, ecosystem support, and alignment with our architectural goals
votes_against: []
abstentions: []
---
## Context
CleverAgents requires a cohesive technology stack that supports a CLI-first interface, LLM orchestration with multiple providers, structured data persistence, code intelligence indexing (full-text, vector, and graph), reactive event processing, sandboxed execution, and both local and server deployment modes. The stack must be Python-only for consistency and alignment with the LLM ecosystem. Every technology choice must have a clear rationale and defined minimum version.
## Decision Drivers
- Must be Python-only for consistency and alignment with the LLM ecosystem (LangChain, MCP SDK)
- Requires CLI-first interface, LLM orchestration with multiple providers, and structured data persistence
- Need configurable backends at every infrastructure layer (full-text, vector, graph indexing) swappable via configuration
- Must support both local single-process and multi-user server deployment modes
- Testing stack must cover behavioral (BDD), integration, unit, and property-based testing
- Every technology choice must have a clear rationale and defined minimum version for reproducibility
## Decision
CleverAgents is built on **Python >= 3.13** as the sole implementation language. The stack is organized by functional area, with specific libraries chosen for CLI, LLM runtime, data persistence, indexing, configuration, testing, code quality, infrastructure, and observability.
## Design
### Core Runtime
| Technology | Min Version | Role |
|---|---|---|
| Python | >= 3.13 | Primary language. Type hint maturity, async/await, pattern matching, performance. |
| Hatchling | >= 1.21.0 | PEP 517 build backend. Lightweight, standards-compliant. |
| uv | >= 0.8.0 | Package installer and resolver. 10-100x faster than pip. Used in CI, Docker, and Nox. |
### CLI and Presentation
| Technology | Min Version | Role |
|---|---|---|
| Typer | >= 0.9.0 | CLI framework. Automatic help, type inference, Rich integration, nested command groups. |
| Rich | (transitive) | Terminal rendering. Panel, Table, Tree, Syntax, Progress, Spinner, Markdown. |
| Textual | (future) | TUI framework. Reactive terminal UI, servable as web app via Textual Web. |
### LLM and AI Runtime
| Technology | Min Version | Role |
|---|---|---|
| LangChain | >= 0.2.14 | LLM abstraction. Provider-agnostic chat models, embeddings, output parsing, tool calling. |
| LangGraph | (transitive) | Stateful workflow orchestration. `StateGraph` with conditional edges, checkpointing, streaming. |
| LangChain Provider Packages | varies | `langchain-openai` (>= 0.2.0), `langchain-google-genai` (>= 0.2.0), `langchain-anthropic`, `langchain-groq`, `langchain-together`, `langchain-cohere`. |
| LangChain Community | >= 0.2.14 | FAISS vector store, FakeListLLM/FakeEmbeddings for testing. |
| RxPY | >= 3.2.0 | Reactive streams. Subject, operators (map, filter, flat_map, debounce, throttle, scan) for event routing and backpressure. |
| MCP SDK | >= 1.4.0 | Model Context Protocol client. Tool discovery and invocation against MCP servers. |
### Data and Persistence
| Technology | Min Version | Role |
|---|---|---|
| SQLite | (system) | Primary database. Zero-config, file-based, ACID, WAL mode for concurrent reads. |
| SQLAlchemy | (transitive) | ORM. Declarative mapping, session management, Unit of Work, dialect abstraction. |
| Alembic | >= 1.13.1 | Database migrations. Version-controlled schema upgrade/downgrade. Auto-applied on init. |
| python-ulid | >= 2.7.0 | ULID generation. Lexicographically sortable by creation time for all entity IDs. |
### Indexing and Code Intelligence
| Technology | Min Version | Role |
|---|---|---|
| Tantivy (tantivy-py) | configurable | Full-text search. Rust-based, sub-millisecond. Alternative: SQLite FTS5. |
| FAISS (faiss-cpu) | >= 1.7.4 | Vector similarity search. Approximate nearest neighbor for semantic code search. |
| Qdrant | configurable | Vector search alternative for production horizontal scaling. |
| Neo4j | configurable | Knowledge graph for structural code relationships. Alternative: rdflib. |
| rdflib | >= 7.1.4 | In-process RDF graph store for lightweight structural analysis. |
| OpenAI Embeddings | configurable | Default embedding provider (`text-embedding-3-small`). Alternatives: Anthropic, local models. |
### Configuration and Validation
| Technology | Min Version | Role |
|---|---|---|
| Pydantic | >= 2.7.0 | Data validation and modeling for all domain models and config objects. |
| Pydantic Settings | >= 2.11.0 | Environment configuration with `CLEVERAGENTS_` prefix and `.env` support. |
| Jinja2 | >= 3.1.0 | Template rendering for actor prompts, YAML interpolation, prompt templates. |
| TOML (tomllib) | stdlib | Global configuration format. Native nested table support. |
| YAML (PyYAML) | (transitive) | Entity configuration format. All entity definitions use YAML. |
### Testing
| Technology | Min Version | Role |
|---|---|---|
| Behave | >= 1.2.6 | BDD feature tests. Gherkin scenarios, custom parallel runner. |
| Robot Framework | >= 7.3.2 | Integration tests. Keyword-driven CLI workflow tests, parallel via pabot. |
| pytest | >= 8.0.0 | Unit tests. Fixtures, parametrization, plugin support. |
| pytest-asyncio | >= 0.23.0 | Async test support for LangGraph workflows and RxPY streams. |
| pytest-cov / coverage | >= 7.11.0 | Coverage reporting. Target: 85% line coverage. |
| Hypothesis | >= 6.136.6 | Property-based testing for invariant verification and merge strategies. |
| ASV | >= 0.6.5 | Performance benchmarks. Plan generation latency, CLI startup, indexing throughput. |
### Code Quality
| Technology | Min Version | Role |
|---|---|---|
| Ruff | >= 0.1.0 | Linting and formatting. Rules: E, F, W, B, UP, I, SIM, RUF. Line length 88, double quotes. |
| Pyright | >= 1.1.350 | Static type checking in strict mode for the entire codebase. |
### Infrastructure and Deployment
| Technology | Min Version | Role |
|---|---|---|
| Docker | multi-stage | `python:3.13-slim`, non-root user (`appuser`, uid 1000). Entrypoint: `python -m cleveragents`. |
| Helm | (chart) | Kubernetes deployment for server mode. Chart in `k8s/`. |
| Nox | >= 2025.4.22 | Task automation. Sessions for lint, test, build, docs, benchmarks, coverage. Uses uv backend. |
| Forgejo CI | (workflow) | CI pipeline: lint → typecheck → behave (matrix 3.11/3.12/3.13) → build → docker → helm. |
### Monitoring and Observability
| Technology | Min Version | Role |
|---|---|---|
| structlog | >= 24.4.0 | Structured JSON logging with context binding (plan_id, decision_id, actor_name, tool_name). |
| LangSmith | (optional) | LLM observability. Tracing, token usage, latency, cost. Via `CLEVERAGENTS_LANGSMITH_*` env vars. |
### Additional Libraries
| Technology | Min Version | Role |
|---|---|---|
| dependency-injector | >= 4.41.0 | DI container. DeclarativeContainer with Singleton, Factory, Configuration providers. |
| watchdog | >= 4.0.0 | File system monitoring. Auto re-indexing on file changes. |
| numpy | >= 2.1.0 | Vector operations for embedding similarity and confidence score aggregation. |
| uvicorn | >= 0.30.1 | ASGI server for hosting the A2A JSON-RPC 2.0 endpoint in server mode. |
## Constraints
- The entire codebase is Python-only. No secondary languages for core functionality.
- Version constraints are minimum versions; newer compatible versions are acceptable.
- All dependencies must be installable via `uv` and declared in `pyproject.toml`.
- Configurable backends (Tantivy vs. SQLite FTS5, FAISS vs. Qdrant, Neo4j vs. rdflib) must be swappable via configuration without code changes, enabled by the hexagonal architecture.
- The Docker production image must use `python:3.13-slim` with a non-root user.
## Consequences
### Positive
- A single language across the entire stack simplifies hiring, tooling, and maintenance.
- The LLM ecosystem (LangChain, MCP SDK) is Python-native, avoiding FFI overhead.
- Configurable backends at every infrastructure layer allow adaptation to different deployment scales.
- The testing stack (Behave + Robot + pytest + Hypothesis) covers behavioral, integration, unit, and property-based testing.
### Negative
- Python's runtime performance is lower than compiled languages; CPU-intensive operations (embedding computation, large index operations) rely on native extensions (numpy, FAISS, Tantivy).
- The large dependency tree increases the attack surface and requires active dependency management.
- Supporting multiple configurable backends per infrastructure layer increases the testing matrix.
### Risks
- Major version upgrades to LangChain or Pydantic could require significant migration effort.
- Tantivy and FAISS are native extensions that may have platform-specific build issues.
- The breadth of the stack means that not all developers will be expert in every component.
## Alternatives Considered
None — specification-driven requirement. The technical stack is enumerated in the specification with specific version constraints and rationales for each choice.
## Compliance
- **Dependency pinning**: `pyproject.toml` declares minimum versions for all dependencies. `uv.lock` pins exact versions for reproducible builds.
- **CI matrix**: The Forgejo CI pipeline tests against Python 3.11, 3.12, and 3.13 to ensure version compatibility.
- **Vulnerability scanning**: Automated dependency vulnerability scanning (e.g., `pip-audit` or `safety`) runs in CI.
- **Build verification**: Docker image builds are tested in CI to catch platform-specific build failures for native extensions.
- **Performance benchmarks**: ASV tracks key performance metrics across commits to detect regressions from dependency changes.