cleveragents-core/docs/adr/ADR-022-langchain-langgraph-integration.md

adr_number, title, status_history, tier, authors, superseded_by, related_adrs, acceptance

adr_number	title	status_history	tier	authors	superseded_by	related_adrs	acceptance
22	LangChain/LangGraph Integration	2026-02-16 Proposed Jeffrey Phillips Freeman 2026-02-16 Accepted Jeffrey Phillips Freeman	4	Jeffrey Phillips Freeman	null	number title relationship 1 Layered Architecture LangChain/LangGraph implementations reside in the Infrastructure Layer number title relationship 3 Dependency Injection LLM providers and LangGraph components are registered through the DI container number title relationship 5 Technical Stack LangChain and LangGraph are core stack choices for LLM orchestration number title relationship 10 Actor and Agent Architecture Actors are implemented as LangGraph StateGraph instances number title relationship 29 Model Context Protocol (MCP) Adoption MCP SDK is part of the LLM runtime stack; MCPToolAdapter bridges MCP tools to LangGraph tool nodes number title relationship 31 Actor Abstraction Definition Actors are implemented as LangGraph StateGraph instances; LangGraph is the actor graph runtime number title relationship 48 Server Application Architecture The server deploys actor StateGraphs to LangGraph Platform and invokes them via RemoteGraph for remote plan execution	votes_for votes_against abstentions voter comment Jeffrey Phillips Freeman <Jeffrey.Freeman@CleverThis.com> LangChain for provider abstraction and LangGraph for stateful orchestration are the best fit for our actor model

Context

CleverAgents must orchestrate LLM interactions across multiple providers (OpenAI, Anthropic, Google, Groq, Together, Cohere, Azure), manage stateful multi-step workflows (plan generation graphs, auto-debug loops), support tool calling, prompt templating, output parsing, and streaming execution. Building these capabilities from scratch would be a massive engineering effort and would not benefit from the rapidly evolving LLM ecosystem. The system needs a provider-agnostic abstraction layer and a stateful workflow orchestration framework.

Decision Drivers

• Must support 6+ LLM providers (OpenAI, Anthropic, Google, Groq, Together, Cohere) without provider-specific code in the application layer
• Actors require stateful multi-step workflows with conditional routing, checkpointing, and streaming execution
• Building provider abstraction, tool calling, prompt templating, and output parsing from scratch is prohibitively expensive
• Actor graphs need checkpointing for resume-after-interruption and decision replay
• Reactive event streams must bridge with synchronous graph execution for concurrent plan coordination

Decision

CleverAgents uses LangChain (>= 0.2.14) as the provider-agnostic LLM abstraction layer and LangGraph (transitive dependency) for stateful workflow orchestration. LangChain provides the BaseLanguageModel protocol for swappable providers, while LangGraph provides StateGraph with conditional edges, checkpointing, and streaming execution. RxPY (>= 3.2.0) bridges reactive event streams with LangGraph state graphs.

Design

LangChain as LLM Abstraction

LangChain provides a unified interface across LLM providers:

• BaseLanguageModel protocol: Enables swapping providers without changing application code. The DI container maps provider configurations to concrete ChatModel implementations.
• Provider packages: langchain-openai, langchain-anthropic, langchain-google-genai, langchain-groq, langchain-together, langchain-cohere. Each provides a ChatModel implementation conforming to the base protocol.
• Embeddings: Provider-agnostic embedding interface for vector generation (default: text-embedding-3-small via OpenAI).
• Output parsing: Structured output parsing for extracting decisions, plans, and tool calls from LLM responses.
• Prompt templates: Jinja2-compatible prompt template rendering with {{ context.* }} variable expansion.
• Tool calling: Standardized tool call protocol across providers.

LangGraph as Workflow Orchestration

Every actor in CleverAgents is a LangGraph StateGraph (ADR-010). LangGraph provides:

• StateGraph: Defines actor graphs as a set of nodes (LLM actors, tool nodes) and edges (including conditional edges for routing).
• Conditional edges: Enable dynamic routing based on graph state — e.g., routing to different implementation actors based on the programming language, or branching based on validation results.
• Checkpointing (MemorySaver): Saves graph state at each step, enabling resume after interruption and providing the actor_state_ref used in decision context snapshots (ADR-007).
• Streaming execution: Supports streaming LLM output to the terminal for real-time user feedback.

Key Workflows

Plan generation graph: load_context → analyze → generate → validate. Used during the Strategize phase to produce the decision tree.

Auto-debug graph: execute_tool → check_result → diagnose_error → fix → re_execute. Used during Execute for automatic error recovery.

Actor composition graphs: Hierarchically composed actors where graph nodes reference other actors by name, enabling recursive delegation.

RxPY Bridge

RxPY reactive streams bridge the gap between LangGraph's synchronous state machine execution and the system's event-driven architecture:

• Subject, BehaviorSubject, ReplaySubject for event publishing.
• Operators (map, filter, flat_map, debounce, throttle, scan) for real-time event routing between actors.
• Stream-to-graph bridging: Events from RxPY streams can trigger graph execution, and graph outputs can be published to streams.
• Backpressure management: RxPY operators manage the rate of event processing when multiple concurrent plans are running.

Provider Registry

The infrastructure layer maintains a provider registry that maps provider names to LangChain provider package implementations. Provider configuration (API keys, base URLs, organization IDs) is loaded from the TOML configuration and environment variables via Pydantic Settings. The DI container exposes provider instances as Factory providers, parameterized by the actor's provider and model fields.

Testing Support

LangChain Community provides FakeListLLM and FakeEmbeddings for testing without real API calls. These are wired into the DI container's test configuration, replacing real providers.

Constraints

• All LLM interactions must go through LangChain's BaseLanguageModel interface. Direct HTTP calls to provider APIs are prohibited.
• Actor graphs must be LangGraph StateGraph instances. No alternative workflow engines.
• Provider API keys are managed exclusively through the configuration system (provider.* config keys or environment variables). Keys must never be hardcoded.
• LangGraph checkpoints are stored via MemorySaver and referenced by decision records. Checkpoint data must be available for decision replay.
• RxPY streams must use backpressure management for concurrent plan execution. Unbounded buffering is prohibited.

Consequences

Positive

• Provider-agnostic abstraction enables switching between OpenAI, Anthropic, Google, and others without code changes.
• LangGraph provides battle-tested stateful workflow orchestration with checkpointing and streaming.
• The LLM ecosystem standardizes on LangChain, ensuring compatibility with new providers and tools as they emerge.
• FakeListLLM enables deterministic testing without API costs or network dependencies.
• RxPY provides sophisticated event routing and backpressure management for concurrent execution.

Negative

• LangChain is a large, rapidly evolving dependency. Breaking changes between versions require migration effort.
• LangGraph's StateGraph API is less flexible than custom workflow engines for non-standard orchestration patterns.
• The RxPY bridge adds complexity at the boundary between reactive streams and synchronous graph execution.

Risks

• LangChain version upgrades may introduce breaking changes in provider interfaces, prompt template APIs, or tool calling protocols.
• LangGraph checkpointing with MemorySaver stores state in memory by default — persistence requires explicit configuration.
• Provider-specific features (e.g., Anthropic's thinking tokens, OpenAI's function calling nuances) may not be fully exposed through the abstraction layer.

Alternatives Considered

Direct provider SDKs (no LangChain) — Would require implementing provider abstraction, tool calling, prompt templating, and output parsing from scratch for each provider. The maintenance burden of tracking API changes across 6+ providers would be significant.

LlamaIndex instead of LangChain — LlamaIndex focuses on RAG and data retrieval rather than general-purpose LLM orchestration. LangChain's broader scope (tool calling, agent patterns, provider abstraction, workflow orchestration) better matches CleverAgents' requirements.

Custom workflow engine instead of LangGraph — Would provide more control but would not benefit from LangGraph's checkpointing, streaming, and conditional routing capabilities. The actor-as-graph model maps directly to LangGraph's StateGraph.

Compliance

• Provider abstraction tests: Tests verify that switching provider configurations produces correct ChatModel instances without code changes.
• Graph execution tests: Tests verify that actor graphs execute correctly with the expected node traversal, conditional routing, and state management.
• Checkpoint tests: Tests verify that LangGraph checkpoints can be saved, loaded, and used for decision replay.
• Mock provider tests: Tests verify that FakeListLLM and FakeEmbeddings produce deterministic results suitable for unit and integration testing.
• RxPY integration tests: Tests verify that reactive streams correctly bridge with LangGraph execution and that backpressure is managed under concurrent load.