cleveragents/cleveragents-core

Fork 3

UAT: `ProviderRegistry.create_llm()` fails with `ValueError: Unsupported provider type: openrouter` — OpenRouter not handled in `_create_provider_llm` #1892

New issue

Open

opened 2026-04-03 00:08:45 +00:00 by freemo · 2 comments

freemo commented

2026-04-03 00:08:45 +00:00

Owner

Metadata

Branch: fix/provider-registry-openrouter-create-llm
Commit Message: fix(providers): add OpenRouter case to ProviderRegistry._create_provider_llm()
Milestone: v3.7.0
Parent Epic: (see orphan note below)

Background and Context

ProviderRegistry._create_provider_llm() in src/cleveragents/providers/registry.py handles OPENAI, ANTHROPIC, GOOGLE/GEMINI, AZURE, GROQ, TOGETHER, and COHERE providers, but does not handle ProviderType.OPENROUTER. This is a critical inconsistency because:

ProviderType.OPENROUTER is registered in PROVIDER_KEY_ATTRS, FALLBACK_ORDER, DEFAULT_CAPABILITIES, and DEFAULT_MODELS — meaning it is a fully supported provider in every other respect.
create_ai_provider() correctly handles OpenRouter via OpenRouterChatProvider.
But create_llm() → _create_provider_llm() raises ValueError: Unsupported provider type: openrouter.

Current Behavior

Calling registry.create_llm(provider_type='openrouter', ...) (or any path that resolves OpenRouter as the selected provider) raises:

ValueError: Unsupported provider type: openrouter

Steps to Reproduce:

from cleveragents.providers.registry import ProviderRegistry
from cleveragents.config.settings import Settings

settings = Settings()
settings.openai_api_key = None
settings.anthropic_api_key = None
settings.google_api_key = None
settings.gemini_api_key = None
settings.openrouter_api_key = 'sk-test-openrouter'

registry = ProviderRegistry(settings=settings)
# OpenRouter is the only configured provider, so it becomes the default
llm = registry.create_llm()  # Raises ValueError: Unsupported provider type: openrouter

Code Location: src/cleveragents/providers/registry.py, method _create_provider_llm() (around line 455) — the method handles all other providers but is missing the ProviderType.OPENROUTER case.

Expected Behavior

create_llm() should return a ChatOpenAI instance configured with the OpenRouter base URL (https://openrouter.ai/api/v1), consistent with how OpenRouterChatProvider works.

Impact

If OpenRouter is the only configured provider (or the highest-priority configured provider), calling registry.create_llm() will fail at runtime.
LLMStrategizeActor and LLMExecuteActor use registry.create_llm() — if a plan's strategy_actor or execution_actor is set to openrouter/..., the strategize/execute phases will fail entirely.
This is a runtime failure that only manifests when OpenRouter is the selected provider, making it easy to miss in testing with other providers configured.

Proposed Fix

Add an OpenRouter case to _create_provider_llm():

if provider_type == ProviderType.OPENROUTER:
    from langchain_openai import ChatOpenAI
    return ChatOpenAI(
        model=model_id or "anthropic/claude-sonnet-4-20250514",
        openai_api_base="https://openrouter.ai/api/v1",
        openai_api_key=getattr(self._settings, 'openrouter_api_key', None),
        **kwargs,
    )

The existing create_ai_provider() method correctly handles OpenRouter via OpenRouterChatProvider, so the fix should be consistent with that implementation.

Acceptance Criteria

ProviderRegistry._create_provider_llm() handles ProviderType.OPENROUTER without raising ValueError.
registry.create_llm(provider_type='openrouter', model_id='...') returns a ChatOpenAI instance configured with openai_api_base="https://openrouter.ai/api/v1".
When OpenRouter is the only configured provider, registry.create_llm() (no explicit provider) succeeds and returns the OpenRouter-backed LLM.
The fix is consistent with how create_ai_provider() handles OpenRouter via OpenRouterChatProvider.
All existing tests continue to pass.

Subtasks

Add ProviderType.OPENROUTER branch to _create_provider_llm() in src/cleveragents/providers/registry.py
Tests (Behave): Add scenario — create_llm() with OpenRouter as sole configured provider returns ChatOpenAI with correct base URL
Tests (Behave): Add scenario — create_llm(provider_type='openrouter', model_id='...') returns correct LLM instance
Tests (Robot): Add integration test for OpenRouter LLM creation via ProviderRegistry
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (fix(providers): add OpenRouter case to ProviderRegistry._create_provider_llm()), followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (fix/provider-registry-openrouter-create-llm).
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/provider-registry-openrouter-create-llm` - **Commit Message**: `fix(providers): add OpenRouter case to ProviderRegistry._create_provider_llm()` - **Milestone**: v3.7.0 - **Parent Epic**: *(see orphan note below)* ## Background and Context `ProviderRegistry._create_provider_llm()` in `src/cleveragents/providers/registry.py` handles OPENAI, ANTHROPIC, GOOGLE/GEMINI, AZURE, GROQ, TOGETHER, and COHERE providers, but does **not** handle `ProviderType.OPENROUTER`. This is a critical inconsistency because: 1. `ProviderType.OPENROUTER` is registered in `PROVIDER_KEY_ATTRS`, `FALLBACK_ORDER`, `DEFAULT_CAPABILITIES`, and `DEFAULT_MODELS` — meaning it is a fully supported provider in every other respect. 2. `create_ai_provider()` correctly handles OpenRouter via `OpenRouterChatProvider`. 3. But `create_llm()` → `_create_provider_llm()` raises `ValueError: Unsupported provider type: openrouter`. ## Current Behavior Calling `registry.create_llm(provider_type='openrouter', ...)` (or any path that resolves OpenRouter as the selected provider) raises: ``` ValueError: Unsupported provider type: openrouter ``` **Steps to Reproduce:** ```python from cleveragents.providers.registry import ProviderRegistry from cleveragents.config.settings import Settings settings = Settings() settings.openai_api_key = None settings.anthropic_api_key = None settings.google_api_key = None settings.gemini_api_key = None settings.openrouter_api_key = 'sk-test-openrouter' registry = ProviderRegistry(settings=settings) # OpenRouter is the only configured provider, so it becomes the default llm = registry.create_llm() # Raises ValueError: Unsupported provider type: openrouter ``` **Code Location:** `src/cleveragents/providers/registry.py`, method `_create_provider_llm()` (around line 455) — the method handles all other providers but is missing the `ProviderType.OPENROUTER` case. ## Expected Behavior `create_llm()` should return a `ChatOpenAI` instance configured with the OpenRouter base URL (`https://openrouter.ai/api/v1`), consistent with how `OpenRouterChatProvider` works. ## Impact - If OpenRouter is the only configured provider (or the highest-priority configured provider), calling `registry.create_llm()` will fail at runtime. - `LLMStrategizeActor` and `LLMExecuteActor` use `registry.create_llm()` — if a plan's `strategy_actor` or `execution_actor` is set to `openrouter/...`, the strategize/execute phases will fail entirely. - This is a runtime failure that only manifests when OpenRouter is the selected provider, making it easy to miss in testing with other providers configured. ## Proposed Fix Add an OpenRouter case to `_create_provider_llm()`: ```python if provider_type == ProviderType.OPENROUTER: from langchain_openai import ChatOpenAI return ChatOpenAI( model=model_id or "anthropic/claude-sonnet-4-20250514", openai_api_base="https://openrouter.ai/api/v1", openai_api_key=getattr(self._settings, 'openrouter_api_key', None), **kwargs, ) ``` The existing `create_ai_provider()` method correctly handles OpenRouter via `OpenRouterChatProvider`, so the fix should be consistent with that implementation. ## Acceptance Criteria - [ ] `ProviderRegistry._create_provider_llm()` handles `ProviderType.OPENROUTER` without raising `ValueError`. - [ ] `registry.create_llm(provider_type='openrouter', model_id='...')` returns a `ChatOpenAI` instance configured with `openai_api_base="https://openrouter.ai/api/v1"`. - [ ] When OpenRouter is the only configured provider, `registry.create_llm()` (no explicit provider) succeeds and returns the OpenRouter-backed LLM. - [ ] The fix is consistent with how `create_ai_provider()` handles OpenRouter via `OpenRouterChatProvider`. - [ ] All existing tests continue to pass. ## Subtasks - [ ] Add `ProviderType.OPENROUTER` branch to `_create_provider_llm()` in `src/cleveragents/providers/registry.py` - [ ] Tests (Behave): Add scenario — `create_llm()` with OpenRouter as sole configured provider returns `ChatOpenAI` with correct base URL - [ ] Tests (Behave): Add scenario — `create_llm(provider_type='openrouter', model_id='...')` returns correct LLM instance - [ ] Tests (Robot): Add integration test for OpenRouter LLM creation via `ProviderRegistry` - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`fix(providers): add OpenRouter case to ProviderRegistry._create_provider_llm()`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`fix/provider-registry-openrouter-create-llm`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.7.0 milestone

2026-04-03 00:09:18 +00:00

freemo added the

labels

2026-04-03 00:10:11 +00:00

freemo commented

2026-04-03 00:10:46 +00:00

Author

Owner

⚠️ Orphan Issue — Manual Parent Epic Linking Required

This issue was created automatically during UAT testing and could not be linked to a parent Epic because no suitable open Epic was found.

Search performed for:

Type/Epic open issues → only #1678 (CI Execution Time Optimization) found, which is unrelated
Type/Legendary open issues → none found
Epics #217 (LangChain/LangGraph Foundation) and #218 (Core LangGraph Workflows) are both closed
No open Epic covering Provider Registry, LLM integration, or Execution Pipeline was found

Action required: A maintainer should either:

Link this issue to an existing open Epic that covers provider/LLM integration (if one exists that was not found), or
Create a new Epic for "Provider Registry Completeness" or "LangChain/LLM Integration Fixes" and link this issue as a child (this issue blocks the parent Epic).

Per CONTRIBUTING.md, the correct dependency direction is: this child issue blocks the parent Epic (the parent cannot be complete until this child is done).

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

⚠️ **Orphan Issue — Manual Parent Epic Linking Required** This issue was created automatically during UAT testing and could not be linked to a parent Epic because no suitable open Epic was found. **Search performed for:** - `Type/Epic` open issues → only #1678 (CI Execution Time Optimization) found, which is unrelated - `Type/Legendary` open issues → none found - Epics #217 (LangChain/LangGraph Foundation) and #218 (Core LangGraph Workflows) are both **closed** - No open Epic covering Provider Registry, LLM integration, or Execution Pipeline was found **Action required:** A maintainer should either: 1. Link this issue to an existing open Epic that covers provider/LLM integration (if one exists that was not found), or 2. Create a new Epic for "Provider Registry Completeness" or "LangChain/LLM Integration Fixes" and link this issue as a child (this issue **blocks** the parent Epic). Per CONTRIBUTING.md, the correct dependency direction is: this child issue **blocks** the parent Epic (the parent cannot be complete until this child is done). --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo referenced this issue

2026-04-03 00:18:37 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 481 — v3.7.0 #1938

freemo removed the

labels

2026-04-03 00:28:02 +00:00

freemo referenced this issue

2026-04-03 04:10:19 +00:00

[Milestone Review] v3.7.0 — Holistic Integration Review (2026-04-03) #2113

freemo commented

2026-04-05 07:23:52 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified ✅
Priority: High (unchanged)
Milestone: v3.7.0 (already assigned)
MoSCoW: Should Have — OpenRouter is a supported provider per the specification's LLM backend requirements. The ProviderRegistry.create_llm() failing for OpenRouter means users cannot use this provider, which is a significant gap in provider support.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified ✅ - **Priority**: High (unchanged) - **Milestone**: v3.7.0 (already assigned) - **MoSCoW**: Should Have — OpenRouter is a supported provider per the specification's LLM backend requirements. The `ProviderRegistry.create_llm()` failing for OpenRouter means users cannot use this provider, which is a significant gap in provider support. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-05 07:24:42 +00:00

[CA-AUTO] Project Owner — project-owner-1 — Session Tracker #2793