cleveragents/cleveragents-core

Fork 3

UAT: `LangChainChatProvider` catches all exceptions as generic strings — LangChain rate limit and API errors not translated to `RateLimitError`/`ProviderError` #5817

New issue

Open

opened 2026-04-09 10:09:42 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 10:09:42 +00:00

Owner

Bug Report

Feature Area: LLM Provider Backends
Milestone: v3.6.0
Severity: Priority/Backlog (error handling gap, retry logic may not work correctly)

What Was Tested

Code-level analysis of src/cleveragents/providers/llm/langchain_chat_provider.py and src/cleveragents/core/exceptions.py against the specification's error handling requirements.

Expected Behavior

The codebase defines ProviderError and RateLimitError in src/cleveragents/core/exceptions.py:

class ProviderError(CleverAgentsError):
    """Provider-level error (API failures, auth errors, etc.)"""

class RateLimitError(ProviderError):
    """Rate limit exceeded. Has retry_after attribute."""
    def __init__(self, message: str, retry_after: int | None = None): ...

The retry patterns in src/cleveragents/core/retry_patterns.py are configured to retry on ProviderError and RateLimitError:

"exceptions": (ProviderError, RateLimitError),

LangChain raises provider-specific exceptions (e.g., openai.RateLimitError, anthropic.RateLimitError) that should be caught and translated to CleverAgents' RateLimitError/ProviderError.

Actual Behavior

LangChainChatProvider.generate_changes() (line 140):

except Exception as exc:  # pragma: no cover - defensive path
    ...
    return ProviderResponse(
        changes=[],
        model_used=self._model_id,
        token_count=token_count,
        error_message=str(exc),  # ← Exception swallowed as string
    )

LangChainChatProvider.stream_changes() (line 231):

except Exception:
    if progress_callback:
        progress_callback(100)
    raise  # ← Re-raised as original LangChain exception type

Problems:

generate_changes() catches all exceptions and returns them as error_message strings — LangChain RateLimitError is silently swallowed, preventing retry logic from triggering
stream_changes() re-raises the original LangChain exception type, not RateLimitError — the retry patterns won't match
No translation layer converts LangChain provider exceptions to CleverAgents domain exceptions

Code Location

LangChain provider: src/cleveragents/providers/llm/langchain_chat_provider.py lines 136-152, 231-234
Domain exceptions: src/cleveragents/core/exceptions.py lines 204-227
Retry patterns: src/cleveragents/core/retry_patterns.py lines 126

Impact

Rate limit errors from OpenAI/Anthropic/Google are silently swallowed in generate_changes() — the caller gets an empty ProviderResponse with an error string instead of a retryable exception
The retry_with_exponential_backoff decorator configured for RateLimitError will never trigger for provider rate limits
The FallbackSelector cannot detect rate-limited providers and fall back to alternatives

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area**: LLM Provider Backends **Milestone**: v3.6.0 **Severity**: Priority/Backlog (error handling gap, retry logic may not work correctly) ## What Was Tested Code-level analysis of `src/cleveragents/providers/llm/langchain_chat_provider.py` and `src/cleveragents/core/exceptions.py` against the specification's error handling requirements. ## Expected Behavior The codebase defines `ProviderError` and `RateLimitError` in `src/cleveragents/core/exceptions.py`: ```python class ProviderError(CleverAgentsError): """Provider-level error (API failures, auth errors, etc.)""" class RateLimitError(ProviderError): """Rate limit exceeded. Has retry_after attribute.""" def __init__(self, message: str, retry_after: int | None = None): ... ``` The retry patterns in `src/cleveragents/core/retry_patterns.py` are configured to retry on `ProviderError` and `RateLimitError`: ```python "exceptions": (ProviderError, RateLimitError), ``` LangChain raises provider-specific exceptions (e.g., `openai.RateLimitError`, `anthropic.RateLimitError`) that should be caught and translated to CleverAgents' `RateLimitError`/`ProviderError`. ## Actual Behavior `LangChainChatProvider.generate_changes()` (line 140): ```python except Exception as exc: # pragma: no cover - defensive path ... return ProviderResponse( changes=[], model_used=self._model_id, token_count=token_count, error_message=str(exc), # ← Exception swallowed as string ) ``` `LangChainChatProvider.stream_changes()` (line 231): ```python except Exception: if progress_callback: progress_callback(100) raise # ← Re-raised as original LangChain exception type ``` **Problems:** 1. `generate_changes()` catches all exceptions and returns them as `error_message` strings — LangChain `RateLimitError` is silently swallowed, preventing retry logic from triggering 2. `stream_changes()` re-raises the original LangChain exception type, not `RateLimitError` — the retry patterns won't match 3. No translation layer converts LangChain provider exceptions to CleverAgents domain exceptions ## Code Location - **LangChain provider**: `src/cleveragents/providers/llm/langchain_chat_provider.py` lines 136-152, 231-234 - **Domain exceptions**: `src/cleveragents/core/exceptions.py` lines 204-227 - **Retry patterns**: `src/cleveragents/core/retry_patterns.py` lines 126 ## Impact 1. Rate limit errors from OpenAI/Anthropic/Google are silently swallowed in `generate_changes()` — the caller gets an empty `ProviderResponse` with an error string instead of a retryable exception 2. The `retry_with_exponential_backoff` decorator configured for `RateLimitError` will never trigger for provider rate limits 3. The `FallbackSelector` cannot detect rate-limited providers and fall back to alternatives --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 10:18:59 +00:00

HAL9000 added this to the v3.6.0 milestone

2026-04-09 10:26:11 +00:00

HAL9000 commented

2026-04-09 10:32:46 +00:00

Author

Owner

Label compliance fix applied:

Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing labels and/or milestone to bring issue into compliance with CONTRIBUTING.md --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer

HAL9000 added

and removed

labels

2026-04-09 10:36:51 +00:00

HAL9000 referenced this issue

2026-04-09 10:48:15 +00:00

[AUTO-PROJ-OWN] Project Owner Report (Cycle 29) #5788

HAL9000 added