BUG-HUNT: [error-handling] Broad exception in LangChainChatProvider.generate_changes #1761

New issue

Open

opened 2026-04-02 23:44:57 +00:00 by freemo · 1 comment

freemo commented

2026-04-02 23:44:57 +00:00

Owner

Metadata

Branch: fix/error-handling-langchain-chat-provider-generate-changes
Commit Message: fix(providers): replace broad Exception catch with specific exceptions and full traceback logging in LangChainChatProvider.generate_changes
Milestone: v3.7.0
Parent Epic: #1669

Background and Context

The generate_changes method in LangChainChatProvider uses a broad except Exception as exc: catch-all handler. Per the project's error-handling standards (CONTRIBUTING.md), exceptions should propagate to the top level where they can be properly logged and handled. Catch-all handlers without re-raising suppress errors, making debugging difficult and violating the fail-fast principle.

Current Behavior

The generate_changes method in src/cleveragents/providers/llm/langchain_chat_provider.py (~line 200) catches all exceptions broadly and silently returns an empty ProviderResponse:

        try:
            with self._usage_tracker(llm) as tracker:
                usage_tracker = cast(Any | None, tracker)
                state = _run_workflow()
        except Exception as exc:  # pragma: no cover - defensive path
            if progress_callback:
                progress_callback(100)

            token_count = self._resolve_token_count(llm, plan, contexts, usage_tracker)
            cost = self._resolve_token_cost(usage_tracker)
            self._log_usage(model=self._model_id, tokens=token_count, cost=cost)
            return ProviderResponse(
                changes=[],
                model_used=self._model_id,
                token_count=token_count,
                error_message=str(exc),
            )

This can swallow any exception — including programming errors, unexpected runtime failures, and third-party library exceptions — and return an empty ProviderResponse with only str(exc) as the error message. The full traceback is lost, making debugging very difficult.

Expected Behavior

The full traceback of the exception should be logged (e.g., via logger.exception(...) or logging.exception(...)).
More specific exceptions should be caught where possible (e.g., LangChain-specific exceptions, network errors, timeout errors) rather than the broad Exception.
If a catch-all is truly necessary as a last resort, it must log the full traceback before returning the error response.
The ProviderResponse error message should include enough detail for diagnosis.

Acceptance Criteria

The broad except Exception is replaced with specific exception types relevant to LangChain workflow execution, or at minimum logs the full traceback via logger.exception(...) before returning.
No exception information is silently discarded — the full traceback is always captured in logs.
The ProviderResponse.error_message includes meaningful diagnostic information.
All existing tests continue to pass.
New BDD scenarios cover the error path to ensure the logging and response behaviour is correct.

Supporting Information

File: src/cleveragents/providers/llm/langchain_chat_provider.py
Function/Class: LangChainChatProvider.generate_changes
Lines: ~200
Severity: Medium — can swallow any exception and return an empty ProviderResponse, making debugging difficult
Likelihood: High — any unexpected error in the workflow will hit this path
Related issues: #1682 (Unhandled exception in from_yaml_file), #1680 (Unhandled exception in ActionConfigSchema.from_yaml), #1722 (ToolResult validation raises generic ValueError)

Subtasks

Identify all specific exception types that _run_workflow() and self._usage_tracker(llm) can raise (LangChain exceptions, network errors, timeouts, etc.)
Replace the broad except Exception with specific exception types where possible
Add logger.exception(...) call to log the full traceback in the exception handler
Ensure ProviderResponse.error_message includes sufficient diagnostic detail
Tests (Behave): Add scenario for exception path in generate_changes verifying traceback is logged and response is correct
Tests (Robot): Add integration test for error path in LangChainChatProvider.generate_changes
Verify coverage >=97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/error-handling-langchain-chat-provider-generate-changes` - **Commit Message**: `fix(providers): replace broad Exception catch with specific exceptions and full traceback logging in LangChainChatProvider.generate_changes` - **Milestone**: v3.7.0 - **Parent Epic**: #1669 ## Background and Context The `generate_changes` method in `LangChainChatProvider` uses a broad `except Exception as exc:` catch-all handler. Per the project's error-handling standards (CONTRIBUTING.md), exceptions should propagate to the top level where they can be properly logged and handled. Catch-all handlers without re-raising suppress errors, making debugging difficult and violating the fail-fast principle. ## Current Behavior The `generate_changes` method in `src/cleveragents/providers/llm/langchain_chat_provider.py` (~line 200) catches all exceptions broadly and silently returns an empty `ProviderResponse`: ```python try: with self._usage_tracker(llm) as tracker: usage_tracker = cast(Any | None, tracker) state = _run_workflow() except Exception as exc: # pragma: no cover - defensive path if progress_callback: progress_callback(100) token_count = self._resolve_token_count(llm, plan, contexts, usage_tracker) cost = self._resolve_token_cost(usage_tracker) self._log_usage(model=self._model_id, tokens=token_count, cost=cost) return ProviderResponse( changes=[], model_used=self._model_id, token_count=token_count, error_message=str(exc), ) ``` This can swallow any exception — including programming errors, unexpected runtime failures, and third-party library exceptions — and return an empty `ProviderResponse` with only `str(exc)` as the error message. The full traceback is lost, making debugging very difficult. ## Expected Behavior - The full traceback of the exception should be logged (e.g., via `logger.exception(...)` or `logging.exception(...)`). - More specific exceptions should be caught where possible (e.g., LangChain-specific exceptions, network errors, timeout errors) rather than the broad `Exception`. - If a catch-all is truly necessary as a last resort, it must log the full traceback before returning the error response. - The `ProviderResponse` error message should include enough detail for diagnosis. ## Acceptance Criteria - [ ] The broad `except Exception` is replaced with specific exception types relevant to LangChain workflow execution, or at minimum logs the full traceback via `logger.exception(...)` before returning. - [ ] No exception information is silently discarded — the full traceback is always captured in logs. - [ ] The `ProviderResponse.error_message` includes meaningful diagnostic information. - [ ] All existing tests continue to pass. - [ ] New BDD scenarios cover the error path to ensure the logging and response behaviour is correct. ## Supporting Information - **File**: `src/cleveragents/providers/llm/langchain_chat_provider.py` - **Function/Class**: `LangChainChatProvider.generate_changes` - **Lines**: ~200 - **Severity**: Medium — can swallow any exception and return an empty `ProviderResponse`, making debugging difficult - **Likelihood**: High — any unexpected error in the workflow will hit this path - Related issues: #1682 (Unhandled exception in `from_yaml_file`), #1680 (Unhandled exception in `ActionConfigSchema.from_yaml`), #1722 (ToolResult validation raises generic ValueError) ## Subtasks - [ ] Identify all specific exception types that `_run_workflow()` and `self._usage_tracker(llm)` can raise (LangChain exceptions, network errors, timeouts, etc.) - [ ] Replace the broad `except Exception` with specific exception types where possible - [ ] Add `logger.exception(...)` call to log the full traceback in the exception handler - [ ] Ensure `ProviderResponse.error_message` includes sufficient diagnostic detail - [ ] Tests (Behave): Add scenario for exception path in `generate_changes` verifying traceback is logged and response is correct - [ ] Tests (Robot): Add integration test for error path in `LangChainChatProvider.generate_changes` - [ ] Verify coverage >=97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-02 23:45:15 +00:00

freemo added a new dependency

2026-04-02 23:50:52 +00:00

#1669 Bug Hunting Session

freemo commented

2026-04-02 23:51:12 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: MoSCoW/Should Have — bug or spec compliance issue.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Should Have — bug or spec compliance issue. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-02 23:55:09 +00:00

BUG-HUNT: [error-handling] Broad exception in _handle_file_search #1813

freemo referenced this issue

2026-04-05 02:40:05 +00:00

BUG-HUNT: [error-handling] Broad exception handling in LangChainChatProvider #2884