Files
cleveragents-core/features/langchain_chat_provider_coverage.feature
T

135 lines
7.6 KiB
Gherkin

Feature: LangChain chat provider coverage
As a maintainer focused on provider reliability
I want Behave scenarios that exercise the LangChain chat provider
So that uncovered lines in langchain_chat_provider.py gain coverage
@coverage @langchain
Scenario: LangChain provider surfaces validation failures as user-facing error
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with a validation failure response
Then the provider response should contain the generated change data and validation error
And the progress callback should record the LangChain workflow milestones
And the graph should be invoked with the supplied project context
@coverage @langchain
Scenario: LangChain provider returns an error response when the graph invocation fails
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes and the graph raises an exception
Then the provider response should capture the graph failure
And the progress callback should end at 100 percent even on failure
@coverage @langchain @streaming
Scenario: LangChain provider streams node events for progress updates
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider streams changes with incremental graph events
Then the provider response should contain the streamed change payload
And the progress callback should include streaming milestones
And the LangChain graph stream should receive the thread-aware configuration
And the provider should report the estimated token usage
@coverage @langchain @streaming @api
Scenario: LangChain provider stream API emits workflow events with fallback token estimation
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider stream API emits workflow events with fallback token estimation
Then the stream API should emit workflow events with the provider response
And the progress callback should include streaming milestones
And the LangChain graph stream should receive the thread-aware configuration
And the provider response should contain the streamed change payload
And the provider should report the estimated token usage
@coverage @langchain @retry
Scenario: LangChain provider unwraps nested retry failures
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes and retries exhaust with nested errors
Then the provider response should capture the nested retry failure
And the progress callback should end at 100 percent even on failure
@coverage @langchain @no_progress
Scenario: LangChain provider handles invalid state data without a progress callback
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes without a progress callback and the graph returns invalid response data
Then the provider response should surface the state error without changes
And the progress callback should not receive updates
@coverage @langchain @streaming @api @errors
Scenario: LangChain provider stream API propagates midstream errors with cleanup
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider stream API raises an error after partial events
Then the stream API should propagate the streaming error to the caller
And the stream API progress should include the final failure signal
@coverage @langchain @streaming @errors
Scenario: LangChain provider records partial streaming progress when the stream fails
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider streams changes and the stream raises an exception after unknown node events
Then the provider response should capture the graph failure
And the progress callback should capture partial streaming progress before failure
@coverage @langchain @streaming @fallback
Scenario: LangChain provider replays workflow milestones when streaming is disabled
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider streams changes without streaming support
Then the fallback stream should replay workflow milestones with the response
@coverage @langchain @openai
Scenario: LangChain provider records OpenAI callback token usage
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with an OpenAI usage callback
Then the provider should report the callback-derived usage metrics
@coverage @langchain @usage
Scenario: LangChain provider skips usage logging when no metrics exist
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider logs usage without token or cost data
Then the provider should skip emitting usage metrics
@coverage @langchain @streaming @no_progress
Scenario: LangChain provider stream API handles non-dict payloads without progress callbacks
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider stream API emits non-dict payloads without a progress callback
Then the stream API should emit workflow events with the provider response
And the stream API should preserve non-dict payloads in events
And the progress callback should not receive updates
And the LangChain graph stream should receive the thread-aware configuration
And the provider should report the estimated token usage
@coverage @langchain @streaming @no_progress @errors
Scenario: LangChain provider stream API raises errors without progress callbacks
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider stream API raises an error without a progress callback
Then the stream API should propagate the streaming error to the caller
And the progress callback should not receive updates
@coverage @langchain @errors
Scenario: LangChain provider unwraps opaque nested error containers
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with opaque nested error containers
Then the provider response should expose the opaque nested error message
@coverage @langchain @errors @depth
Scenario: LangChain provider truncates overly deep error chains
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with overly deep error containers
Then the provider response should include the truncated deep error context
@coverage @langchain @errors @exceptions
Scenario: LangChain provider surfaces exception based errors
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with exception based errors
Then the provider response should capture the graph failure
And the progress callback should end at 100 percent even on failure
@coverage @langchain @errors @fallback
Scenario: LangChain provider stringifies opaque error mappings
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with opaque error mappings
Then the provider response should stringify the opaque error mapping
@coverage @langchain @errors @fallback
Scenario: LangChain provider stringifies numeric error payloads
Given a LangChain chat provider is configured with a fake LangChain graph
When the provider generates changes with numeric error payloads
Then the provider response should capture the graph failure