135 lines
7.6 KiB
Gherkin
135 lines
7.6 KiB
Gherkin
Feature: LangChain chat provider coverage
|
|
As a maintainer focused on provider reliability
|
|
I want Behave scenarios that exercise the LangChain chat provider
|
|
So that uncovered lines in langchain_chat_provider.py gain coverage
|
|
|
|
@coverage @langchain
|
|
Scenario: LangChain provider surfaces validation failures as user-facing error
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with a validation failure response
|
|
Then the provider response should contain the generated change data and validation error
|
|
And the progress callback should record the LangChain workflow milestones
|
|
And the graph should be invoked with the supplied project context
|
|
|
|
@coverage @langchain
|
|
Scenario: LangChain provider returns an error response when the graph invocation fails
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes and the graph raises an exception
|
|
Then the provider response should capture the graph failure
|
|
And the progress callback should end at 100 percent even on failure
|
|
|
|
@coverage @langchain @streaming
|
|
Scenario: LangChain provider streams node events for progress updates
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider streams changes with incremental graph events
|
|
Then the provider response should contain the streamed change payload
|
|
And the progress callback should include streaming milestones
|
|
And the LangChain graph stream should receive the thread-aware configuration
|
|
And the provider should report the estimated token usage
|
|
|
|
@coverage @langchain @streaming @api
|
|
Scenario: LangChain provider stream API emits workflow events with fallback token estimation
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider stream API emits workflow events with fallback token estimation
|
|
Then the stream API should emit workflow events with the provider response
|
|
And the progress callback should include streaming milestones
|
|
And the LangChain graph stream should receive the thread-aware configuration
|
|
And the provider response should contain the streamed change payload
|
|
And the provider should report the estimated token usage
|
|
|
|
@coverage @langchain @retry
|
|
Scenario: LangChain provider unwraps nested retry failures
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes and retries exhaust with nested errors
|
|
Then the provider response should capture the nested retry failure
|
|
And the progress callback should end at 100 percent even on failure
|
|
|
|
@coverage @langchain @no_progress
|
|
Scenario: LangChain provider handles invalid state data without a progress callback
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes without a progress callback and the graph returns invalid response data
|
|
Then the provider response should surface the state error without changes
|
|
And the progress callback should not receive updates
|
|
|
|
@coverage @langchain @streaming @api @errors
|
|
Scenario: LangChain provider stream API propagates midstream errors with cleanup
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider stream API raises an error after partial events
|
|
Then the stream API should propagate the streaming error to the caller
|
|
And the stream API progress should include the final failure signal
|
|
|
|
@coverage @langchain @streaming @errors
|
|
Scenario: LangChain provider records partial streaming progress when the stream fails
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider streams changes and the stream raises an exception after unknown node events
|
|
Then the provider response should capture the graph failure
|
|
And the progress callback should capture partial streaming progress before failure
|
|
|
|
|
|
|
|
@coverage @langchain @streaming @fallback
|
|
Scenario: LangChain provider replays workflow milestones when streaming is disabled
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider streams changes without streaming support
|
|
Then the fallback stream should replay workflow milestones with the response
|
|
|
|
@coverage @langchain @openai
|
|
Scenario: LangChain provider records OpenAI callback token usage
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with an OpenAI usage callback
|
|
Then the provider should report the callback-derived usage metrics
|
|
|
|
@coverage @langchain @usage
|
|
Scenario: LangChain provider skips usage logging when no metrics exist
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider logs usage without token or cost data
|
|
Then the provider should skip emitting usage metrics
|
|
|
|
@coverage @langchain @streaming @no_progress
|
|
Scenario: LangChain provider stream API handles non-dict payloads without progress callbacks
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider stream API emits non-dict payloads without a progress callback
|
|
Then the stream API should emit workflow events with the provider response
|
|
And the stream API should preserve non-dict payloads in events
|
|
And the progress callback should not receive updates
|
|
And the LangChain graph stream should receive the thread-aware configuration
|
|
And the provider should report the estimated token usage
|
|
|
|
@coverage @langchain @streaming @no_progress @errors
|
|
Scenario: LangChain provider stream API raises errors without progress callbacks
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider stream API raises an error without a progress callback
|
|
Then the stream API should propagate the streaming error to the caller
|
|
And the progress callback should not receive updates
|
|
|
|
@coverage @langchain @errors
|
|
Scenario: LangChain provider unwraps opaque nested error containers
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with opaque nested error containers
|
|
Then the provider response should expose the opaque nested error message
|
|
|
|
@coverage @langchain @errors @depth
|
|
Scenario: LangChain provider truncates overly deep error chains
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with overly deep error containers
|
|
Then the provider response should include the truncated deep error context
|
|
|
|
@coverage @langchain @errors @exceptions
|
|
Scenario: LangChain provider surfaces exception based errors
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with exception based errors
|
|
Then the provider response should capture the graph failure
|
|
And the progress callback should end at 100 percent even on failure
|
|
|
|
@coverage @langchain @errors @fallback
|
|
Scenario: LangChain provider stringifies opaque error mappings
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with opaque error mappings
|
|
Then the provider response should stringify the opaque error mapping
|
|
|
|
@coverage @langchain @errors @fallback
|
|
Scenario: LangChain provider stringifies numeric error payloads
|
|
Given a LangChain chat provider is configured with a fake LangChain graph
|
|
When the provider generates changes with numeric error payloads
|
|
Then the provider response should capture the graph failure
|