- Replace MagicMock() LLM with FakeListLLM (a proper LangChain Runnable)
to avoid TypeError when PromptTemplate.__or__ evaluates the chain expression
- Use patch.object() context manager to mock _chain_with_retry cleanly
- Fix type annotations from lowercase any to typing.Any
- Separate validation response setup from chain mocking for cleaner test flow
ISSUES CLOSED: #10746
Fix duplicate step_impl function names in Behave test steps that caused
only the last-defined step to be registered with Behave, making all
scenarios fail with undefined step errors. Each step now has a unique
function name following the step_given/step_when/step_then convention.
Also fix the step parameter handling: Gherkin passes quoted string
parameters with their surrounding quotes included, so strip quotes from
the response and status parameters before comparison.
Remove the redundant if/else branch in the validation node step that
called the same code path in both branches.
Add CHANGELOG entry for the fix.
ISSUES CLOSED: #10746
Remove the len(all_code) > 10 fallback in the _validate method that
was overriding the LLM validation response. Previously, any code longer
than 10 characters would cause validation to automatically pass regardless
of the LLM's assessment, making the validation check ineffective.
The fix ensures validation status is determined solely by whether the LLM
response contains 'PASS', making the validation meaningful.
A regression test was added to verify that FAIL/REJECTED LLM responses
are properly handled even for long code blocks.
ISSUES CLOSED: #10746
controller-ci-rerun
HAL9000